hodapp/blag
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

More refactoring in modularity post

Browse Source
This commit is contained in:
hodapp 2020-01-01 15:37:20 -05:00
parent c5e4134782
commit 76a33703bf
1 changed files with 58 additions and 59 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

117
drafts/2017-04-20-modularity.org
View File

@ -121,6 +121,35 @@ some details implies that a module is handling those details, that the
details shouldn't matter, and what does matter is the abstraction one details shouldn't matter, and what does matter is the abstraction one
is using. is using.
In a practical sense: Where someone "factors out" something that
occurs in similar or identical form in multiple places ("decouples"
also works fine as a term), they're often creating a module (from what
was factored out) and some number of abstractions (from the break that
created). Consider some examples:
- Some configurable functionality in a larger application is extracted
out into a system of plugins. The details of the application are
abstracted over (as far as the plugin cares), and the details of the
plugin are abstracted over (as far as the application cares). The
API that the application and plugins use to communicate is the new
abstraction now available. The plugins are modules, and the
application itself is a module of a different sort. (Witness that
sometimes another application will implement the same plugin API.)
"Composition" is also important. The ability of modules to be
*composed* together is a big part of their usefulness. The ability to
*decompose* something is often a requirement to create useful modules.
([[https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/][Composition over convention]] talks about this in an important take on
why /frameworks/ can also run completely counter to modularity.)
Almost any resource on software engineering will use many other terms
for related concepts and approaches. To name a few:
- [[https://en.wikipedia.org/wiki/SOLID][SOLID]]
- [[https://en.wikipedia.org/wiki/Cross-cutting_concern][Cross-cutting concerns]] and [[https://en.wikipedia.org/wiki/Aspect-oriented_programming][Aspect-oriented programming]]
- [[https://en.wikipedia.org/wiki/Separation_of_concerns][Separation of Concerns]]
- [[https://en.wikipedia.org/wiki/Abstraction_principle_(computer_programming)][Abstraction principle]]
- [[https://en.wikipedia.org/wiki/Don%2527t_repeat_yourself][Don't repeat yourself]]
# ----- # -----
Consider the information this module deals in, in essence. Consider the information this module deals in, in essence.
@ -133,29 +162,16 @@ contexts?
# ----- # -----
In a practical sense: Where someone "factors out" something that
occurs in similar or identical form in multiple places (incidentally,
"decouples" also works fine as a term), they're often creating a
module (from what was factored out) and some number of abstractions
(from the break that created). Consider some examples:
- Some configurable functionality in a larger application is extracted
out into a system of plugins. The details of the application are
abstracted over (as far as the plugin cares), and the details of the
plugin are abstracted over (as far as the application cares). The
API that the application and plugins use to communicate is the new
abstraction now available. The plugins are modules, and the
application itself is a module of a different sort. (Witness that
sometimes another application will implement the same plugin API.)
It has a very pragmatic reason behind it: When something is a module It has a very pragmatic reason behind it: When something is a module
unto itself, presumably it is relying on specific abstractions, and it unto itself, presumably it is relying on specific abstractions, and it
is possible to freely change this module's internal details (provided is possible to freely change this module's internal details (provided
that it still handles the same abstractions), to move this module to that it still respects the same abstractions), to move this module to
other contexts (anything providing the same abstractions), to replace other contexts (anywhere that provides the same abstractions), and to
it with other modules (anything using the same abstractions). replace it with other modules (anything that respects the same
abstractions).
It also has a more abstract reason: When something is a module unto It also has a more abstract reason: When something is a module unto
itself, the way it is designed and implemented often presents more itself, the way it is designed and implemented usually presents more
insight into the fundamentals of the problem it is solving. It insight into the fundamentals of the problem it is solving. It
contains fewer incidental details, and more essential details. contains fewer incidental details, and more essential details.
@ -168,9 +184,9 @@ emerge (or, perhaps, old ones become more widespread) to solve
problems that I wasn't even aware existed. problems that I wasn't even aware existed.
[[https://circleci.com/blog/it-really-is-the-future/][It really is the future]] talks about a lot of more recent forms of [[https://circleci.com/blog/it-really-is-the-future/][It really is the future]] talks about a lot of more recent forms of
modularity, most of which are beyond me and were completely unheard-of modularity from the land of devops, most of which were completely
in, say, 2010. [[https://www.functionalgeekery.com/episode-75-eric-b-merritt/][Functional Geekery episode 75]] talks about many similar unheard-of in, say, 2010. [[https://www.functionalgeekery.com/episode-75-eric-b-merritt/][Functional Geekery episode 75]] talks about
things. many similar things.
[[https://jupyter.org/][Jupyter Notebook]] is one of my favorites here. It provides a notebook [[https://jupyter.org/][Jupyter Notebook]] is one of my favorites here. It provides a notebook
interface (similar to something like Maple or Mathematica) which: interface (similar to something like Maple or Mathematica) which:
@ -187,19 +203,20 @@ I love notebook interfaces already because they simplify experimenting
by handling a lot of things I'd otherwise have to do manually - like by handling a lot of things I'd otherwise have to do manually - like
saving results and keeping them lined up with the exact code that saving results and keeping them lined up with the exact code that
produced them. Jupyter adds some other use-cases I find marvelous - produced them. Jupyter adds some other use-cases I find marvelous -
for instance, I can let the interpreter run on my much faster for instance, I can let the interpreter run on my workstation which
workstation, but I can access it across the Internet from my much has all of the computing power, but I can access it across the
slower laptop. Internet from my laptop.
[[https://zeppelin.apache.org/][Apache Zeppelin]] does similar things with different languages; I just [[https://zeppelin.apache.org/][Apache Zeppelin]] does similar things with different languages; I've
use it less. just used it much less.
Another favorite of mine is [[https://nixos.org/nix/][Nix]]. One excellent article, [[http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/][The Another favorite of mine is [[https://nixos.org/nix/][Nix]]. One excellent article, [[http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/][The
fundamental problem of programming language package management]], fundamental problem of programming language package management]],
doesn't ever mention Nix but does a great job explaining the sorts of doesn't ever mention Nix but explains very well the problems it sets
problems it exists to solve. To be able to combine nearly all of the out to solve. To be able to combine nearly all of the
programming-language specific package managers into a single module is programming-language specific package managers into a single module is
a very lofty goal, but Nix appears to do a decent job of it. a very lofty goal, but Nix appears to do a decent job of it (among
other things).
The [[https://www.lua.org/][Lua]] programming language is noteworthy here. It's written in The [[https://www.lua.org/][Lua]] programming language is noteworthy here. It's written in
clean C with minimal dependencies, so it runs nearly anywhere that a C clean C with minimal dependencies, so it runs nearly anywhere that a C
@ -218,7 +235,7 @@ particularly if you allow dead ones created during the object-oriented
hype of the '90s. This seems to happen in systems where the object hype of the '90s. This seems to happen in systems where the object
hierarchy is in effect "bigger" than the language. hierarchy is in effect "bigger" than the language.
ZeroMQ is another example as a set of cross-language abstractions for [[https://zeromq.org/][ZeroMQ]] is another example: a set of cross-language abstractions for
communication patterns in a distributed system. I know it's likely communication patterns in a distributed system. I know it's likely
not unique, but it is one of the better-known and the first I thought not unique, but it is one of the better-known and the first I thought
of, and I think their [[http://zguide.zeromq.org/page:all][guide]] is excellent. of, and I think their [[http://zguide.zeromq.org/page:all][guide]] is excellent.
@ -235,7 +252,8 @@ programming language.
[[https://web.hypothes.is/][hypothes.is]] is a curious one that I find fascinating. They're trying [[https://web.hypothes.is/][hypothes.is]] is a curious one that I find fascinating. They're trying
to factor out annotation and commenting from something that is handled to factor out annotation and commenting from something that is handled
on a per-webpage basis and turn it into its own module, and I really on a per-webpage basis and turn it into its own module, and I really
like what I've seen. like what I've seen. However, it does not seem to have caught on
much.
The Unix tradition lives on in certain modern tools. [[https://stedolan.github.io/jq/][jq]] has proven The Unix tradition lives on in certain modern tools. [[https://stedolan.github.io/jq/][jq]] has proven
very useful anytime I've had to mess with JSON data. [[http://www.dest-unreach.org/socat/][socat]] and [[http://netcat.sourceforge.net/][netcat]] very useful anytime I've had to mess with JSON data. [[http://www.dest-unreach.org/socat/][socat]] and [[http://netcat.sourceforge.net/][netcat]]
@ -252,7 +270,7 @@ mention...
# Also, caches - of all types. (CPU, disk...) # Also, caches - of all types. (CPU, disk...)
# One key is how the above let you *reason* about things without # One key is how the above let you *reason* about things without
# knowing their specifics.a # knowing their specifics.
People know that I love Emacs, but I also do believe many of the People know that I love Emacs, but I also do believe many of the
complaints on how large it is. Despite that it is basically its own complaints on how large it is. Despite that it is basically its own
@ -267,25 +285,21 @@ machine learning conflicts with proper modularity and abstraction.
a good post and shows some sorts of abstraction that still exist a good post and shows some sorts of abstraction that still exist
at least in neural networks.) at least in neural networks.)
Even DOS presented useful abstractions. Things like Even DOS had useful abstractions. Things like
DriveSpace/DoubleSpace/Stacker worked well enough because most DriveSpace/DoubleSpace/Stacker worked well enough because most
software that needed files relied on DOS's normal abstractions to software that needed files relied on DOS's normal abstractions to
access them - so it did not matter to them that the underlying access them - so it did not matter to them that the underlying
filesystem was actually compressed, or in a RAM disk or whatever. filesystem was actually compressed, or was actually a RAM disk, or was
Likewise, for the silliness known as [[https://en.wikipedia.org/wiki/Expanded_memory][EMS]], applications that accessed on some obscure SCSI interface. Likewise, for the silliness known as
memory through the EMS abstraction could disregard whether it was a [[https://en.wikipedia.org/wiki/Expanded_memory][EMS]], applications that accessed memory through the EMS abstraction
"real" EMS board providing access to that memory, whether it was an could disregard whether it was a "real" EMS board providing access to
expanded memory manager providing indirect access to some other memory that memory, whether it was an expanded memory manager providing
or even to a hard disk pretending to be memory. indirect access to some other memory or even to a hard disk pretending
to be memory.
Even more abstractly: emulators work because so much software Even more abstractly: emulators work because so much software
respected the abstraction of some specific CPU and hardware platform. respected the abstraction of some specific CPU and hardware platform.
# Does the below belong here? that isn't really an example.
[[https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/][Composition over convention]] is an important read on why /frameworks/
can also run completely counter to modularity.
Submitted without further comment: Submitted without further comment:
https://github.com/stevemao/left-pad/issues/4 https://github.com/stevemao/left-pad/issues/4
@ -298,9 +312,6 @@ https://github.com/stevemao/left-pad/issues/4
- Multiple hosts - Multiple hosts
- [[Notes - Paper, 2016-11-13]] - [[Notes - Paper, 2016-11-13]]
- Any Plan 9 papers? (Will have to dig deep in the archives)
- http://plan9.bell-labs.com/sys/doc/
- Link is now down
- Tanenbaum vs. Linus war & microkernels - Tanenbaum vs. Linus war & microkernels
- TBL: "The choice of language is a common design choice. The low - TBL: "The choice of language is a common design choice. The low
power end of the scale is typically simpler to design, implement and power end of the scale is typically simpler to design, implement and
@ -334,11 +345,6 @@ https://github.com/stevemao/left-pad/issues/4
- https://twitter.com/fchollet/status/962074070513631232 - https://twitter.com/fchollet/status/962074070513631232
- How does this fit with /composition/? Does it?
- The ability to sensibly compose things depends on them having some
sort of well-defined, compatible boundary - right?
- Note also /decomposition/ here, as in /decomposing/ something into
parts.
- [[https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-9.html#%25_chap_1][From SICP chapter 1 intro]]: "The acts of the mind, wherein it exerts - [[https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-9.html#%25_chap_1][From SICP chapter 1 intro]]: "The acts of the mind, wherein it exerts
its power over simple ideas, are chiefly these three: 1. Combining its power over simple ideas, are chiefly these three: 1. Combining
several simple ideas into one compound one, and thus all complex several simple ideas into one compound one, and thus all complex
@ -349,10 +355,6 @@ https://github.com/stevemao/left-pad/issues/4
from all other ideas that accompany them in their real existence: from all other ideas that accompany them in their real existence:
this is called abstraction, and thus all its general ideas are this is called abstraction, and thus all its general ideas are
made." -John Locke, An Essay Concerning Human Understanding (1690) made." -John Locke, An Essay Concerning Human Understanding (1690)
- [[https://en.wikipedia.org/wiki/Cross-cutting_concern][Cross-cutting concerns]], [[https://en.wikipedia.org/wiki/Aspect-oriented_programming][aspect-oriented programming]] (as an attempt
to take tangled things and pull them into modules)
- [[https://en.wikipedia.org/wiki/Separation_of_concerns][Separation of Concerns]]
- Abstraction as an information channel... module as a what?
- One point I have ignored (maybe): You clearly separate the 'inside' - One point I have ignored (maybe): You clearly separate the 'inside'
of a module (its implementation) from the 'outside' (that is - its of a module (its implementation) from the 'outside' (that is - its
boundaries, the abstractions that it interfaces with or that it boundaries, the abstractions that it interfaces with or that it
@ -364,9 +366,6 @@ https://github.com/stevemao/left-pad/issues/4
other modules do not need to change to conform) other modules do not need to change to conform)
- What is more key? Communication, information content, contracts, - What is more key? Communication, information content, contracts,
details? details?
- [[https://en.wikipedia.org/wiki/Abstraction_principle_(computer_programming)][Abstraction principle]]
- Reduce duplication of information
- [[https://en.wikipedia.org/wiki/Don%2527t_repeat_yourself][Don't repeat yourself]]
- [[https://simplyphilosophy.org/study/aristotles-definitions/][Aristotle & theory of definitions]] - [[https://simplyphilosophy.org/study/aristotles-definitions/][Aristotle & theory of definitions]]
- this isn't right. I need to find the quote in the Durant book - this isn't right. I need to find the quote in the Durant book
(which will probably have an actual source) that pertains to how (which will probably have an actual source) that pertains to how