diff --git a/drafts/2017-04-20-modularity.org b/drafts/2017-04-20-modularity.org index b391830..7e1bbac 100644 --- a/drafts/2017-04-20-modularity.org +++ b/drafts/2017-04-20-modularity.org @@ -121,6 +121,35 @@ some details implies that a module is handling those details, that the details shouldn't matter, and what does matter is the abstraction one is using. +In a practical sense: Where someone "factors out" something that +occurs in similar or identical form in multiple places ("decouples" +also works fine as a term), they're often creating a module (from what +was factored out) and some number of abstractions (from the break that +created). Consider some examples: +- Some configurable functionality in a larger application is extracted + out into a system of plugins. The details of the application are + abstracted over (as far as the plugin cares), and the details of the + plugin are abstracted over (as far as the application cares). The + API that the application and plugins use to communicate is the new + abstraction now available. The plugins are modules, and the + application itself is a module of a different sort. (Witness that + sometimes another application will implement the same plugin API.) + +"Composition" is also important. The ability of modules to be +*composed* together is a big part of their usefulness. The ability to +*decompose* something is often a requirement to create useful modules. +([[https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/][Composition over convention]] talks about this in an important take on +why /frameworks/ can also run completely counter to modularity.) + +Almost any resource on software engineering will use many other terms +for related concepts and approaches. To name a few: + +- [[https://en.wikipedia.org/wiki/SOLID][SOLID]] +- [[https://en.wikipedia.org/wiki/Cross-cutting_concern][Cross-cutting concerns]] and [[https://en.wikipedia.org/wiki/Aspect-oriented_programming][Aspect-oriented programming]] +- [[https://en.wikipedia.org/wiki/Separation_of_concerns][Separation of Concerns]] +- [[https://en.wikipedia.org/wiki/Abstraction_principle_(computer_programming)][Abstraction principle]] +- [[https://en.wikipedia.org/wiki/Don%2527t_repeat_yourself][Don't repeat yourself]] + # ----- Consider the information this module deals in, in essence. @@ -133,29 +162,16 @@ contexts? # ----- -In a practical sense: Where someone "factors out" something that -occurs in similar or identical form in multiple places (incidentally, -"decouples" also works fine as a term), they're often creating a -module (from what was factored out) and some number of abstractions -(from the break that created). Consider some examples: -- Some configurable functionality in a larger application is extracted - out into a system of plugins. The details of the application are - abstracted over (as far as the plugin cares), and the details of the - plugin are abstracted over (as far as the application cares). The - API that the application and plugins use to communicate is the new - abstraction now available. The plugins are modules, and the - application itself is a module of a different sort. (Witness that - sometimes another application will implement the same plugin API.) - It has a very pragmatic reason behind it: When something is a module unto itself, presumably it is relying on specific abstractions, and it is possible to freely change this module's internal details (provided -that it still handles the same abstractions), to move this module to -other contexts (anything providing the same abstractions), to replace -it with other modules (anything using the same abstractions). +that it still respects the same abstractions), to move this module to +other contexts (anywhere that provides the same abstractions), and to +replace it with other modules (anything that respects the same +abstractions). It also has a more abstract reason: When something is a module unto -itself, the way it is designed and implemented often presents more +itself, the way it is designed and implemented usually presents more insight into the fundamentals of the problem it is solving. It contains fewer incidental details, and more essential details. @@ -168,9 +184,9 @@ emerge (or, perhaps, old ones become more widespread) to solve problems that I wasn't even aware existed. [[https://circleci.com/blog/it-really-is-the-future/][It really is the future]] talks about a lot of more recent forms of -modularity, most of which are beyond me and were completely unheard-of -in, say, 2010. [[https://www.functionalgeekery.com/episode-75-eric-b-merritt/][Functional Geekery episode 75]] talks about many similar -things. +modularity from the land of devops, most of which were completely +unheard-of in, say, 2010. [[https://www.functionalgeekery.com/episode-75-eric-b-merritt/][Functional Geekery episode 75]] talks about +many similar things. [[https://jupyter.org/][Jupyter Notebook]] is one of my favorites here. It provides a notebook interface (similar to something like Maple or Mathematica) which: @@ -187,19 +203,20 @@ I love notebook interfaces already because they simplify experimenting by handling a lot of things I'd otherwise have to do manually - like saving results and keeping them lined up with the exact code that produced them. Jupyter adds some other use-cases I find marvelous - -for instance, I can let the interpreter run on my much faster -workstation, but I can access it across the Internet from my much -slower laptop. +for instance, I can let the interpreter run on my workstation which +has all of the computing power, but I can access it across the +Internet from my laptop. -[[https://zeppelin.apache.org/][Apache Zeppelin]] does similar things with different languages; I just -use it less. +[[https://zeppelin.apache.org/][Apache Zeppelin]] does similar things with different languages; I've +just used it much less. Another favorite of mine is [[https://nixos.org/nix/][Nix]]. One excellent article, [[http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/][The fundamental problem of programming language package management]], -doesn't ever mention Nix but does a great job explaining the sorts of -problems it exists to solve. To be able to combine nearly all of the +doesn't ever mention Nix but explains very well the problems it sets +out to solve. To be able to combine nearly all of the programming-language specific package managers into a single module is -a very lofty goal, but Nix appears to do a decent job of it. +a very lofty goal, but Nix appears to do a decent job of it (among +other things). The [[https://www.lua.org/][Lua]] programming language is noteworthy here. It's written in clean C with minimal dependencies, so it runs nearly anywhere that a C @@ -218,7 +235,7 @@ particularly if you allow dead ones created during the object-oriented hype of the '90s. This seems to happen in systems where the object hierarchy is in effect "bigger" than the language. -ZeroMQ is another example as a set of cross-language abstractions for +[[https://zeromq.org/][ZeroMQ]] is another example: a set of cross-language abstractions for communication patterns in a distributed system. I know it's likely not unique, but it is one of the better-known and the first I thought of, and I think their [[http://zguide.zeromq.org/page:all][guide]] is excellent. @@ -235,7 +252,8 @@ programming language. [[https://web.hypothes.is/][hypothes.is]] is a curious one that I find fascinating. They're trying to factor out annotation and commenting from something that is handled on a per-webpage basis and turn it into its own module, and I really -like what I've seen. +like what I've seen. However, it does not seem to have caught on +much. The Unix tradition lives on in certain modern tools. [[https://stedolan.github.io/jq/][jq]] has proven very useful anytime I've had to mess with JSON data. [[http://www.dest-unreach.org/socat/][socat]] and [[http://netcat.sourceforge.net/][netcat]] @@ -252,7 +270,7 @@ mention... # Also, caches - of all types. (CPU, disk...) # One key is how the above let you *reason* about things without -# knowing their specifics.a +# knowing their specifics. People know that I love Emacs, but I also do believe many of the complaints on how large it is. Despite that it is basically its own @@ -267,25 +285,21 @@ machine learning conflicts with proper modularity and abstraction. a good post and shows some sorts of abstraction that still exist at least in neural networks.) -Even DOS presented useful abstractions. Things like +Even DOS had useful abstractions. Things like DriveSpace/DoubleSpace/Stacker worked well enough because most software that needed files relied on DOS's normal abstractions to access them - so it did not matter to them that the underlying -filesystem was actually compressed, or in a RAM disk or whatever. -Likewise, for the silliness known as [[https://en.wikipedia.org/wiki/Expanded_memory][EMS]], applications that accessed -memory through the EMS abstraction could disregard whether it was a -"real" EMS board providing access to that memory, whether it was an -expanded memory manager providing indirect access to some other memory -or even to a hard disk pretending to be memory. +filesystem was actually compressed, or was actually a RAM disk, or was +on some obscure SCSI interface. Likewise, for the silliness known as +[[https://en.wikipedia.org/wiki/Expanded_memory][EMS]], applications that accessed memory through the EMS abstraction +could disregard whether it was a "real" EMS board providing access to +that memory, whether it was an expanded memory manager providing +indirect access to some other memory or even to a hard disk pretending +to be memory. Even more abstractly: emulators work because so much software respected the abstraction of some specific CPU and hardware platform. -# Does the below belong here? that isn't really an example. - -[[https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/][Composition over convention]] is an important read on why /frameworks/ -can also run completely counter to modularity. - Submitted without further comment: https://github.com/stevemao/left-pad/issues/4 @@ -298,9 +312,6 @@ https://github.com/stevemao/left-pad/issues/4 - Multiple hosts - [[Notes - Paper, 2016-11-13]] -- Any Plan 9 papers? (Will have to dig deep in the archives) - - http://plan9.bell-labs.com/sys/doc/ - - Link is now down - Tanenbaum vs. Linus war & microkernels - TBL: "The choice of language is a common design choice. The low power end of the scale is typically simpler to design, implement and @@ -334,11 +345,6 @@ https://github.com/stevemao/left-pad/issues/4 - https://twitter.com/fchollet/status/962074070513631232 -- How does this fit with /composition/? Does it? - - The ability to sensibly compose things depends on them having some - sort of well-defined, compatible boundary - right? - - Note also /decomposition/ here, as in /decomposing/ something into - parts. - [[https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-9.html#%25_chap_1][From SICP chapter 1 intro]]: "The acts of the mind, wherein it exerts its power over simple ideas, are chiefly these three: 1. Combining several simple ideas into one compound one, and thus all complex @@ -349,10 +355,6 @@ https://github.com/stevemao/left-pad/issues/4 from all other ideas that accompany them in their real existence: this is called abstraction, and thus all its general ideas are made." -John Locke, An Essay Concerning Human Understanding (1690) -- [[https://en.wikipedia.org/wiki/Cross-cutting_concern][Cross-cutting concerns]], [[https://en.wikipedia.org/wiki/Aspect-oriented_programming][aspect-oriented programming]] (as an attempt - to take tangled things and pull them into modules) - - [[https://en.wikipedia.org/wiki/Separation_of_concerns][Separation of Concerns]] -- Abstraction as an information channel... module as a what? - One point I have ignored (maybe): You clearly separate the 'inside' of a module (its implementation) from the 'outside' (that is - its boundaries, the abstractions that it interfaces with or that it @@ -364,9 +366,6 @@ https://github.com/stevemao/left-pad/issues/4 other modules do not need to change to conform) - What is more key? Communication, information content, contracts, details? -- [[https://en.wikipedia.org/wiki/Abstraction_principle_(computer_programming)][Abstraction principle]] - - Reduce duplication of information - - [[https://en.wikipedia.org/wiki/Don%2527t_repeat_yourself][Don't repeat yourself]] - [[https://simplyphilosophy.org/study/aristotles-definitions/][Aristotle & theory of definitions]] - this isn't right. I need to find the quote in the Durant book (which will probably have an actual source) that pertains to how