From 60cc97f219807442b7ddd5145e1187fa67708fea Mon Sep 17 00:00:00 2001 From: Chris Hodapp Date: Mon, 11 Dec 2017 12:48:59 -0500 Subject: [PATCH] Add some more scraps to modularity post --- drafts/2017-04-20-modularity.org | 197 +++++++++++++++++++++++++++++++ posts/2017-04-20-modularity.md | 40 ------- 2 files changed, 197 insertions(+), 40 deletions(-) create mode 100644 drafts/2017-04-20-modularity.org delete mode 100644 posts/2017-04-20-modularity.md diff --git a/drafts/2017-04-20-modularity.org b/drafts/2017-04-20-modularity.org new file mode 100644 index 0000000..847a0e8 --- /dev/null +++ b/drafts/2017-04-20-modularity.org @@ -0,0 +1,197 @@ +#+TITLE: Modularity & Abstraction (working title) +#+AUTHOR: Chris Hodapp +#+DATE: April 20, 2017 +#+TAGS: technobabble + +Two central concepts that feature prominently anywhere that computers +do are _modularity_ and _abstraction_. This is meant very broadly: it +applies to designing software, using software, integrating software, +and to a lot of hardware as well. It certainly applies elsewhere too, +particularly other fields of engineering, but it appears to be +particularly crucial anywhere software is involved. + +They're generally accepted as desireable, but a bit ill-understood at +times. It's common to find people who treat "abstraction" as +something that always stands in their way, adds overhead, and confuses +things. At the same time, it's common to find people who treat +modularity as being present anytime something is broken into pieces. + +"Being abstract is something profoundly different from being vague. +The purpose of abstraction is not to be vague, but to create a new +semantic level in which one can be absolutely precise." E. W. Dijkstra + +"Modular design hinges on the simplicity and abstract nature of the +interface definition between the modules. A design in which the +insides of each module need to know all about each other is not a +modular design but an arbitrary partitioning of the bits." (Tim +Berners-Lee in [[https://www.w3.org/DesignIssues/Principles.html][Principles of Design]].) + +"Its is not only necessary to make sure your own system is designed to +be made of modular parts. It is also necessary to realize that your +own system, no matter how big and wonderful it seems now, should +always be designed to be a part of another larger system." (Same) + +*Abstraction* and *modularity* are tied other inextricably. This is +because abstractions draw out the boundaries that modules sit inside, +and the interfaces (APIs, communication channels) through which they +talk. It need not necessarily be a standardized interface or a +well-documented boundary, though that helps. + +*Decoupling*, *loose coupling*, and *tight coupling* relate to this as +well. When two things are decoupled, it often suggests something +about at least one of them being a module. + +Available abstractions vary. The language itself may set boundaries +on what abstractions can be created, or practically created (a +language like Haskell contains various abstractions done largely +within the type system that cannot be expressed in many others; +languages like Python, Ruby, or JavaScript might have various +abstractions that are meaningful only in the context of dynamic +typing). Some languages more readily permit the creation of new +abstractions, and this might lead to a broader range of abstractions +implemented in libraries. The operating system and standard library +provide abstractions that may cross languages (what is a process? +what is a thread? what is a library? what is a filesystem?). This +extends into electronics (where agreed-upon protocols permit +interconnectivity and an ignorance of internals that is required for +abstraction) where "modules" take on a tangible form. Abstractions +also change over time - both in specifics (consult any list of dead +protocols and technologies) and in broader classes (consider how we +now have entire abstractions devoted to provisioning virtual +resources, and consider how CGI and FastCGI used to be pretty +widely-used interfaces). + +In a practical sense: Where someone "factors out" something that +occurs in similar or identical form in multiple places (incidentally, +"decouples" also works fine as a term), they're often creating a +module (from what was factored out) and some number of abstractions +(from the break that created). Consider some examples: +- Some configurable functionality in a larger application is extracted + out into a system of plugins. The details of the application are + abstracted over (as far as the plugin cares), and the details of the + plugin are abstracted over (as far as the application cares). The + API that the application and plugins use to communicate is the new + abstraction now available. The plugins are modules, and the + application itself is a module of a different sort. (Witness that + sometimes another application will implement the same plugin API.) + +Given the strong ties between modularity and abstraction, the +possible and sensible ways to modularize things then also vary. + +The modules that make sense may change over time. ("Have modern +tools tried to keep embodying the 'Unix philosophy'? Have they +extended it, even to other forms of abstraction that weren't +previously considered?") + +One reason behind this is more practical in nature: When something is +a module unto itself, presumably it is relying on specific +abstractions, and it is possible to move this module to other contexts +(anything providing the same abstractions) or to replace it with other +modules (anything using the same abstractions). + +Another reason is more abstract: When something it a module unto +itself, the way it is designed and implemented often presents more +insight into the fundamentals of the problem it is solving. It +contains fewer incidental details, and more essential details. + +* Other fluff + +I was around to see what was normal for software made on Windows +3.1, Windows 95, and the like. My take is that most of these pieces +of software were sufficiently GUI-oriented that they tried to remove +most modularity from the user's perspective. Things like scripting +and automation were almost solely as afterthoughts, since most +interaction was designed explicitly around the GUI. + +People know that I love Emacs, but I also do believe many of the +complaints on how large it is. On the one hand, it is basically its +own operating system and within this it has considerable modularity. +On the other hand, there is a perfectly usable operating system +underneath, and this makes me wonder why it needs explicit support for +[[https://www.gnu.org/software/tramp/][network transparency]]. + +* Fragments + +- Abstracting over... + - Multiple applications + - Multiple users + - Multiple CPUs + - Multiple hosts + +- Nix, Guix +- [[Notes - Distributed stuff notes (from turtl)]] +- [[Notes - Paper, 2016-11-13]] +- See notes on functional geekery #75 +- Jupyter +- Any Plan 9 papers? (Will have to dig deep in the archives) + - http://plan9.bell-labs.com/sys/doc/ +- Tanenbaum vs. Linus war & microkernels +- Conjecture: A module is most useful when available in the most + general or most accessible context (e.g. Linux commandline tool + vs. a Wordpress plugin or an Emacs package) - the TBL quote on + least-power sort of corroborates this, but stands separate in some + other ways too. + - "most general" might not be right here. +- Other conjecture attempt: A module's power is related to how many + other modules it can communicate with, without requiring substantial + adaptation. An abstraction's power is related to the modularity it + accomodates. +- Another conjecture attempt: An abstraction's power isn't related to + how broad it is, but to how well it connects things. A needlessly + simplistic abstraction requires a lot of other adaptation to be + useful. A needlessly specific one excludes a lot of potential + modues. +- hypothes.is is a sort of module unto itself here too, trying to + remove commenting and annotation from existing, very siloed + solutions. +- "Self" paper & structural reification? + - I'm still not sure how this relates, but it may perhaps relate to + how *not* to make things modular (structural reification is a sort + of check on the scope of objects/classes) +- What by Rich Hickey? +- SICP? + - https://mitpress.mit.edu/sicp/full-text/sicp/book/node50.html +- "On Understanding Data Abstraction, Revisited" +- Frameworks Don't Compose ([composition][]) +- "On the Criteria to be Used in Decomposing System into Modules" (Barnas) +- suckless, and their tools & methodology + - https://suckless.org/philosophy + - even though they can take things waaaay too far... +- Containers? +- http://www.catb.org/~esr/writings/taoup/html/apb.html#Baldwin-Clark - + Carliss Baldwin and Kim Clark. Design Rules, Vol 1: The Power of + Modularity. 2000. MIT Press. ISBN 0-262-024667. +- https://colah.github.io/posts/2015-09-NN-Types-FP/ - Was this the + one that talked about 'modularity' in deep learning? +- NodeRED might be interesting here, but first I need a clear idea of + what it factored out into a separate component. +- Lua is notable here for the effort spent in making it easy to both + embed (e.g. as a scripting or extension language) and extend + (e.g. with other C libraries). Guile may be similar. +- NeoVim is also an interesting case here as it is designed to be + embedded, though I'm not sure what this means yet. + +- Find the link to Les Hatton's slides (cyclomatic complexity?) on the + empirical effects of too many / too large modules + +- Examples of more 'modern' tools: + - socat + - jq (the JSON processor) + - Nix and some related tools (which take related functionality that + is present in numerous PL-specific package managers) + - Jupyter + +* Link-pile: + +- [[http://www.catb.org/~esr/writings/taoup/html/][The Art of Unix Programming (Eric S. Raymond)]] +- [[https://circleci.com/blog/its-the-future/][It's the Future]] +- [[https://circleci.com/blog/it-really-is-the-future/][It really is the future]] +- [[https://www.youtube.com/watch?v%253Dtc4ROCJYbm0][AT&T Archives: The UNIX Operating System]] +- [[http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/][The fundamental problem of programming language package management]] +- https://www.reddit.com/r/programming/comments/4bjss2/an_11_line_npm_package_called_leftpad_with_only/ +- https://www.functionalgeekery.com/episode-75-eric-b-merritt/ +- https://www.w3.org/DesignIssues/ +- https://www.w3.org/DesignIssues/Modularity.html +- http://www.w3.org/DesignIssues/Principles.html +- http://www.freecode.com/articles/editorial-the-two-edged-sword +- https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/ diff --git a/posts/2017-04-20-modularity.md b/posts/2017-04-20-modularity.md deleted file mode 100644 index 4032eb9..0000000 --- a/posts/2017-04-20-modularity.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -title: Modularity (working title) -author: Chris Hodapp -date: April 20, 2017 -tags: technobabble ---- - -- Nix, Guix -- [[Notes - Distributed stuff notes (from turtl)]] -- [[Notes - Paper, 2016-11-13]] -- See notes on functional geekery #75 -- Jupyter -- Any Plan 9 papers? (Will have to dig deep in the archives) - - http://plan9.bell-labs.com/sys/doc/ -- Tanenbaum vs. Linus war & microkernels -- Abstraction vs. modularity - - tbl_design quote: "Modular design hinges on the simplicity and abstract nature of the interface definition between the modules. A design in which the insides of each module need to know all about each other is not a modular design but an arbitrary partitioning of the bits." -- Modules: Most useful when available in the most general or most accessible context, versus more specific, more restricted contexts? (e.g. Linux commandline tool vs. a Wordpress plugin or an Emacs package) - the TBL quote on least-power sort of corroborates this. -- hypothes.is is a sort of module unto itself here too, trying to remove commenting and annotation from existing, very siloed solutions. -- "Self" paper & structural reification? -- What by Rich Hickey? -- SICP? -- "On Understanding Data Abstraction, Revisited" -- Frameworks Don't Compose ([composition][]) - -- Abstraction enables modularity. Why: Abstraction barriers are what separate modules. -- The modules that make sense may change over time. ("Have modern tools tried to keep embodying the 'Unix philosophy'? Have they extended it, even to other forms of abstraction that weren't previously considered?") - -[taoup]: http://www.catb.org/~esr/writings/taoup/html/ "The Art of Unix Programming (Eric S. Raymond)" -[future]: https://circleci.com/blog/its-the-future/ "It's the Future" -[future2]: https://circleci.com/blog/it-really-is-the-future/ "It really is the future" -[attvideo]: https://www.youtube.com/watch?v%3Dtc4ROCJYbm0 "AT&T Archives: The UNIX Operating System" -[ezyang_pkg]: http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/ "The fundamental problem of programming language package management" -[npmgate]: https://www.reddit.com/r/programming/comments/4bjss2/an_11_line_npm_package_called_leftpad_with_only/ -[geekery75]: https://www.functionalgeekery.com/episode-75-eric-b-merritt/ -[w3]: https://www.w3.org/DesignIssues/ -[tbl_modularity]: https://www.w3.org/DesignIssues/Modularity.html -[tbl_design]: http://www.w3.org/DesignIssues/Principles.html -[sword]: http://www.freecode.com/articles/editorial-the-two-edged-sword -[composition]: https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/