198 lines
9.8 KiB
Org Mode
198 lines
9.8 KiB
Org Mode
#+TITLE: Modularity & Abstraction (working title)
|
|
#+AUTHOR: Chris Hodapp
|
|
#+DATE: April 20, 2017
|
|
#+TAGS: technobabble
|
|
|
|
Two central concepts that feature prominently anywhere that computers
|
|
do are _modularity_ and _abstraction_. This is meant very broadly: it
|
|
applies to designing software, using software, integrating software,
|
|
and to a lot of hardware as well. It certainly applies elsewhere too,
|
|
particularly other fields of engineering, but it appears to be
|
|
particularly crucial anywhere software is involved.
|
|
|
|
They're generally accepted as desireable, but a bit ill-understood at
|
|
times. It's common to find people who treat "abstraction" as
|
|
something that always stands in their way, adds overhead, and confuses
|
|
things. At the same time, it's common to find people who treat
|
|
modularity as being present anytime something is broken into pieces.
|
|
|
|
"Being abstract is something profoundly different from being vague.
|
|
The purpose of abstraction is not to be vague, but to create a new
|
|
semantic level in which one can be absolutely precise." E. W. Dijkstra
|
|
|
|
"Modular design hinges on the simplicity and abstract nature of the
|
|
interface definition between the modules. A design in which the
|
|
insides of each module need to know all about each other is not a
|
|
modular design but an arbitrary partitioning of the bits." (Tim
|
|
Berners-Lee in [[https://www.w3.org/DesignIssues/Principles.html][Principles of Design]].)
|
|
|
|
"Its is not only necessary to make sure your own system is designed to
|
|
be made of modular parts. It is also necessary to realize that your
|
|
own system, no matter how big and wonderful it seems now, should
|
|
always be designed to be a part of another larger system." (Same)
|
|
|
|
*Abstraction* and *modularity* are tied other inextricably. This is
|
|
because abstractions draw out the boundaries that modules sit inside,
|
|
and the interfaces (APIs, communication channels) through which they
|
|
talk. It need not necessarily be a standardized interface or a
|
|
well-documented boundary, though that helps.
|
|
|
|
*Decoupling*, *loose coupling*, and *tight coupling* relate to this as
|
|
well. When two things are decoupled, it often suggests something
|
|
about at least one of them being a module.
|
|
|
|
Available abstractions vary. The language itself may set boundaries
|
|
on what abstractions can be created, or practically created (a
|
|
language like Haskell contains various abstractions done largely
|
|
within the type system that cannot be expressed in many others;
|
|
languages like Python, Ruby, or JavaScript might have various
|
|
abstractions that are meaningful only in the context of dynamic
|
|
typing). Some languages more readily permit the creation of new
|
|
abstractions, and this might lead to a broader range of abstractions
|
|
implemented in libraries. The operating system and standard library
|
|
provide abstractions that may cross languages (what is a process?
|
|
what is a thread? what is a library? what is a filesystem?). This
|
|
extends into electronics (where agreed-upon protocols permit
|
|
interconnectivity and an ignorance of internals that is required for
|
|
abstraction) where "modules" take on a tangible form. Abstractions
|
|
also change over time - both in specifics (consult any list of dead
|
|
protocols and technologies) and in broader classes (consider how we
|
|
now have entire abstractions devoted to provisioning virtual
|
|
resources, and consider how CGI and FastCGI used to be pretty
|
|
widely-used interfaces).
|
|
|
|
In a practical sense: Where someone "factors out" something that
|
|
occurs in similar or identical form in multiple places (incidentally,
|
|
"decouples" also works fine as a term), they're often creating a
|
|
module (from what was factored out) and some number of abstractions
|
|
(from the break that created). Consider some examples:
|
|
- Some configurable functionality in a larger application is extracted
|
|
out into a system of plugins. The details of the application are
|
|
abstracted over (as far as the plugin cares), and the details of the
|
|
plugin are abstracted over (as far as the application cares). The
|
|
API that the application and plugins use to communicate is the new
|
|
abstraction now available. The plugins are modules, and the
|
|
application itself is a module of a different sort. (Witness that
|
|
sometimes another application will implement the same plugin API.)
|
|
|
|
Given the strong ties between modularity and abstraction, the
|
|
possible and sensible ways to modularize things then also vary.
|
|
|
|
The modules that make sense may change over time. ("Have modern
|
|
tools tried to keep embodying the 'Unix philosophy'? Have they
|
|
extended it, even to other forms of abstraction that weren't
|
|
previously considered?")
|
|
|
|
One reason behind this is more practical in nature: When something is
|
|
a module unto itself, presumably it is relying on specific
|
|
abstractions, and it is possible to move this module to other contexts
|
|
(anything providing the same abstractions) or to replace it with other
|
|
modules (anything using the same abstractions).
|
|
|
|
Another reason is more abstract: When something it a module unto
|
|
itself, the way it is designed and implemented often presents more
|
|
insight into the fundamentals of the problem it is solving. It
|
|
contains fewer incidental details, and more essential details.
|
|
|
|
* Other fluff
|
|
|
|
I was around to see what was normal for software made on Windows
|
|
3.1, Windows 95, and the like. My take is that most of these pieces
|
|
of software were sufficiently GUI-oriented that they tried to remove
|
|
most modularity from the user's perspective. Things like scripting
|
|
and automation were almost solely as afterthoughts, since most
|
|
interaction was designed explicitly around the GUI.
|
|
|
|
People know that I love Emacs, but I also do believe many of the
|
|
complaints on how large it is. On the one hand, it is basically its
|
|
own operating system and within this it has considerable modularity.
|
|
On the other hand, there is a perfectly usable operating system
|
|
underneath, and this makes me wonder why it needs explicit support for
|
|
[[https://www.gnu.org/software/tramp/][network transparency]].
|
|
|
|
* Fragments
|
|
|
|
- Abstracting over...
|
|
- Multiple applications
|
|
- Multiple users
|
|
- Multiple CPUs
|
|
- Multiple hosts
|
|
|
|
- Nix, Guix
|
|
- [[Notes - Distributed stuff notes (from turtl)]]
|
|
- [[Notes - Paper, 2016-11-13]]
|
|
- See notes on functional geekery #75
|
|
- Jupyter
|
|
- Any Plan 9 papers? (Will have to dig deep in the archives)
|
|
- http://plan9.bell-labs.com/sys/doc/
|
|
- Tanenbaum vs. Linus war & microkernels
|
|
- Conjecture: A module is most useful when available in the most
|
|
general or most accessible context (e.g. Linux commandline tool
|
|
vs. a Wordpress plugin or an Emacs package) - the TBL quote on
|
|
least-power sort of corroborates this, but stands separate in some
|
|
other ways too.
|
|
- "most general" might not be right here.
|
|
- Other conjecture attempt: A module's power is related to how many
|
|
other modules it can communicate with, without requiring substantial
|
|
adaptation. An abstraction's power is related to the modularity it
|
|
accomodates.
|
|
- Another conjecture attempt: An abstraction's power isn't related to
|
|
how broad it is, but to how well it connects things. A needlessly
|
|
simplistic abstraction requires a lot of other adaptation to be
|
|
useful. A needlessly specific one excludes a lot of potential
|
|
modues.
|
|
- hypothes.is is a sort of module unto itself here too, trying to
|
|
remove commenting and annotation from existing, very siloed
|
|
solutions.
|
|
- "Self" paper & structural reification?
|
|
- I'm still not sure how this relates, but it may perhaps relate to
|
|
how *not* to make things modular (structural reification is a sort
|
|
of check on the scope of objects/classes)
|
|
- What by Rich Hickey?
|
|
- SICP?
|
|
- https://mitpress.mit.edu/sicp/full-text/sicp/book/node50.html
|
|
- "On Understanding Data Abstraction, Revisited"
|
|
- Frameworks Don't Compose ([composition][])
|
|
- "On the Criteria to be Used in Decomposing System into Modules" (Barnas)
|
|
- suckless, and their tools & methodology
|
|
- https://suckless.org/philosophy
|
|
- even though they can take things waaaay too far...
|
|
- Containers?
|
|
- http://www.catb.org/~esr/writings/taoup/html/apb.html#Baldwin-Clark -
|
|
Carliss Baldwin and Kim Clark. Design Rules, Vol 1: The Power of
|
|
Modularity. 2000. MIT Press. ISBN 0-262-024667.
|
|
- https://colah.github.io/posts/2015-09-NN-Types-FP/ - Was this the
|
|
one that talked about 'modularity' in deep learning?
|
|
- NodeRED might be interesting here, but first I need a clear idea of
|
|
what it factored out into a separate component.
|
|
- Lua is notable here for the effort spent in making it easy to both
|
|
embed (e.g. as a scripting or extension language) and extend
|
|
(e.g. with other C libraries). Guile may be similar.
|
|
- NeoVim is also an interesting case here as it is designed to be
|
|
embedded, though I'm not sure what this means yet.
|
|
|
|
- Find the link to Les Hatton's slides (cyclomatic complexity?) on the
|
|
empirical effects of too many / too large modules
|
|
|
|
- Examples of more 'modern' tools:
|
|
- socat
|
|
- jq (the JSON processor)
|
|
- Nix and some related tools (which take related functionality that
|
|
is present in numerous PL-specific package managers)
|
|
- Jupyter
|
|
|
|
* Link-pile:
|
|
|
|
- [[http://www.catb.org/~esr/writings/taoup/html/][The Art of Unix Programming (Eric S. Raymond)]]
|
|
- [[https://circleci.com/blog/its-the-future/][It's the Future]]
|
|
- [[https://circleci.com/blog/it-really-is-the-future/][It really is the future]]
|
|
- [[https://www.youtube.com/watch?v%253Dtc4ROCJYbm0][AT&T Archives: The UNIX Operating System]]
|
|
- [[http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/][The fundamental problem of programming language package management]]
|
|
- https://www.reddit.com/r/programming/comments/4bjss2/an_11_line_npm_package_called_leftpad_with_only/
|
|
- https://www.functionalgeekery.com/episode-75-eric-b-merritt/
|
|
- https://www.w3.org/DesignIssues/
|
|
- https://www.w3.org/DesignIssues/Modularity.html
|
|
- http://www.w3.org/DesignIssues/Principles.html
|
|
- http://www.freecode.com/articles/editorial-the-two-edged-sword
|
|
- https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/
|