Add some more scraps to modularity post
This commit is contained in:
Chris Hodapp
197
drafts/2017-04-20-modularity.org
Normal file
197
drafts/2017-04-20-modularity.org
Normal file
@@ -0,0 +1,197 @@
|
||||
#+TITLE: Modularity & Abstraction (working title)
|
||||
#+AUTHOR: Chris Hodapp
|
||||
#+DATE: April 20, 2017
|
||||
#+TAGS: technobabble
|
||||
|
||||
Two central concepts that feature prominently anywhere that computers
|
||||
do are _modularity_ and _abstraction_. This is meant very broadly: it
|
||||
applies to designing software, using software, integrating software,
|
||||
and to a lot of hardware as well. It certainly applies elsewhere too,
|
||||
particularly other fields of engineering, but it appears to be
|
||||
particularly crucial anywhere software is involved.
|
||||
|
||||
They're generally accepted as desireable, but a bit ill-understood at
|
||||
times. It's common to find people who treat "abstraction" as
|
||||
something that always stands in their way, adds overhead, and confuses
|
||||
things. At the same time, it's common to find people who treat
|
||||
modularity as being present anytime something is broken into pieces.
|
||||
|
||||
"Being abstract is something profoundly different from being vague.
|
||||
The purpose of abstraction is not to be vague, but to create a new
|
||||
semantic level in which one can be absolutely precise." E. W. Dijkstra
|
||||
|
||||
"Modular design hinges on the simplicity and abstract nature of the
|
||||
interface definition between the modules. A design in which the
|
||||
insides of each module need to know all about each other is not a
|
||||
modular design but an arbitrary partitioning of the bits." (Tim
|
||||
Berners-Lee in [[https://www.w3.org/DesignIssues/Principles.html][Principles of Design]].)
|
||||
|
||||
"Its is not only necessary to make sure your own system is designed to
|
||||
be made of modular parts. It is also necessary to realize that your
|
||||
own system, no matter how big and wonderful it seems now, should
|
||||
always be designed to be a part of another larger system." (Same)
|
||||
|
||||
*Abstraction* and *modularity* are tied other inextricably. This is
|
||||
because abstractions draw out the boundaries that modules sit inside,
|
||||
and the interfaces (APIs, communication channels) through which they
|
||||
talk. It need not necessarily be a standardized interface or a
|
||||
well-documented boundary, though that helps.
|
||||
|
||||
*Decoupling*, *loose coupling*, and *tight coupling* relate to this as
|
||||
well. When two things are decoupled, it often suggests something
|
||||
about at least one of them being a module.
|
||||
|
||||
Available abstractions vary. The language itself may set boundaries
|
||||
on what abstractions can be created, or practically created (a
|
||||
language like Haskell contains various abstractions done largely
|
||||
within the type system that cannot be expressed in many others;
|
||||
languages like Python, Ruby, or JavaScript might have various
|
||||
abstractions that are meaningful only in the context of dynamic
|
||||
typing). Some languages more readily permit the creation of new
|
||||
abstractions, and this might lead to a broader range of abstractions
|
||||
implemented in libraries. The operating system and standard library
|
||||
provide abstractions that may cross languages (what is a process?
|
||||
what is a thread? what is a library? what is a filesystem?). This
|
||||
extends into electronics (where agreed-upon protocols permit
|
||||
interconnectivity and an ignorance of internals that is required for
|
||||
abstraction) where "modules" take on a tangible form. Abstractions
|
||||
also change over time - both in specifics (consult any list of dead
|
||||
protocols and technologies) and in broader classes (consider how we
|
||||
now have entire abstractions devoted to provisioning virtual
|
||||
resources, and consider how CGI and FastCGI used to be pretty
|
||||
widely-used interfaces).
|
||||
|
||||
In a practical sense: Where someone "factors out" something that
|
||||
occurs in similar or identical form in multiple places (incidentally,
|
||||
"decouples" also works fine as a term), they're often creating a
|
||||
module (from what was factored out) and some number of abstractions
|
||||
(from the break that created). Consider some examples:
|
||||
- Some configurable functionality in a larger application is extracted
|
||||
out into a system of plugins. The details of the application are
|
||||
abstracted over (as far as the plugin cares), and the details of the
|
||||
plugin are abstracted over (as far as the application cares). The
|
||||
API that the application and plugins use to communicate is the new
|
||||
abstraction now available. The plugins are modules, and the
|
||||
application itself is a module of a different sort. (Witness that
|
||||
sometimes another application will implement the same plugin API.)
|
||||
|
||||
Given the strong ties between modularity and abstraction, the
|
||||
possible and sensible ways to modularize things then also vary.
|
||||
|
||||
The modules that make sense may change over time. ("Have modern
|
||||
tools tried to keep embodying the 'Unix philosophy'? Have they
|
||||
extended it, even to other forms of abstraction that weren't
|
||||
previously considered?")
|
||||
|
||||
One reason behind this is more practical in nature: When something is
|
||||
a module unto itself, presumably it is relying on specific
|
||||
abstractions, and it is possible to move this module to other contexts
|
||||
(anything providing the same abstractions) or to replace it with other
|
||||
modules (anything using the same abstractions).
|
||||
|
||||
Another reason is more abstract: When something it a module unto
|
||||
itself, the way it is designed and implemented often presents more
|
||||
insight into the fundamentals of the problem it is solving. It
|
||||
contains fewer incidental details, and more essential details.
|
||||
|
||||
* Other fluff
|
||||
|
||||
I was around to see what was normal for software made on Windows
|
||||
3.1, Windows 95, and the like. My take is that most of these pieces
|
||||
of software were sufficiently GUI-oriented that they tried to remove
|
||||
most modularity from the user's perspective. Things like scripting
|
||||
and automation were almost solely as afterthoughts, since most
|
||||
interaction was designed explicitly around the GUI.
|
||||
|
||||
People know that I love Emacs, but I also do believe many of the
|
||||
complaints on how large it is. On the one hand, it is basically its
|
||||
own operating system and within this it has considerable modularity.
|
||||
On the other hand, there is a perfectly usable operating system
|
||||
underneath, and this makes me wonder why it needs explicit support for
|
||||
[[https://www.gnu.org/software/tramp/][network transparency]].
|
||||
|
||||
* Fragments
|
||||
|
||||
- Abstracting over...
|
||||
- Multiple applications
|
||||
- Multiple users
|
||||
- Multiple CPUs
|
||||
- Multiple hosts
|
||||
|
||||
- Nix, Guix
|
||||
- [[Notes - Distributed stuff notes (from turtl)]]
|
||||
- [[Notes - Paper, 2016-11-13]]
|
||||
- See notes on functional geekery #75
|
||||
- Jupyter
|
||||
- Any Plan 9 papers? (Will have to dig deep in the archives)
|
||||
- http://plan9.bell-labs.com/sys/doc/
|
||||
- Tanenbaum vs. Linus war & microkernels
|
||||
- Conjecture: A module is most useful when available in the most
|
||||
general or most accessible context (e.g. Linux commandline tool
|
||||
vs. a Wordpress plugin or an Emacs package) - the TBL quote on
|
||||
least-power sort of corroborates this, but stands separate in some
|
||||
other ways too.
|
||||
- "most general" might not be right here.
|
||||
- Other conjecture attempt: A module's power is related to how many
|
||||
other modules it can communicate with, without requiring substantial
|
||||
adaptation. An abstraction's power is related to the modularity it
|
||||
accomodates.
|
||||
- Another conjecture attempt: An abstraction's power isn't related to
|
||||
how broad it is, but to how well it connects things. A needlessly
|
||||
simplistic abstraction requires a lot of other adaptation to be
|
||||
useful. A needlessly specific one excludes a lot of potential
|
||||
modues.
|
||||
- hypothes.is is a sort of module unto itself here too, trying to
|
||||
remove commenting and annotation from existing, very siloed
|
||||
solutions.
|
||||
- "Self" paper & structural reification?
|
||||
- I'm still not sure how this relates, but it may perhaps relate to
|
||||
how *not* to make things modular (structural reification is a sort
|
||||
of check on the scope of objects/classes)
|
||||
- What by Rich Hickey?
|
||||
- SICP?
|
||||
- https://mitpress.mit.edu/sicp/full-text/sicp/book/node50.html
|
||||
- "On Understanding Data Abstraction, Revisited"
|
||||
- Frameworks Don't Compose ([composition][])
|
||||
- "On the Criteria to be Used in Decomposing System into Modules" (Barnas)
|
||||
- suckless, and their tools & methodology
|
||||
- https://suckless.org/philosophy
|
||||
- even though they can take things waaaay too far...
|
||||
- Containers?
|
||||
- http://www.catb.org/~esr/writings/taoup/html/apb.html#Baldwin-Clark -
|
||||
Carliss Baldwin and Kim Clark. Design Rules, Vol 1: The Power of
|
||||
Modularity. 2000. MIT Press. ISBN 0-262-024667.
|
||||
- https://colah.github.io/posts/2015-09-NN-Types-FP/ - Was this the
|
||||
one that talked about 'modularity' in deep learning?
|
||||
- NodeRED might be interesting here, but first I need a clear idea of
|
||||
what it factored out into a separate component.
|
||||
- Lua is notable here for the effort spent in making it easy to both
|
||||
embed (e.g. as a scripting or extension language) and extend
|
||||
(e.g. with other C libraries). Guile may be similar.
|
||||
- NeoVim is also an interesting case here as it is designed to be
|
||||
embedded, though I'm not sure what this means yet.
|
||||
|
||||
- Find the link to Les Hatton's slides (cyclomatic complexity?) on the
|
||||
empirical effects of too many / too large modules
|
||||
|
||||
- Examples of more 'modern' tools:
|
||||
- socat
|
||||
- jq (the JSON processor)
|
||||
- Nix and some related tools (which take related functionality that
|
||||
is present in numerous PL-specific package managers)
|
||||
- Jupyter
|
||||
|
||||
* Link-pile:
|
||||
|
||||
- [[http://www.catb.org/~esr/writings/taoup/html/][The Art of Unix Programming (Eric S. Raymond)]]
|
||||
- [[https://circleci.com/blog/its-the-future/][It's the Future]]
|
||||
- [[https://circleci.com/blog/it-really-is-the-future/][It really is the future]]
|
||||
- [[https://www.youtube.com/watch?v%253Dtc4ROCJYbm0][AT&T Archives: The UNIX Operating System]]
|
||||
- [[http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/][The fundamental problem of programming language package management]]
|
||||
- https://www.reddit.com/r/programming/comments/4bjss2/an_11_line_npm_package_called_leftpad_with_only/
|
||||
- https://www.functionalgeekery.com/episode-75-eric-b-merritt/
|
||||
- https://www.w3.org/DesignIssues/
|
||||
- https://www.w3.org/DesignIssues/Modularity.html
|
||||
- http://www.w3.org/DesignIssues/Principles.html
|
||||
- http://www.freecode.com/articles/editorial-the-two-edged-sword
|
||||
- https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/
|
||||
Reference in New Issue
Block a user