hodapp/blag
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add some more scraps to modularity post

Browse Source
This commit is contained in:
Chris Hodapp 2017-12-11 12:48:59 -05:00
parent 08fac4b94d
commit 60cc97f219
2 changed files with 197 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

197
drafts/2017-04-20-modularity.org Normal file
View File

@ -0,0 +1,197 @@
#+TITLE: Modularity & Abstraction (working title)
#+AUTHOR: Chris Hodapp
#+DATE: April 20, 2017
#+TAGS: technobabble
Two central concepts that feature prominently anywhere that computers
do are _modularity_ and _abstraction_. This is meant very broadly: it
applies to designing software, using software, integrating software,
and to a lot of hardware as well. It certainly applies elsewhere too,
particularly other fields of engineering, but it appears to be
particularly crucial anywhere software is involved.
They're generally accepted as desireable, but a bit ill-understood at
times. It's common to find people who treat "abstraction" as
something that always stands in their way, adds overhead, and confuses
things. At the same time, it's common to find people who treat
modularity as being present anytime something is broken into pieces.
"Being abstract is something profoundly different from being vague.
The purpose of abstraction is not to be vague, but to create a new
semantic level in which one can be absolutely precise." E. W. Dijkstra
"Modular design hinges on the simplicity and abstract nature of the
interface definition between the modules. A design in which the
insides of each module need to know all about each other is not a
modular design but an arbitrary partitioning of the bits." (Tim
Berners-Lee in [[https://www.w3.org/DesignIssues/Principles.html][Principles of Design]].)
"Its is not only necessary to make sure your own system is designed to
be made of modular parts. It is also necessary to realize that your
own system, no matter how big and wonderful it seems now, should
always be designed to be a part of another larger system." (Same)
*Abstraction* and *modularity* are tied other inextricably. This is
because abstractions draw out the boundaries that modules sit inside,
and the interfaces (APIs, communication channels) through which they
talk. It need not necessarily be a standardized interface or a
well-documented boundary, though that helps.
*Decoupling*, *loose coupling*, and *tight coupling* relate to this as
well. When two things are decoupled, it often suggests something
about at least one of them being a module.
Available abstractions vary. The language itself may set boundaries
on what abstractions can be created, or practically created (a
language like Haskell contains various abstractions done largely
within the type system that cannot be expressed in many others;
languages like Python, Ruby, or JavaScript might have various
abstractions that are meaningful only in the context of dynamic
typing). Some languages more readily permit the creation of new
abstractions, and this might lead to a broader range of abstractions
implemented in libraries. The operating system and standard library
provide abstractions that may cross languages (what is a process?
what is a thread? what is a library? what is a filesystem?). This
extends into electronics (where agreed-upon protocols permit
interconnectivity and an ignorance of internals that is required for
abstraction) where "modules" take on a tangible form. Abstractions
also change over time - both in specifics (consult any list of dead
protocols and technologies) and in broader classes (consider how we
now have entire abstractions devoted to provisioning virtual
resources, and consider how CGI and FastCGI used to be pretty
widely-used interfaces).
In a practical sense: Where someone "factors out" something that
occurs in similar or identical form in multiple places (incidentally,
"decouples" also works fine as a term), they're often creating a
module (from what was factored out) and some number of abstractions
(from the break that created). Consider some examples:
- Some configurable functionality in a larger application is extracted
out into a system of plugins. The details of the application are
abstracted over (as far as the plugin cares), and the details of the
plugin are abstracted over (as far as the application cares). The
API that the application and plugins use to communicate is the new
abstraction now available. The plugins are modules, and the
application itself is a module of a different sort. (Witness that
sometimes another application will implement the same plugin API.)
Given the strong ties between modularity and abstraction, the
possible and sensible ways to modularize things then also vary.
The modules that make sense may change over time. ("Have modern
tools tried to keep embodying the 'Unix philosophy'? Have they
extended it, even to other forms of abstraction that weren't
previously considered?")
One reason behind this is more practical in nature: When something is
a module unto itself, presumably it is relying on specific
abstractions, and it is possible to move this module to other contexts
(anything providing the same abstractions) or to replace it with other
modules (anything using the same abstractions).
Another reason is more abstract: When something it a module unto
itself, the way it is designed and implemented often presents more
insight into the fundamentals of the problem it is solving. It
contains fewer incidental details, and more essential details.
* Other fluff
I was around to see what was normal for software made on Windows
3.1, Windows 95, and the like. My take is that most of these pieces
of software were sufficiently GUI-oriented that they tried to remove
most modularity from the user's perspective. Things like scripting
and automation were almost solely as afterthoughts, since most
interaction was designed explicitly around the GUI.
People know that I love Emacs, but I also do believe many of the
complaints on how large it is. On the one hand, it is basically its
own operating system and within this it has considerable modularity.
On the other hand, there is a perfectly usable operating system
underneath, and this makes me wonder why it needs explicit support for
[[https://www.gnu.org/software/tramp/][network transparency]].
* Fragments
- Abstracting over...
- Multiple applications
- Multiple users
- Multiple CPUs
- Multiple hosts
- Nix, Guix
- [[Notes - Distributed stuff notes (from turtl)]]
- [[Notes - Paper, 2016-11-13]]
- See notes on functional geekery #75
- Jupyter
- Any Plan 9 papers? (Will have to dig deep in the archives)
- http://plan9.bell-labs.com/sys/doc/
- Tanenbaum vs. Linus war & microkernels
- Conjecture: A module is most useful when available in the most
general or most accessible context (e.g. Linux commandline tool
vs. a Wordpress plugin or an Emacs package) - the TBL quote on
least-power sort of corroborates this, but stands separate in some
other ways too.
- "most general" might not be right here.
- Other conjecture attempt: A module's power is related to how many
other modules it can communicate with, without requiring substantial
adaptation. An abstraction's power is related to the modularity it
accomodates.
- Another conjecture attempt: An abstraction's power isn't related to
how broad it is, but to how well it connects things. A needlessly
simplistic abstraction requires a lot of other adaptation to be
useful. A needlessly specific one excludes a lot of potential
modues.
- hypothes.is is a sort of module unto itself here too, trying to
remove commenting and annotation from existing, very siloed
solutions.
- "Self" paper & structural reification?
- I'm still not sure how this relates, but it may perhaps relate to
how *not* to make things modular (structural reification is a sort
of check on the scope of objects/classes)
- What by Rich Hickey?
- SICP?
- https://mitpress.mit.edu/sicp/full-text/sicp/book/node50.html
- "On Understanding Data Abstraction, Revisited"
- Frameworks Don't Compose ([composition][])
- "On the Criteria to be Used in Decomposing System into Modules" (Barnas)
- suckless, and their tools & methodology
- https://suckless.org/philosophy
- even though they can take things waaaay too far...
- Containers?
- http://www.catb.org/~esr/writings/taoup/html/apb.html#Baldwin-Clark -
Carliss Baldwin and Kim Clark. Design Rules, Vol 1: The Power of
Modularity. 2000. MIT Press. ISBN 0-262-024667.
- https://colah.github.io/posts/2015-09-NN-Types-FP/ - Was this the
one that talked about 'modularity' in deep learning?
- NodeRED might be interesting here, but first I need a clear idea of
what it factored out into a separate component.
- Lua is notable here for the effort spent in making it easy to both
embed (e.g. as a scripting or extension language) and extend
(e.g. with other C libraries). Guile may be similar.
- NeoVim is also an interesting case here as it is designed to be
embedded, though I'm not sure what this means yet.
- Find the link to Les Hatton's slides (cyclomatic complexity?) on the
empirical effects of too many / too large modules
- Examples of more 'modern' tools:
- socat
- jq (the JSON processor)
- Nix and some related tools (which take related functionality that
is present in numerous PL-specific package managers)
- Jupyter
* Link-pile:
- [[http://www.catb.org/~esr/writings/taoup/html/][The Art of Unix Programming (Eric S. Raymond)]]
- [[https://circleci.com/blog/its-the-future/][It's the Future]]
- [[https://circleci.com/blog/it-really-is-the-future/][It really is the future]]
- [[https://www.youtube.com/watch?v%253Dtc4ROCJYbm0][AT&T Archives: The UNIX Operating System]]
- [[http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/][The fundamental problem of programming language package management]]
- https://www.reddit.com/r/programming/comments/4bjss2/an_11_line_npm_package_called_leftpad_with_only/
- https://www.functionalgeekery.com/episode-75-eric-b-merritt/
- https://www.w3.org/DesignIssues/
- https://www.w3.org/DesignIssues/Modularity.html
- http://www.w3.org/DesignIssues/Principles.html
- http://www.freecode.com/articles/editorial-the-two-edged-sword
- https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/

40
posts/2017-04-20-modularity.md
View File

@ -1,40 +0,0 @@
---
title: Modularity (working title)
author: Chris Hodapp
date: April 20, 2017
tags: technobabble
---
- Nix, Guix
- [[Notes - Distributed stuff notes (from turtl)]]
- [[Notes - Paper, 2016-11-13]]
- See notes on functional geekery #75
- Jupyter
- Any Plan 9 papers? (Will have to dig deep in the archives)
- http://plan9.bell-labs.com/sys/doc/
- Tanenbaum vs. Linus war & microkernels
- Abstraction vs. modularity
- tbl_design quote: "Modular design hinges on the simplicity and abstract nature of the interface definition between the modules. A design in which the insides of each module need to know all about each other is not a modular design but an arbitrary partitioning of the bits."
- Modules: Most useful when available in the most general or most accessible context, versus more specific, more restricted contexts? (e.g. Linux commandline tool vs. a Wordpress plugin or an Emacs package) - the TBL quote on least-power sort of corroborates this.
- hypothes.is is a sort of module unto itself here too, trying to remove commenting and annotation from existing, very siloed solutions.
- "Self" paper & structural reification?
- What by Rich Hickey?
- SICP?
- "On Understanding Data Abstraction, Revisited"
- Frameworks Don't Compose ([composition][])
- Abstraction enables modularity. Why: Abstraction barriers are what separate modules.
- The modules that make sense may change over time. ("Have modern tools tried to keep embodying the 'Unix philosophy'? Have they extended it, even to other forms of abstraction that weren't previously considered?")
[taoup]: http://www.catb.org/~esr/writings/taoup/html/ "The Art of Unix Programming (Eric S. Raymond)"
[future]: https://circleci.com/blog/its-the-future/ "It's the Future"
[future2]: https://circleci.com/blog/it-really-is-the-future/ "It really is the future"
[attvideo]: https://www.youtube.com/watch?v%3Dtc4ROCJYbm0 "AT&T Archives: The UNIX Operating System"
[ezyang_pkg]: http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/ "The fundamental problem of programming language package management"
[npmgate]: https://www.reddit.com/r/programming/comments/4bjss2/an_11_line_npm_package_called_leftpad_with_only/
[geekery75]: https://www.functionalgeekery.com/episode-75-eric-b-merritt/
[w3]: https://www.w3.org/DesignIssues/
[tbl_modularity]: https://www.w3.org/DesignIssues/Modularity.html
[tbl_design]: http://www.w3.org/DesignIssues/Principles.html
[sword]: http://www.freecode.com/articles/editorial-the-two-edged-sword
[composition]: https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/