hodapp/blag
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

More refactoring in modularity post

Browse Source
This commit is contained in:
hodapp 2020-01-01 15:37:20 -05:00
parent c5e4134782
commit 76a33703bf
1 changed files with 58 additions and 59 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

117
drafts/2017-04-20-modularity.org
View File

@ -121,6 +121,35 @@ some details implies that a module is handling those details, that the
details shouldn't matter, and what does matter is the abstraction one
is using.
In a practical sense: Where someone "factors out" something that
occurs in similar or identical form in multiple places ("decouples"
also works fine as a term), they're often creating a module (from what
was factored out) and some number of abstractions (from the break that
created). Consider some examples:
- Some configurable functionality in a larger application is extracted
out into a system of plugins. The details of the application are
abstracted over (as far as the plugin cares), and the details of the
plugin are abstracted over (as far as the application cares). The
API that the application and plugins use to communicate is the new
abstraction now available. The plugins are modules, and the
application itself is a module of a different sort. (Witness that
sometimes another application will implement the same plugin API.)
"Composition" is also important. The ability of modules to be
*composed* together is a big part of their usefulness. The ability to
*decompose* something is often a requirement to create useful modules.
([[https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/][Composition over convention]] talks about this in an important take on
why /frameworks/ can also run completely counter to modularity.)
Almost any resource on software engineering will use many other terms
for related concepts and approaches. To name a few:
- [[https://en.wikipedia.org/wiki/SOLID][SOLID]]
- [[https://en.wikipedia.org/wiki/Cross-cutting_concern][Cross-cutting concerns]] and [[https://en.wikipedia.org/wiki/Aspect-oriented_programming][Aspect-oriented programming]]
- [[https://en.wikipedia.org/wiki/Separation_of_concerns][Separation of Concerns]]
- [[https://en.wikipedia.org/wiki/Abstraction_principle_(computer_programming)][Abstraction principle]]
- [[https://en.wikipedia.org/wiki/Don%2527t_repeat_yourself][Don't repeat yourself]]
# -----
Consider the information this module deals in, in essence.
@ -133,29 +162,16 @@ contexts?
# -----
In a practical sense: Where someone "factors out" something that
occurs in similar or identical form in multiple places (incidentally,
"decouples" also works fine as a term), they're often creating a
module (from what was factored out) and some number of abstractions
(from the break that created). Consider some examples:
- Some configurable functionality in a larger application is extracted
out into a system of plugins. The details of the application are
abstracted over (as far as the plugin cares), and the details of the
plugin are abstracted over (as far as the application cares). The
API that the application and plugins use to communicate is the new
abstraction now available. The plugins are modules, and the
application itself is a module of a different sort. (Witness that
sometimes another application will implement the same plugin API.)
It has a very pragmatic reason behind it: When something is a module
unto itself, presumably it is relying on specific abstractions, and it
is possible to freely change this module's internal details (provided
that it still handles the same abstractions), to move this module to
other contexts (anything providing the same abstractions), to replace
it with other modules (anything using the same abstractions).
that it still respects the same abstractions), to move this module to
other contexts (anywhere that provides the same abstractions), and to
replace it with other modules (anything that respects the same
abstractions).
It also has a more abstract reason: When something is a module unto
itself, the way it is designed and implemented often presents more
itself, the way it is designed and implemented usually presents more
insight into the fundamentals of the problem it is solving. It
contains fewer incidental details, and more essential details.
@ -168,9 +184,9 @@ emerge (or, perhaps, old ones become more widespread) to solve
problems that I wasn't even aware existed.
[[https://circleci.com/blog/it-really-is-the-future/][It really is the future]] talks about a lot of more recent forms of
modularity, most of which are beyond me and were completely unheard-of
in, say, 2010. [[https://www.functionalgeekery.com/episode-75-eric-b-merritt/][Functional Geekery episode 75]] talks about many similar
things.
modularity from the land of devops, most of which were completely
unheard-of in, say, 2010. [[https://www.functionalgeekery.com/episode-75-eric-b-merritt/][Functional Geekery episode 75]] talks about
many similar things.
[[https://jupyter.org/][Jupyter Notebook]] is one of my favorites here. It provides a notebook
interface (similar to something like Maple or Mathematica) which:
@ -187,19 +203,20 @@ I love notebook interfaces already because they simplify experimenting
by handling a lot of things I'd otherwise have to do manually - like
saving results and keeping them lined up with the exact code that
produced them. Jupyter adds some other use-cases I find marvelous -
for instance, I can let the interpreter run on my much faster
workstation, but I can access it across the Internet from my much
slower laptop.
for instance, I can let the interpreter run on my workstation which
has all of the computing power, but I can access it across the
Internet from my laptop.
[[https://zeppelin.apache.org/][Apache Zeppelin]] does similar things with different languages; I just
use it less.
[[https://zeppelin.apache.org/][Apache Zeppelin]] does similar things with different languages; I've
just used it much less.
Another favorite of mine is [[https://nixos.org/nix/][Nix]]. One excellent article, [[http://blog.ezyang.com/2014/08/the-fundamental-problem-of-programming-language-package-management/][The
fundamental problem of programming language package management]],
doesn't ever mention Nix but does a great job explaining the sorts of
problems it exists to solve. To be able to combine nearly all of the
doesn't ever mention Nix but explains very well the problems it sets
out to solve. To be able to combine nearly all of the
programming-language specific package managers into a single module is
a very lofty goal, but Nix appears to do a decent job of it.
a very lofty goal, but Nix appears to do a decent job of it (among
other things).
The [[https://www.lua.org/][Lua]] programming language is noteworthy here. It's written in
clean C with minimal dependencies, so it runs nearly anywhere that a C
@ -218,7 +235,7 @@ particularly if you allow dead ones created during the object-oriented
hype of the '90s. This seems to happen in systems where the object
hierarchy is in effect "bigger" than the language.
ZeroMQ is another example as a set of cross-language abstractions for
[[https://zeromq.org/][ZeroMQ]] is another example: a set of cross-language abstractions for
communication patterns in a distributed system. I know it's likely
not unique, but it is one of the better-known and the first I thought
of, and I think their [[http://zguide.zeromq.org/page:all][guide]] is excellent.
@ -235,7 +252,8 @@ programming language.
[[https://web.hypothes.is/][hypothes.is]] is a curious one that I find fascinating. They're trying
to factor out annotation and commenting from something that is handled
on a per-webpage basis and turn it into its own module, and I really
like what I've seen.
like what I've seen. However, it does not seem to have caught on
much.
The Unix tradition lives on in certain modern tools. [[https://stedolan.github.io/jq/][jq]] has proven
very useful anytime I've had to mess with JSON data. [[http://www.dest-unreach.org/socat/][socat]] and [[http://netcat.sourceforge.net/][netcat]]
@ -252,7 +270,7 @@ mention...
# Also, caches - of all types. (CPU, disk...)
# One key is how the above let you *reason* about things without
# knowing their specifics.a
# knowing their specifics.
People know that I love Emacs, but I also do believe many of the
complaints on how large it is. Despite that it is basically its own
@ -267,25 +285,21 @@ machine learning conflicts with proper modularity and abstraction.
a good post and shows some sorts of abstraction that still exist
at least in neural networks.)
Even DOS presented useful abstractions. Things like
Even DOS had useful abstractions. Things like
DriveSpace/DoubleSpace/Stacker worked well enough because most
software that needed files relied on DOS's normal abstractions to
access them - so it did not matter to them that the underlying
filesystem was actually compressed, or in a RAM disk or whatever.
Likewise, for the silliness known as [[https://en.wikipedia.org/wiki/Expanded_memory][EMS]], applications that accessed
memory through the EMS abstraction could disregard whether it was a
"real" EMS board providing access to that memory, whether it was an
expanded memory manager providing indirect access to some other memory
or even to a hard disk pretending to be memory.
filesystem was actually compressed, or was actually a RAM disk, or was
on some obscure SCSI interface. Likewise, for the silliness known as
[[https://en.wikipedia.org/wiki/Expanded_memory][EMS]], applications that accessed memory through the EMS abstraction
could disregard whether it was a "real" EMS board providing access to
that memory, whether it was an expanded memory manager providing
indirect access to some other memory or even to a hard disk pretending
to be memory.
Even more abstractly: emulators work because so much software
respected the abstraction of some specific CPU and hardware platform.
# Does the below belong here? that isn't really an example.
[[https://clojurefun.wordpress.com/2012/08/17/composition-over-convention/][Composition over convention]] is an important read on why /frameworks/
can also run completely counter to modularity.
Submitted without further comment:
https://github.com/stevemao/left-pad/issues/4
@ -298,9 +312,6 @@ https://github.com/stevemao/left-pad/issues/4
- Multiple hosts
- [[Notes - Paper, 2016-11-13]]
- Any Plan 9 papers? (Will have to dig deep in the archives)
- http://plan9.bell-labs.com/sys/doc/
- Link is now down
- Tanenbaum vs. Linus war & microkernels
- TBL: "The choice of language is a common design choice. The low
power end of the scale is typically simpler to design, implement and
@ -334,11 +345,6 @@ https://github.com/stevemao/left-pad/issues/4
- https://twitter.com/fchollet/status/962074070513631232
- How does this fit with /composition/? Does it?
- The ability to sensibly compose things depends on them having some
sort of well-defined, compatible boundary - right?
- Note also /decomposition/ here, as in /decomposing/ something into
parts.
- [[https://mitpress.mit.edu/sites/default/files/sicp/full-text/book/book-Z-H-9.html#%25_chap_1][From SICP chapter 1 intro]]: "The acts of the mind, wherein it exerts
its power over simple ideas, are chiefly these three: 1. Combining
several simple ideas into one compound one, and thus all complex
@ -349,10 +355,6 @@ https://github.com/stevemao/left-pad/issues/4
from all other ideas that accompany them in their real existence:
this is called abstraction, and thus all its general ideas are
made." -John Locke, An Essay Concerning Human Understanding (1690)
- [[https://en.wikipedia.org/wiki/Cross-cutting_concern][Cross-cutting concerns]], [[https://en.wikipedia.org/wiki/Aspect-oriented_programming][aspect-oriented programming]] (as an attempt
to take tangled things and pull them into modules)
- [[https://en.wikipedia.org/wiki/Separation_of_concerns][Separation of Concerns]]
- Abstraction as an information channel... module as a what?
- One point I have ignored (maybe): You clearly separate the 'inside'
of a module (its implementation) from the 'outside' (that is - its
boundaries, the abstractions that it interfaces with or that it
@ -364,9 +366,6 @@ https://github.com/stevemao/left-pad/issues/4
other modules do not need to change to conform)
- What is more key? Communication, information content, contracts,
details?
- [[https://en.wikipedia.org/wiki/Abstraction_principle_(computer_programming)][Abstraction principle]]
- Reduce duplication of information
- [[https://en.wikipedia.org/wiki/Don%2527t_repeat_yourself][Don't repeat yourself]]
- [[https://simplyphilosophy.org/study/aristotles-definitions/][Aristotle & theory of definitions]]
- this isn't right. I need to find the quote in the Durant book
(which will probably have an actual source) that pertains to how