Misc script stuff?

content/posts/2021-07-27-procedural-meshes/index.org

@@ -0,0 +1,206 @@
+---
+title: "(post on procedural meshes needs a title)"
+author: Chris Hodapp
+date: "2021-07-27"
+tags:
+- procedural graphics
+draft: true
+---
+
+# (TODO: a note to me, reading later: you don't need to give your entire
+# life story here.)
+
+# (TODO: pictures will make this post make a *lot* more sense, and it
+# may need a lot of them)
+
+Context Free is one of my favorite projects since I discovered it
+about 2010.  It's one I've written about before (TODO: link to my
+posts), played around in (TODO: link to images), presented on, as well
+as re-implemented myself in different ways (see: [[https://github.com/hodapp87/contextual][Contextual]]).  That is
+sometimes because I wanted to do something Context Free couldn't, such
+as make it realtime and interactive, and sometimes because
+implementing its system of recursive grammars and replacement rules
+can be an excellent way to learn things in a new language.  (I think
+it's similar to [[https://en.wikipedia.org/wiki/L-system][L-systems]], but I haven't yet learned those very well.)
+
+I've also played around in 3D graphics, particularly raytracing, since
+about 1999 in PolyRay and POV-Ray.  POV-Ray is probably what led me to
+learn about things like implicit surfaces, parametric surfaces, and
+procedural geometry - its scene language is full of constructs for
+that.  Naturally, this led me to wonder how I might extend Context
+Free's model to work more generally with 3D geometry, and let me use
+it to produce procedural geometry.
+
+[[http://structuresynth.sourceforge.net/index.php][Structure Synth]] of course already exists, and is a straightforward
+generalization of Context Free's model to 3D (thank you to Mikael
+Hvidtfeldt Christensen's blog [[http://blog.hvidtfeldts.net/][Syntopia]], another of my favorite things
+ever, for introducing me to it awhile ago).  See also [[https://kronpano.github.io/BrowserSynth/][BrowserSynth]].
+However, at some point I realized they weren't exactly what I wanted.
+Structure Synth lets you combine together 3D primitives to build up a
+more complex scene - but doesn't try to properly handle any sort of
+*joining* of these primitives in a way that respects many of the
+'rules' of geometry that are necessary for a lot of tools, like having
+a well-defined inside/outside, not being self-intersecting, being
+manifold, and so forth.
+
+Tools like [[https://openscad.org/][OpenSCAD]], based on [[https://www.cgal.org/][CGAL]], handle the details of this, and I
+suspect that [[https://www.opencascade.com/][Open CASCADE]] (thus [[https://www.freecadweb.org/][FreeCAD]]) also does.  In CAD work, it's
+crucial.  I experimented with similar recursive systems with some of
+these, but I quickly ran into a problem: they were made for actual
+practical applications in CAD, not so much for my generative art, and
+they scaled quite poorly with the sort of recursion I was asking for.
+
+Implicit surfaces (or one of the many
+equivalent-except-for-when-it's-not names for this, e.g. F-Reps or
+distance bounds or SDFs or isosurfaces) handle almost all of this
+well! They express CSG (TODO: link to CSG) operations, they can be
+rendered directly on the GPU via shaders, operations like blending
+shapes or twisting them are easy... for more on this, see [[http://blog.hvidtfeldts.net/][Syntopia]]
+again, or nearly anything by [[https://iquilezles.org/][Inigo Quilez]], or look up raymarching and
+sphere tracing, or see [[https://ntopology.com/][nTopology]], or Matt Keeter's work with [[https://www.libfive.com/][libfive]]
+and [[https://www.mattkeeter.com/research/mpr/][MPR]].  They're pure magic, they're wonderfully elegant, and I'll
+probably have many other posts on them.
+
+However, there is one big issue: turning implicit surfaces to good
+meshes for rendering /is a huge pain/, and while many renderers can
+handle implicit surfaces directly, Blender's renderers cannot. I will
+have other posts going into more detail on this subject, but for now,
+take it on faith.  This is why I did not try to use implicit surfaces
+for this project.  (TODO: Make those posts.)
+
+With these limitations in mind, around 2018 June I had started jotting
+some ideas down.  The gist is that I wanted to create
+"correct-by-construction" meshes from these recursive grammars.  By
+that, I meant: incrementally producing the desired geometry as a mesh,
+triangle-by-triangle, in such a way that guaranteed that the resultant
+mesh had the desired detail level, was a manifold surface, and that it
+was otherwise a well-behaved mesh (e.g. no degenerate triangles, no
+self-intersection, no high-degree vertices, no triangles of extreme
+angles) - rather than attempting to patch up the mesh after its
+creation, or subdividing it to the necessary detail level.  For
+something similar to what I mean (though I didn't have this in mind at
+the start), consider the [[https://en.wikipedia.org/wiki/Marching_squares][marching squares]] algorithm, which is
+guaranteed to produce closed, manifold meshes.
+
+(TODO: Illustrate this somehow)
+
+The form it took in my notes was in sort of "growing" or "extruding" a
+mesh per these recursive rules, building in these guarantees (some of
+them at least) by way of inductive steps.
+
+My meandering path to implementing it went something like this:
+
+- Write some very ad-hoc Python to generate a mesh of a parametric
+  conversion of my annoying spiral isosurface from 2005 by breaking it
+  into planar "slices" or "frames", which move along the geometry and
+  then are connected together at corresponding vertices.  (TODO: Add
+  link to the automata_scratch repo, whatever it's renamed to)
+- Explore [[https://github.com/thi-ng/geom][thi.ng/geom]] and pretty quickly give up - but in the process,
+  discover [[https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.42.8103][Parallel Transport Approach to Curve Framing]].
+- Implement that paper in Python, reusing the basic model from my
+  prior code. (See [[https://github.com/Hodapp87/parallel_transport][parallel_transport]])
+- Again continue with this model, allowing more arbitrary operations
+  than parallel frame transport, eventually integrating most of what I
+  wanted with the recursive grammars.  (See
+  [[https://github.com/Hodapp87/automata_scratch/tree/master/python_extrude_meshgen][automata_scratch/python_extrude_meshgen]])
+- Keep running into limitations in python_extrude_meshgen, and start
+  [[https://github.com/Hodapp87/prosha][Prosha]] in Rust - partly as a redesign/rewrite to avoid these
+  limitations, and partly because I just wanted to learn Rust.
+- Realize that Rust is the wrong tool for the job, and rewrite *again*
+  in Python but with a rather different design and mindset.
+
+(this is, of course, ignoring many other tangents with things like
+shaders)
+
+(TODO: Maybe split these off into sections for each one?)
+
+I put some serious effort into [[https://github.com/Hodapp87/prosha][Prosha]] and was conflicted on shelving
+the project indefinitely, but the issues didn't look easily solvable.
+Part of those issues were implementation issues with Rust - not that
+Rust could have really done anything "better" here, but that it just
+wasn't the right tool for what I was doing.  In short, I had spent a
+lot of that effort trying to badly and unintentionally implement/embed
+a Lisp inside of Rust instead of just picking a Lispier language, or
+perhaps using an embeddable Rust-based scripting language like [[https://github.com/koto-lang/koto][Koto]] or
+[[https://github.com/rhaiscript/rhai][Rhai]]. I had ignored that many things that functional programming left
+me very accustomed to - like first-class functions and closures - were
+dependent on garbage collection.  When I realized this and did a big
+refactor to remove this entire layer of complexity, I was left with
+very little "core" code - just a handful of library functions, and the
+actual recursive rules for the geometry I was trying to generate.
+That's good and bad: things were much simpler and vastly faster, but
+also, it felt like I had wasted quite a lot of time and effort.  I
+have some more detailed notes on this in the Prosha repository.
+
+Part of the issues also weren't Rust implementation issues - they were
+deeper issues with my original "correct-by-construction" mesh idea
+being half-broken.  It half-worked: I was able to produce closed,
+manifold meshes this way, and it could be tedious, but not *that*
+difficult.  However, all of my attempts to also produce "good" meshes
+this way failed miserably.
+
+(TODO: Can I find examples of this?)
+
+The crux is that the recursive rules I used for generating geometry
+(inspired heavily by those in Context Free) were inherently based
+around discrete steps, generating discrete entities, like vertices,
+edges, and face, and it made no sense to "partially" apply a rule,
+especially if that rule involved some kind of branching - but I kept
+trying to treat it as something continuous for the sake of being able
+to "refine" the mesh to as fine of detail as I wanted.  Further, I was
+almost never consistent with the nature of this continuity: sometimes
+I wanted to treat it like a parametric curve (one continuous
+parameter), sometimes I wanted to treat it like a parametric surface
+(two continuous parameters), sometimes I wanted to treat it like an
+implicit surface (with... theoretically two continuous parameters,
+just not explicit ones?).  It was a mess, and it's part of why my
+Prosha repository is a graveyard of branches.
+
+The recursive rules were still excellent at expressing arbitrarily
+complex, branching geometry - and I really wanted to keep this basic
+model around somehow.  After some reflection, I believed that the only
+way to do this was to completely separate the process of meshing
+(refinement, subdivision, facetization...) from the recursive rules.
+
+This would have been obvious if I read the guides from [[https://graphics.pixar.com/opensubdiv/overview.html][OpenSubdiv]]
+instead of reimplementing it badly.  Their [[https://graphics.pixar.com/opensubdiv/docs/subdivision_surfaces.html][subdivision surface]]
+documentation covers a lot, but I found it incredibly clear and
+readable.  Once I understood how OpenSubdiv was meant to be used, it
+made a lot of sense: I shouldn't be trying to generate the "final"
+mesh, I should be generating a mesh as the /control cage/, which
+guides the final mesh.  Further, I didn't even need to bother with
+OpenSubdiv's C++ API, I just needed to get the geometry into Blender,
+and Blender would handle the subdivision on-demand via OpenSubdiv.
+
+# TODO: This definitely needs examples of a control cage, and of edge
+# creases
+
+One minor issue is that this control cage isn't just a triangle mesh,
+but a triangle mesh plus edge creases.  I needed a way to get this
+data into Blender.  However, the only format Blender can read edge
+creases from is [[http://www.alembic.io/][Alembic]].  Annoyingly, its [[http://docs.alembic.io/reference/index.html#alembic-intro][documentation]] is almost
+completely nonexistent, the [[https://alembic.github.io/cask/][Cask]] bindings still have spotty Python 3.x
+support, and when I tried to run their example code to produce some
+files, and Blender was crashing when importing them.... and this is
+all a yak to shave another day.  I instead generated the mesh data
+directly in Blender (via its Python interpreter), added it to the
+scene, and then set its creases via its Python API.
+
+After the aforementioned refactor in Prosha, I was able to quickly
+translate the Rust code for most of my examples into Python code with
+the help of some library code I'd accumulated from the past projects.
+Debugging this mostly inside Blender also made the process vastly
+faster.  Further, because I was letting Blender handle all of the
+heavy lifting with mesh processing (and it in turn was using things
+like OpenSubdiv), the extra overhead of Python compared to Rust didn't
+matter - I was handling so much less data because I was generating
+only a control cage, not a full mesh.
+
+I'm still a little stuck at how to build higher 'geometric'
+abstractions here and compose them.  I have felt like most of the
+model couples me tightly to low-level mesh constructs - while Context
+Free and Structure Synth definitely don't have this problem.  This is
+particularly annoying because a lot of the power of these recursive
+grammars comes from their ability to be abstracted away and composed.
+
+# (TODO: Show some examples)