diff --git a/drafts/2018-03-09-python-asyncio.org b/drafts/2018-03-09-python-asyncio.org index c59a2a2..7af891a 100644 --- a/drafts/2018-03-09-python-asyncio.org +++ b/drafts/2018-03-09-python-asyncio.org @@ -5,13 +5,29 @@ date: March 9, 2018 tags: technobabble --- -Recently I needed to work a little more in-depth with [[https://docs.python.org/3/library/asyncio.html][asyncio]] in -Python 3.x. While some people (including me) might scoff at this -because cooperative threading is a model that's fresh out of the '90s -and because Python /still/ has the [[https://wiki.python.org/moin/GlobalInterpreterLock][GIL]], it is still preferable to -manually writing code in [[https://en.wikipedia.org/wiki/Continuation-passing_style][continuation-passing-style]] (that's all -callbacks are), and last time I had to write that many callbacks, I -hated it enough that I wrote my own [[https://github.com/HaskellEmbedded/ion][EDSL]] to avoid it. But I digress. +# TODO: Generators? Is it accurate that prior to all this, coroutines +# were still available, but by themselves they offered no way to +# perform anything in the background? + +Recently I needed to work a little more in-depth with Python 3's +[[https://docs.python.org/3/library/asyncio.html][asyncio]]. On the one hand, some people (including me) might scoff at +this because it's just green threads and cooperative threading is a +model that's fresh out of the '90s, and Python /still/ has the [[https://wiki.python.org/moin/GlobalInterpreterLock][GIL]] - +and because Elixir and Erlang and Haskell and [[http://blog.paralleluniverse.co/2013/05/02/quasar-pulsar/][Clojure]] and [[http://docs.paralleluniverse.co/quasar/][Java/Kotlin]] +have handled async and M:N threading fine. However, it's still a +useful enough paradigm that it's already in C via [[https://github.com/libuv/libuv][libuv]], and it's in +the works for [[https://doc.rust-lang.org/nightly/unstable-book/language-features/generators.html][Rust]] (sort of... it had green threads which were removed +in favor of a lighter approach) and the [[http://cr.openjdk.java.net/~rpressler/loom/Loom-Proposal.html][JVM]] (sort of... they're trying +to do [[https://en.wikipedia.org/wiki/Fiber_(computer_science)][fibers]], not green threads). The Python folks have their own set +of complaints, like [[http://lucumr.pocoo.org/2016/10/30/i-dont-understand-asyncio/][I don't understand Python's Asyncio]]. + +On the other hand, asyncio is still preferable to manually writing +code in [[https://en.wikipedia.org/wiki/Continuation-passing_style][continuation-passing-style]] (as that's all callbacks are, and +last time I had to write that many callbacks, I hated it enough that I +[[https://haskellembedded.github.io/posts/2016-09-23-introducing-ion.html][added features to my EDSL]] to avoid it), it's still preferable to a lot +of manual arithmetic on timer values to try to schedule things, and +it's still preferable to doing blocking I/O all over the place and +trying to escape it with other processes. I found the [[https://pymotw.com/3/concurrency.html][Concurrency with Processes, Threads, and Coroutines]] tutorials to be approachable and thorough, and I highly recommend @@ -25,6 +41,7 @@ over to some other code, whereas calling "whenever" means retaining control but queuing up some code to be run in the background asychronously (as much as possible). +|-----------+-----------+-----------------------+-----------------------------------------------| | Call from | Call to | When/where | How | |-----------+-----------+-----------------------+-----------------------------------------------| | Either | Function | Now, same thread | Normal function call | @@ -35,6 +52,11 @@ asychronously (as much as possible). | | | | ~asyncio.ensure_future()~ | | Either | Function | Now, another thread | ~.run_in_executor()~ on ~ThreadPoolExecutor~ | | Either | Function | Now, another process | ~.run_in_executor()~ on ~ProcessPoolExecutor~ | +|-----------+-----------+-----------------------+-----------------------------------------------| + +# TODO: How do I make Pandoc render this table better? It's hardly +# usable right now because you can't see where a column starts and +# ends * Futures & Coroutines @@ -62,7 +84,7 @@ The ability to make a Future from a coroutine was mentioned above; that's [[https://docs.python.org/3/library/asyncio-task.html#task][asyncio.Task]], an implementation of [[https://docs.python.org/3/library/asyncio-task.html#future][asyncio.Future]], but it's not the only way to make a Future. -[[https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future][concurrent.futures.Future]] is another mostly-compatible way. Its +[[https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future][concurrent.futures.Future]] provides other mostly-compatible ways. Its [[https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ThreadPoolExecutor][ThreadPoolExecutor]] provides Futures based on separate threads, and its [[https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.ProcessPoolExecutor][ProcessPoolExecutor]] provides Futures based on separate processes. @@ -90,5 +112,13 @@ There is also [[https://github.com/MagicStack/uvloop\][uvloop]]. I presently ha (nor could I really use it alongside Qt), but it's helpful to know about. -# Also: What about coroutine generators? -# Are they anything special? +* Other References + +There are a couple pieces of "official" documentation that can be good +references as well: + +- [[https://www.python.org/dev/peps/pep-0492/][PEP 492 - Coroutines with async and await syntax]] +- [[https://www.python.org/dev/peps/pep-0525/][PEP 525 - Asynchronous Generators]] +- [[https://www.python.org/dev/peps/pep-3156/][PEP 3156 - Asynchronous IO Support Rebooted: the "asyncio" Module]] + +[[https://www.python.org/dev/peps/pep-0492/][PEP 342]] and [[https://www.python.org/dev/peps/pep-0380/][PEP 380]] are relevant too.