Roll our own os.fork() server on *nix?

[goodboy]

mulitprocessing's forkserver has proven severely unreliable for nested process launching (#84, #134) however it is much much faster then our current trio (aka async wrapped subprocess) spawning backend and, as such, we're keeping around for benchmarking purposes for when we eventually tackle the building our own internal subsystem for fast process launching on linux.

The issues with current unix forking apis is covered in quite good depth in @catern's blog post and I think it's a multiple required read before diving into this.

I've had some (very) brief discussion and there's some ideas I'd like to investigate:

• benchmarking a spawner written with supervise_api
  • the test set is probably handy to read for digging into this
  • understanding the recommended use of sfork()
• looking at how we could build our own version of mp's fork request loop using sfork which does minimal tractor templated process environment copying via a specific supervisor process

I'd love to get @catern's feedback on a sensible direction with the fast forking sub-system design as well 🏄

[goodboy]

Discussion surrounding this is now happening in python-trio/trio#1614 in terms of avoiding using a server all together which would actually be the most ideal.

Heh, and full conclusion is "write your own sync fork server" 😆

[goodboy]

Motivating blog post: https://pythonspeed.com/articles/python-multiprocessing/