Barbara considers the ecosystem challenges of writing a ?Send executor without providing an entire std-like interface

[jbr]

Brief summary

Background: Barbara considers writing a new futures-compatible executor to fill a niche in the executor landscape
Aspect relevant to wg: Barbara hesitates / decides not to, for reasons that primarily are ecosystem and interop related

I'm writing this as an issue before writing the story because it's not clear to me if this is a distinct story as it is a mirror of a lot of the other stories that already have been well expressed. It may very well be a duplicate.

This is a real life story: I was reading #87 and realized that a fair-ish¹ futures-compatible multi-threaded executor for ?Send futures would be perfect for web servers. However, there are ecosystem hurdles that make this substantially more challenging than writing the code:

• Culturally/socially, introducing more executors currently seems like it further fragments a confusing landscape
• Because async-std is the primary futures-compatible executor, there isn't a good way to communicate about being futures-compatible but not async-std. Other crates that would work on a futures-compatible executor tend to have a tokio feature and an async-std feature, making the education story confusing for new executors. Sometimes the async-std feature pulls in async-std, and sometimes it just pulls in futures/futures-util/futures-lite. Code that pulls in async-std but doesn't spawn is probably fine to mix and match with a new executor, but that's a lot of hidden sharp edges to offer support for. Smol users who are not async-global-executor/async-std users currently have this challenge (to the extent that they exist).
• Introducing a new standalone executor that doesn't have an entire standard-library associated with it is challenging. It's hard for people to know which crates they could use in conjunction with a standalone executor, since both of the primary executors have taught people that an executor/runtime comes along with kitchen sink std-like libraries. Smol/async-executor/async-global-executor are the exception to this, but that's not currently the dominant model. Code that's written with the assumption of a sprawling library that also has a global executor often spawns tasks without treating that as a "special" boundary that is meaningfully different from anything one might do within an async task / spawned future.
• This one I'm the least technically sure about: Because async-trait boxes the futures, the user needs to know if their futures are Send (the default) or ?Send. This means it's difficult to use async trait futures in a library context that sometimes will be used in a Send executor but not always, as the actual Send-ness gets erased when it's object-ified.

Shiny opinionated future: Any character can start with a simple/standard executor and swap it out for another one without having to rewrite their entire application. Replacing libraries with ones specific to their new executor will often provide a performance benefit, but can be done over time as needed, not as a mandatory wholesale change when switching executors. Example: Alan starts with async-std's executor and a bunch of executor-independent libraries, writes a substantial application against those types and then decides he really wants the perf characteristics of tokio, so he switches just the executor and sees a benefit for his application. Later on, he finds some time to replace the networking stack with a tokio-tuned one, and sees further benefits. Everything else still is using the neutral executor-independent libraries that provide lowest-common-denominator performance. Similarly, if in a different codebase Alan gets a compiler warning about Send types but isn't sure that spawn_local has the right perf characteristics for his application, he can swap out the executor and everything else still works. Obviously either of these switches to applications-specific executors wouldn't be as seamless to reverse.

Is it worth writing this as one or more stories?

Optional details

• (Optional) Which character(s) would be the best fit and why?
  • Alan: the experienced "GC'd language" developer, new to Rust
  • Grace: the systems programming expert, new to Rust
  • Niklaus: new programmer from an unconventional background
  • Barbara: the experienced Rust developer

¹ fair-ish: creates new !Send futures on whichever runtime thread has the least of them, or some other similar strategy, acknowledging that this is more likely to become lopsided than a work-stealing multithreaded executor of Send futures would be, especially if the spawned tasks take an unpredictable wall-time duration. On that note, however, an optional duration_hint function on the Future trait might be useful to executors

[nikomatsakis]

@jbr I for one would very much like to read more stories about this. I'm not of exactly how they compare to the existing stories, but I still feel like we haven't really covered the "portability" space very fully, and this definitely touches on some of those aspects.

One clarification:

'futures-compatible'

After many reads of this sentence, I realized that you meant compatible with the futures crate, right?

[jbr]

After many reads of this sentence, I realized that you meant compatible with the futures crate, right?

exactly that. futures/futures-util/futures-lite/async-std/smol all share a bunch of types that I was broadly calling "futures-compatible"

[uazu]

I'm coming from the point of view of interfacing the Stakker actor runtime to the async/await ecosystem as a single-threaded executor. As I see it, I don't want to give up the benefits of Stakker which suits the work I do really well, but I would like to bring in third party code that uses async/await, where possible without running it on another thread with its own executor (which is still an option if all else fails).

There should be a number of tasks which require async/await but don't actually need to do I/O directly themselves, for example protocol handlers or whatever. (Whether people have actually structured their crates to work that way or not is a separate question). But in principle it should be possible for there to exist an ecosystem of executor-independent crates that do useful work and can share effort between the different executor-specific ecosystems.

From my point of view, I'm interested in what level of support executor-independent crates might need. Here are some possible things to consider:

• Just Future and Stream (i.e. piping data in/out)
• Waker (for cross-thread wakeups)
• Spawning new jobs
• Setting up timers (from Instant)

If there was a clear API for that (and whatever other common foundation is decided on), then that would enable an ecosystem of executor-independent crates, and give someone with a runtime that they want to interface to the async/await ecosystem a clear target to implement.

I'm working on interfacing Stakker to the ecosystem using the existing traits and interfaces in the ecosystem, to see how I get on. But this is just in spare moments, so I'm not sure whether I'll be able to finish before the period of this review is over.

[jbr]

I'd add Async(Buf)?(Read|Write) traits to that list as absolutely essential for runtime-independent crates to be possible.

I'm not sure that timers are all that much more important than io (network and fs) and that although both would be amazing to standardize, there's probably a lot of complexity there

I think there's probably a lot of nuance hidden in that wishlist, though, since there are a bunch of flavors of spawn for each of the runtimes:

• spawn this ?Send future on this thread, wherever we are right now, and don't move it until it's done (spawn_local)
• here's a function to allocate a new ?Send future on a different thread, but don't move it from there (like actix-rt and the executor discussed above)
• spawn this Send+Sync future anywhere and move it around as whatever work-stealing algorithm considers best (spawn)
• spawn this blocking task on a special blocking thread/threadpool (spawn_blocking)
• any of the above with an associated relative task priority or other per-future hint to the executor

[uazu]

I think timers are vital for some protocols. Anything with a lossy link (UDP?) needs timeouts to retry. Things may have to time out and abort (or maybe attempt recovery) in case of remote failure. This is all independent of how exactly the stream or packet I/O is being performed. Also a timer interface's contract is relatively simple. Yes, I forgot AsyncRead/Write/etc.

I guess the question is what kind of code (or classes of code) can usefully be written as executor-independent. Then what interface do those need? This doesn't have to be a universal executor interface that covers every possible feature. Just an interface that's sufficient for the type of code that's useful to write portably.