RFC: stabilize `std::task` and `std::future::Future`

[aturon]

This RFC proposes to stabilize the library component for the first-class async/await syntax. In particular, it would stabilize:

• All APIs of the std-level task system, i.e. std::task::*.
• The core Future API, i.e. core::future::Future and std::future::Future.

It does not propose to stabilize any of the async/await syntax itself, which will be proposed in a separate step. It also does not cover stabilization of the Pin APIs, which has already been proposed elsewhere.

This is a revised and significantly slimmed down version of the earlier futures RFC, which was postponed until more experience was gained on nightly.

Rendered

RFC status

The following need to be addressed prior to stabilization:

• Detailed experience reports
  • #2592 (comment)
  • #2592 (comment)
  • #2592 (comment)
• Improved API documentation
• Finalizing module structure (possibly nesting within a single top-level module)
• Fuller rationale/improvements around wakeup APIs
• Establish clear plan of record for task locals

[aturon]

cc @rust-lang/lang -- I haven't tagged this as T-lang, since the Lang Team already approved the async/await RFC and this is "just" about library APIs. But of course y'all should feel free to weigh in.

cc @Nemo157 @MajorBreakfast @tinaun @carllerche @seanmonstar @olix0r

[aturon]

@cramertj can you comment on UnsafeWake and whether waiting to stabilize that piece looks problematic?

[aturon]

cc @rust-lang/libs, please take a look!

[aturon]

@cramertj I only briefly mentioned Fuchsia in the RFC, but it might be helpful for you/your team to leave some commentary here about your experience with the various iterations of the futures APIs.

[Ixrec]

Since then, the syntax, the std APIs, and the futures 0.3 crate have all evolved in tandem as we've gained experience with the APIs. A major driver in this experience has been Google's Fuchsia project, which is using all of these features at large scale in an operating system setting.

Although this isn't strictly relevant to the technical merits of the proposed APIs, considering the sheer scope and history of what we're talking about adding to std it seems worth asking: Are there any blog posts discussing Fuchsia's experience in more detail? This is the only part of the historical context I was completely unaware of, and I couldn't find any place that talks about it.

EDIT: I swear I started typing this before aturon's last comment 😅

[carllerche]

Thanks for putting this together.

My experience with the proposed Future trait is that it makes sense as an infrastructure level detail. In a world where the vast majority of async code is written with async / await, the trait makes sense. However, async / await is not there yet. In my experimentation to port existing code to use async / await, I hit limitations pretty quickly. Specifically, as far as I could tell, it is not possible to use a transport properly without extra allocation (due to split).

If the proposed Future is expected to be used be implemented by hand significantly, then moving to it is a net negative. The RFC mentions the ergonomic hit.

Because of this, my plan for Tokio will be to stick on futures 0.1 until async / await has reached a point where implementing Future by hand is no longer required for the user.

Also, most of Tokio would require the UnsafeWake trait, so even with the proposed stabilization, Tokio would not be able to migrate to it.

I understand the desire to drive progress forward. As I said, as far as I can tell, the proposed Future trait is good as an implementation detail of async / await. The ecosystem split will be unfortunate, but perhaps there is no other way.

Edit: I should clarify, Tokio will add support for async / await as it stabilizes, but it will not be considered the primary abstraction until it is able to handle all the cases.

[aturon]

@carllerche You and I have talked about this a bunch on other channels, so I'll be repeating myself, but I want to write a response here so that everyone else is on the same page as well.

There are indeed limitations with async/await today, due not so much to the feature itself as the lack of impl-trait-in-traits (or existential types) working sufficiently well (as well as, ultimately, GATs). They limit the ability to move foundational libraries to use async/await internally, and that's part of the reason we're not ready to stabilize the syntax itself yet. However, to be clear, none of these limitations connect to the Future or task APIs. (And, of course, we also have large-scale usage of these APIs and the current async/await mechanism in Fuchsia to draw from; the limitations largely apply to highly generic code.)

The 0.1/0.3 compatibility system, which allows for fine-grained/incremental migration, ends up doing a lot to lower the stakes. For example, it's already fairly painless to write code for hyper using async fn, and with a little bit more polish it could feel completely "first-class". So, it seems fine for some low-level libraries to stick with the 0.1 model until the above limitations are sufficiently lifted.

I think everything else you raise is discussed in the RFC as well, so I don't have more to add there!

[nrc]

What’s the rationale for having both task and future modules? Since future only includes Future, it seems that having two modules doesn’t pull its weight. Are we expecting to move a bunch of future stuff into std in the future?

[Redrield]

@nrc I'm not sure but perhaps the Iterator-like adapters talked about would be merged into that module once they make their way into std?

[ivandardi]

If Future is a trait then why isn't this example in the RFC

async fn read_frame(socket: &TcpStream) -> Result<Frame, io::Error> { ... }

written as

async fn read_frame(socket: &TcpStream) -> impl Result<Frame, io::Error> { ... }

?

[rpjohnst]

A small bikeshed: Waker and LocalWaker might be better as SyncWaker and Waker, with the un-marked name being the local one by analogy to Arc and Rc. This makes the poll method signature a bit more straightforward to read and aligns with existing naming convention. I also think there might be a better and more noun-y name than Waker- maybe Task/task::Handle/etc, similar to 0.1?

I'm not sure I totally understand all the layers of &LocalWaker, NonNull<dyn UnsafeWake>, &Arc<T> where T: Wake, etc. and it seems like it might still be in flux, but exactly how many layers of indirection do we have here? Ideally there would be one (as if poll took Arc<dyn Wake> or equivalent), though I can sort-of see why there might need to be two (as if poll took &Arc<dyn Wake>), but it almost looks like there are three?

[glaebhoerl]

Is this the same "trampoline" concept which is used in the context of avoiding stack overflows for recursive calls?

[aturon]

Yep!

[Nemo157]

@ivandardi that's related to the async/await syntax rather than the library code provided here. The design as I understand it is that async fn itself introduces the unnamable type and so entirely encompasses and hides the impl Future part of the transformed function signature. If you want to discuss further the async/await tracking issue would be the more appropriate location.

[aturon]

@nrc

What’s the rationale for having both task and future modules? Since future only includes Future, it seems that having two modules doesn’t pull its weight. Are we expecting to move a bunch of future stuff into std in the future?

Yes, both modules are expected to grow substantially over time. The futures crate contains a similar module hierarchy with a much richer set of APIs. In addition, there will eventually be a stream module. Generally in std we tend to have a pretty flat set of top-level modules supporting various types, e.g. std::option and std::result.

[jethrogb]

These don't look like trait objects to me.

[aturon]

They're trait objects internally.

[jethrogb]

Not right now, because dyn Wake is not Wake.

[aturon]

Should be Arc<impl Wake> I suppose.

[jethrogb]

No behavior is specified for when clients do do that. I think we should say something. For example, "implementors may panic".

[Nemo157]

I think we could even go a bit stronger than that, “implementors should panic, but clients may not rely on this”. All async fn futures guarantee this, and I believe so do the current futures 0.3 adaptors.

I think it would also be good to mention that calling poll again must not cause memory unsafety. The current mention that it can do anything at all makes it seem like it is allowed to have undefined behaviour, but since this is not an unsafe fn the implementer cannot rely on the client’s behaviour for memory safety purposes.

[jethrogb]

It's unclear at first glance which of the code blocks starting with trait Wake/struct ExecutorInner/struct Waker are proposed for stabilization.

[brain0]

(Probably none of you know me, yet I'd still like to offer my opinion, if that is appropriate.)

It is my understanding that the purpose of having an unstable API is that the ecosystem can experiment with it to ultimately avoid stabilizing a bad API. I don't see that this has happened here. Tokio has created a shim that essentially wraps an "std future" into a "0.1 future" with the only purpose of allowing async/await style futures. Apart from that, I haven't seen any experimentation with the std::future API. If tokio (as indicated above) is not even planning to use the new API instead of the old futures 0.1 API, then stabilizing it as-is will IMO be very bad for the ecosystem.

The situation for std::task is worse: From what I can see, it hasn't been used at all. The tokio shim merely provides a noop waker to satisfy std::future's poll signature, but that waker cannot be used and even panics when you try to. I'd like to see any implementation that actually uses std::task - I've been following TWIR all year and haven't found anything. I cannot see that there are comprehensive examples in the docs for std::task, or any reference implementations that show how the system is meant to be used as a whole.

My information is probably incomplete, so please tell me if I am missing anything.

As a side note, I started implementing an "as simple as possible" task executor based on the std APIs, just to understand them and play with them. I found the std::task stuff really complicated, and quickly realized that it still couldn't do everything I needed - most importantly, I needed to access some of the internal data in my Wake implementation, but this was not possible with LocalWaker. I would have to resort to storing information in thread-locals again, which defies the purpose of having the waker passed as an argument to poll.

[steveklabnik]

Tokio is not the only user of futures, Fuchsia uses this API, including writing their own executor.

…

On Nov 11, 2018, at 1:28 PM, brain0 ***@***.***> wrote: (Probably none of you know me, yet I'd still like to offer my opinion, if that is appropriate.) It is my understanding that the purpose of having an unstable API is that the ecosystem can experiment with it to ultimately avoid stabilizing a bad API. I don't see that this has happened here. Tokio has created a shim that essentially wraps an "std future" into a "0.1 future" with the only purpose of allowing async/await style futures. Apart from that, I haven't seen any experimentation with the std::future API. If tokio (as indicated above) is not even planning to use the new API instead of the old futures 0.1 API, then stabilizing it as-is will IMO be very bad for the ecosystem. The situation for std::task is worse: From what I can see, it hasn't been used at all. The tokio shim merely provides a noop waker to satisfy std::future's poll signature, but that waker cannot be used and even panics when you try to. I'd like to see any implementation that actually uses std::task - I've been following TWIR all year and haven't found anything. I cannot see that there are comprehensive examples in the docs for std::task, or any reference implementations that show how the system is meant to be used as a whole. My information is probably incomplete, so please tell me if I am missing anything. As a side note, I started implementing an "as simple as possible" task executor based on the std APIs, just to understand them and play with them. I found the std::task stuff really complicated, and quickly realized that it still couldn't do everything I needed - most importantly, I needed to access some of the internal data in my Wake implementation, but this was not possible with LocalWaker. I would have to resort to storing information in thread-locals again, which defies the purpose of having the waker passed as an argument to poll. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[brain0]

Right, sorry, I must have overlooked that, it's even mentioned in the RFC. Is that stuff open source? I'd love to look at it.

[krircc]

I find this https://github.com/fuchsia-mirror/docs/blob/master/development/languages/rust/crates.md

[brain0]

Someone on reddit found this: https://fuchsia.googlesource.com/garnet/+/master/public/rust/fuchsia-async/src/ (executor.rs is interesting, for example).

[aturon]

@brain0 after the weekend, I expect that @cramertj (or others from the Fuchsia team) will write in with more extensive detail about their experiences.

The RFC also went to some length to lay out the historical context. These APIs have seen plenty of use, both on top of Tokio/Hyper (through various shims) in e.g. web framework code, in embedded settings, and in custom operating systems (Fuchsia).

Could you spell out your concern re: the task system? It'd be helpful to keep discussion focused on specifics if possible.

[brain0]

@aturon First things first: Last weekend, I decided to try out the std::task and std::future system by implementing the simplest task executor I could think of, then combine that with mio. Of course, the result would not be as feature-rich or performant as tokio, but it would demonstrate the new APIs.

There were lots of little details that felt "weird" about the task system:

• When polling a future, you need a LocalWaker. From the documentation, it takes quite a while to figure out how to actually construct a LocalWaker using only safe code. The only way to do so is the local_waker_from_nonlocal function. I would have expected an inherent method on LocalWaker. But when you read the LocalWaker docs, you only find unsafe fn new(inner: NonNull<dyn UnsafeWake + 'static>). The whole API feels centered around the unsafe features and the safe features are "second class".

• What we all love about Rust is how its type system ensures thread safety. But somehow, when creating a LocalWaker, you either need to call an unsafe function (local_waker) or construct a waker that does not take advantage of being a local waker (local_waker_from_nonlocal). I don't know if there is a better way, but this whole API feels un-rust-y.
  What could be done here is have two traits LocalWake and Wake (where LocalWake is not necessarily Send + Sync) and create a LocalWaker from Rc<dyn LocalWake>. This LocalWaker could then be transformed to a Waker using a method on LocalWake that returns Arc<dyn Wake>. Of course this would be more code and probably has other downsides and issues. Still, it would feel more like Rust, since the compiler verifies thread safety for us again. (Btw, fuchsia doesn't implement wake_local() either, is there any project that actually implements wake_local() to optimize local task wakeups compared to wake()?)

• Lack of module-level documentation: Usually in Rust, the module-level documentation explains the "big picture" in quite some detail. The std::task documentation is "Types and Traits for working with asynchronous tasks." Only reading the API documentation (starting with LocalWaker, because that's what I needed for poll) didn't help me that much. It's clearly obvious to the people who designed the system, but not so for everyone else. I understand that std::task is new, but I imagine such documentation should exist before stabilization, not after. Due to the lack of documentation, I looked for projects that actually implemented this API and didn't find any (since I did not find the fuchsia executor). Now that I read some of the fuchsia code, I think I put the pieces together correctly.

• Related to the first and third points above: I tried to create a waker using only safe code. As it turns out, the only way to do that is to put a Wake implementation into an Arc. Inside the Wake implementation, I need two things: A way of identifying the task that should be woken and some kind of reference to the executor itself. The latter has to be wrapped inside an Arc again, so you effectively get something like Arc<(Identifier, Arc<Executor>)>. This felt wrong at first, since it was atypical of Rust to force me to nest two Arcs. A small example in the docs would have helped here. Reading the fuchsia code earlier today showed me that I was not entirely wrong, since they're doing the same thing.

• I looked into integrating std::task with the mio crate to do some basic networking to see if my (naive) executor would even work. In mio, to identify which Evented received an event, you get a Token (a newtype around a usize). Of course, I could maintain a map Token->Waker. However, since the mio::Poll is tied to my executor, I thought it would be useful to just extract some internal identifier from my Wake implementation and use that as Token. However, I cannot downcast the Waker's internals to its original type to get that information. (Maybe that problem is mio's fault - mio::Poll could be generic on the Token type, so I could simply use my Waker as a token.)

Sorry for the rather verbose reply, I hope it still helped you understand my concerns.

[Matthias247]

@aturon

Just a quick note here. The rough plan is that, eventually, we will have for await or some such syntax for consuming streams. IOW, Stream in some form will likely be equally "primitive".

Even if we do so, we might get to 2 low-level primitives/traits. That doesn't seem to justify requiring another module. I think they both would just exist fine inside the highest level of one async create (like core::task). And I'm still convinced Stream should be built on top of Future (in the same fashion as .NET IAsyncEnumerator and IAsyncEnumerable are build on top of Task), in order to make it implementable in terms of other Futures and async fns.

It is not possible to implement the current future 0.1 Streams on top of those, since they would not be kept-alive between calls to poll_next - which is required for a lot of Futures.

[withoutboats]

@brain0 It doesn't seem unreasonable for Waker to support downcasting, however I think if you're using the RawWaker interface, it makes more sense to have an API to get the RawWaker out of the Waker, and then compare the address of the wake pointer to the address of your wake pointer. If you're using the RawWaker API, you don't even necessarily have a waker type to downcast to, you just have a struct with some fn pointers in it.

Once we've provided a higher level Wake trait, we could provide a downcasting mechanism which attempts to downcast to a T: Wake, which is the same useful functionality as "cast to raw waker, check the wake pointer equality" but more ergonomic.

@cramertj @Matthias247 I'd be interested in your thoughts on providing some sort of Option<TypeId> field of RawWaker to support a downcasting interface for Waker. Does that make sense?

[carllerche]

I would like the RFC to specify if executor implementations are permitted to "inline" polling a task in the Waker::notify call or if it is specifically forbidden to do so.

Being able to do so would provide significant performance improvements in some cases, but would impose restrictions on Mutex limitations.

This is something that would have to be defined up front as I do not believe it would be backwards compatible to add that ability.

[brain0]

@withoutboats That would help too, but all you'd get would be a *const (). My idea was that this would be "sort of" type safe, i.e. access to whatever is behind the *const () would be restricted to the implementation of the RawWakerVTable.

[carllerche]

This paragraph is not obviously true to me. Could a specific example be provided showing how a raw vtable works better than a trait object? I can't think of a situation in which manually defining the vtable can unlock a capability vs. a trait object.

Additionally, this API seems to provide a significant forwards compatibility hazard. With traits, additional fns can be added w/o breaking changes by including a default implementation. I do not see how this can be accomplished given the raw vtable.

[Centril]

Additionally, this API seems to provide a significant forwards compatibility hazard. With traits, additional fns can be added w/o breaking changes by including a default implementation. I do not see how this can be accomplished given the raw vtable.

This can be solved by using #[non_exhaustive] as well as exposing a constructor function on RawWaker that takes the minimum required things (i.e. the things required today).

[carllerche]

@Centril

While I agree the API can be made forwards compatible, it is still unclear to me why the raw vtable is being used instead of trait objects.

If the raw vtable is used, the RFC should be updated to:

• Include examples illustrating why the raw vtable is a superior option.
• Make the API forwards compatible.

[jethrogb]

How would you write the trait object version of RawWaker without using Box?

[carllerche]

@jethrogb that entirely depends on the details of the executor.

[carllerche]

Assuming you are referencing the mutability issue, it could most likely be *const RawWake instead of *mut.

Otherwise, I'm not sure what wart you are talking about.

I'm not sure how manually defining a vtable (which is basically manually creating a trait object) is easier to grok than just using a trait object. Either way, the RFC states the reasoning as:

it allows for more flexible memory management schemes.

and this is the point that I am disputing.

[jrobsonchase]

This one:

    /// FIXME(cramertj)
    /// This method is intended to have a signature such as:
    ///
    /// ```ignore (not-a-doctest)
    /// fn drop_raw(self: *mut Self);
    /// ```
    ///
    /// Unfortunately in Rust today that signature is not object safe.
    /// Nevertheless it's recommended to implement this function *as if* that
    /// were its signature. As such it is not safe to call on an invalid
    /// pointer, nor is the validity of the pointer guaranteed after this
    /// function returns.

Or is that no longer relevant and just failed to get removed?

[carllerche]

@jrobsonchase IMO that wart seems relatively minor in context.

The forwards compatibility issue is real IMO. @Centril called out future proofing this. I assume that would require the constructor to have const fns? And, every time there is a change, creating a new constructor?

[Matthias247]

One other important conceptual difference is that the raw vtable doesn't imply that much that we are talking about a real object. E.g. current executors/wakeups are implement through various mechanisms:

• As global functions
• As member functions on refcounted objects
• As thread-local functions.

All of those could be modeled with UnsafeWake too, but the casts of that some things into that trait were not really a natural fit. It seemed like mostly people with good experience on how trait objects look like in memory could implement it.

With traits, additional fns can be added w/o breaking changes by including a default implementation. I do not see how this can be accomplished given the raw vtable.

I don't see compatibility issues. Either the default implementation can be added through new constructors, or the relevant field on RawWakerVTable simply stays null, and Waker calls the default method when the thing is not populated.
In the C world this approach is even used to extend things in an ABI compatible way - which is something we even don't require at the moment.

[jethrogb]

I think you can do all of this with trait objects, you just need a zero-sized type in some of those cases.

[seanmonstar]

@cramertj

Waker vs Context

I agree that adding new fields would mean they'd need to be optional, but that seems much better than currently, which is that nothing new can be added ever.

As an example, Kotlin's CoroutineContext allows for users to add additional things besides just the continuation: a scope, scheduler, dispatch strategy, and extras that are essentially like task-locals.

[aturon]

@carllerche

I would like the RFC to specify if executor implementations are permitted to "inline" polling a task in the Waker::notify call or if it is specifically forbidden to do so.

Being able to do so would provide significant performance improvements in some cases, but would impose restrictions on Mutex limitations.

Can you elaborate on this? I'm not able to grasp the request.

[aturon]

@seanmonstar

I agree that adding new fields would mean they'd need to be optional, but that seems much better than currently, which is that nothing new can be added ever.

I don't follow this. Ultimately the definition of Waker itself is opaque, and it is always possible to provide additional constructors/builders later. Can you elaborate on the problem you see there?

[tikue]

@aturon

Can you elaborate on this? I'm not able to grasp the request.

I believe @carllerche is imagining an implementation that looks like a callback. Instead of appending to a task queue, it immediately calls poll() on the woken task.

[carllerche]

@aturon what @tikue said.

wake is implemented to immediately call poll().

I've been exploring scheduler heuristics and there is measurable value in having a heuristic that does ^^ in some cases (not unconditionally).

[aturon]

@carllerche Got it, thanks! That does seem important to clarify 👍

[Matthias247]

wake is implemented to immediately call poll().

A strong no from my point of view. The ability to call a continuation inline (instead of scheduling it) is one of the biggest sources of deadlocks, call stack overflows and other reentrancy isssues in things like C# Tasks, which allow for that.
JS allows for that initially too with callbacks, but removed it in async await, which always runs continuation in a fresh iteration of the eventloop.

I am totally aware of potential performance gains. But very scared around hidden issues, and non deterministic behavior (if one executor is swapped out for another which makes that kind of decisions).

[carllerche]

@Matthias247 fwiw, I have only inlined the poll w/ a depth of 1. So, there is no risk of overflow.

[cramertj]

@carllerche

I would like the RFC to specify if executor implementations are permitted to "inline" polling a task in the Waker::notify call or if it is specifically forbidden to do so.

Presumably this would include the restriction that the task being awoken cannot be a task that is currently being polled (e.g. a task polling itself)? If so, I personally have no strong objection to this. However, it does seem like something that would require very clear documention, such as noting to never .wake() while holding locks. This would require users to lock -> get waker -> unlock -> wake, which could potentially be more expensive than the alternative if there are multiple wakers being stored behind a single mutex which all need to be awoken.

I checked and I don't personally couldn't find any code that couldn't be adapted to follow this pattern, but I certainly have code that violates these rules today.

For the community: does anyone know of code that needs to call wake() while holding a lock for semantic / correctness reasons? (note that this is easy to do by accident and sometimes hard to catch, such as sending a message into an unbounded channel while holding a lock...) The potential footgun-iness makes me wary.

[Matthias247]

@carllerche Ok, that's a reasonable guard against stack overflows. However we still would face the other issues.

@cramertj

For the community: does anyone know of code that needs to call wake() while holding a lock for semantic / correctness reasons?

Yeah, the futures-intrusive stuff I've worked on unfortunately requires it. Because outside of the lock, the field which holds the Waker isn't guaranteed to be alive.
I could wake up one target task by copying the Waker out of the section before waking, but not a list of those.

[Matthias247]

@withoutboats

@cramertj @Matthias247 I'd be interested in your thoughts on providing some sort of Option field of RawWaker to support a downcasting interface for Waker. Does that make sense?

I don't see a benefit of that. You can downcast inside the functions that get stored in the vtable, which is what will typically happen. Adding any other query functions to Waker seems to remove the opaque attribute from it.

[Nemo157]

Interesting question just raised on i.rl.o, should core::task::Poll be #[must_use] to reduce chance of accidentally dropping Poll::Pending (or is there some other way to deal with that in those sorts of situations)?

[Matthias247]

@Nemo157 I don't know exactly about that one, but encountered a related question:

If APIs return concrete Futures, we can use must_use, to decrease the probability that users forget to compose or poll them. However if the APIs return impl Future, there is not such a way. I wonder whether and how that can be added. I guess with special compiler support for sure, but maybe there are other ways too.

[Nemo157]

@Matthias247 luckily #[must_use] on traits was added recently, not using an impl Future type gives a decent message:

warning: unused implementer of `std::future::Future` that must be used
 --> src/main.rs:8:5
  |
8 |     foo();
  |     ^^^^^^
  |
  = note: #[warn(unused_must_use)] on by default
  = note: futures do nothing unless polled

[Matthias247]

@Nemo157 Oh cool, I totally missed that. Last time I tried it didn't work.