Tracking Issue for #![feature(async_iterator)]

[yoshuawuyts]

This is a tracking issue for the RFC "2996" (rust-lang/rfcs#2996).
The feature gate for the issue is #![feature(async_iterator)].

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

• Implement the RFC (cc @rust-lang/XXX -- can anyone write up mentoring
  instructions?)
• Adjust documentation (see instructions on rustc-dev-guide)
• Stabilization PR (see instructions on rustc-dev-guide)

Unresolved Questions

• Clarify the panic behavior of Stream and Iterator Add core::stream::Stream #79023 (comment)
  • Add a panic section to Iterator, clarifying panic behavior. The panic behavior between Stream and Iterator should be consistent.
• Restore Stream::next. This was removed from the RFC because it prevents dynamic dispatch, and subsequently removed from the implementation. This should be resolved before stabilizing.
  • Investigate whether we can move the trait from fn poll_next to async fn next once we can use async in traits.
• Investigate as part of keyword-generics whether we can merge Iterator and AsyncIterator into a single trait which is generic over "asyncness".
• Should we name this API AsyncIterator instead? Tracking Issue for #![feature(async_iterator)] #79024 (comment)

Implementation history

• PR introducing core::stream::{Stream, Next}
• Rename from Stream to AsyncIterator

[kaimast]

Is there a plan to include a core::stream::StreamExt (similar to the one in the futures crate) as well?

[yoshuawuyts]

Is there a plan to include a core::stream::StreamExt (similar to the one in the futures crate) as well?

The plan is to add methods directly onto Stream much like methods exist on Iterator, but we want to do so in a way that won't cause ambiguities with ecosystem-defined methods in order to not accidentally break existing codebases when upgrading to newer Rust versions.

[joshtriplett]

Some discussion on Zulip raised the question of naming. With full acknowledgement to the fact that the name Stream has a long history in the async ecosystem, multiple people observed that something like AsyncIterator or similar might be much more evocative for new users. Such a name would allow people to map their existing understanding of iterators. "I understand Iterator, and I understand async, and this is an async version of Iterator".

[brainstorm]

As a new, inexperienced user, I find AsyncIterator way more foreign than Stream, to be honest... perhaps I'm not too involved in compiler discussions and I don't see how the jargon clicks together though :-S ... also, blogposts are already being written about Stream, so changing the name mid-flight will only breed confusion, I reckon?

[yoshuawuyts]

Prior art on "iterator" and "async iterator" naming schemes in other languages:

• JavaScript: Symbol.Iterator, Symbol.AsyncIterator
• C#: IEnumerable, IAsyncEnumerable
• Python: __iter__, __aiter__
• Swift: Sequence, AsyncSequence

[bbros-dev]

Context: We're prototyping some simple CLI app functionality. We're following the mini-redis example code where we can.

We've bumped out head on streams, and the need for the crates async-streams, and parallel-streams.

In our experience some *Iterator terminology would have helped clarify what streams are.

Given the need for the crates we cited, especially the async-streams RFC, we wonder if there isn't a need for:

• Iterator
• ConcurrentIterator
• ParallelIterator

Or is the intention that async-stream and parallel-stream crate efforts all able to converge into a stream that covers the concurrent and parallel use cases?

[benkay86]

Rayon provides a whole ecosystem of parallel iterators on top of a work-stealing threadpool, and is currently the de facto Rust standard for parallel iteration. But I don't foresee a parallel iterator trait getting into the Rust standard library anytime soon.

Streams are supposed to cover the use case of concurrent iterators only (per my understanding). Hopefully once streams are stabilized into the Rust standard library we can have some syntax to use them in concurrent for loops just like we currently use synchronous iterators in for loops. However, there are still some issues to work out like what to do if a a stream panics or is dropped. Settling on a name (Stream vs AsyncIterator vs ConcurrentIterator) will be the easy part! 😜

Unfortunately, it's difficult to combine parallel and concurrent iteration at the moment. This would require Rayon to support a way to move tasks from worker threads to an async executor thread, or for an async executor like Tokio to support parallel thread pools in a more sophisticated way than tokio::task::spawn_blocking(). Until that happens, most programmers try to get all their data into memory first on a concurrent executor-driven threadpool and then offload the synchronous computation to a parallel threadpool (e.g. managed by Rayon).

[yoshuawuyts]

multiple people observed that something like AsyncIterator or similar might be much more evocative for new users. Such a name would allow people to map their existing understanding of iterators. "I understand Iterator, and I understand async, and this is an async version of Iterator".

I've filed a PR to the RFCs repo, updating the "streams" terminology to "async iterator" instead: rust-lang/rfcs#3208.

[noelzubin]

why did .next make it. where can i get more info on this ?

[clarfonthey]

One potential concern re: methods on AsyncIterator, which is less of a concern and more of a justification for requiring them, is methods that use internal versus external iteration.

For example, it can be very efficient to perform a fold on an iterator which is composed of several chains, but it can be much less efficient to directly call next on said iterators.

My concern is that without parity between the methods on Iterator and AsyncIterator, much of these optimisations that already exist for Iterator will be lost in the conversion to AsyncIterator. Since, as it stands, there's not really a way to run a for_each or fold on a regular iterator if the loop contains async operations.

I know for a fact that the current async ecosystem with futures::stream::Stream does not have parity with Iterator, with some notable surprises including the fact that try_fold requires the iterator item to also be composed of results, rather than just the return type of the function.

It would be extremely detrimental to the idea that "this is just the async version of an iterator" if there weren't parity there IMHO, since users might notice slowdowns in code that simply calls std::async_iter::from_iter without adding any extra async code at all.

[yoshuawuyts]

@clarfonthey yes, definitely. The concerns you raise are valid, and the working group is currently actively investigating how we can ensure parity between sync and async Rust APIs. We don't yet (but should) have guidelines on how methods on async traits should be translated from sync to async, but that likely needs us to land async closures / async traits first.

[joshtriplett]

Given the current trend of the async working group, I wouldn't be surprised if this can become async fn next rather than fn poll_next.

[yoshuawuyts]

@joshtriplett ah yes, that's definitely something we've been discussing within the working group, and we're currently working towards enabling that. The other thing we're currently researching is keyword-generics, which may allow us to merge the separate Iterator and AsyncIterator traits into a single Iterator trait which is generic over "asyncness".

I'll update the tracking issue to reflect both these items.

[withoutboats]

(NOT A CONTRIBUTION)

Given the current trend of the async working group, I wouldn't be surprised if this can become async fn next rather than fn poll_next.

ah yes, that's definitely something we've been discussing within the working group, and we're currently working towards enabling that.

I think this is not the right direction for this feature. I hold this view very strongly. AsyncIterator::poll_next enables library authors to write explicit, low-level async iteration primitives for unsafe optimizations. Relying on the compiler generated futures of an async function will make the layout of these types much less predictable or controllable and will take this away from users.

For ease of use where this fine-grained laying control is not desirable, generators are the play (async or otherwise), rather than having to write next methods (async or otherwise) at all.

You need to have the low level APIs (Future::poll, Iterator::next and AsyncIterator::poll_next) for hand-rolled optimized code that the compiler can't be relied on to generate. You need to have the high level syntax (async functions, generators, and async generators) for when people just want to get things done and don't care about these kinds of optimizations. You need both. An async next method would be doing each side of it only halfway (low-level iterator, high-level asynchrony), and would basically trap Rust in a local maxima that looks appealing from where we are now but would not be the best final state.

EDIT: What I mean when I say that "generators are the play" and "local maxima" is that I think because Iterator::next has always existed and has a superficially simple API (ie no Pin, no Context), its not as obvious that for ease of use implementing an iterator with a next method is actually an awful experience for users. Yielding from generators would be much easier. So when you want the ease-of-use story, you want generators, and those can be made async just as easily as functions can. But generators can't guarantee the representation that gets the codegen from for loops over slices looking so good, and similarly won't guarantee the optimizations some async code will want as well. You should be thinking of implementing Iterator::next as really as low-level as implementing Future::poll.

---

EDIT2: I think the counterargument to this is that mixing-and-matching high-level and low-level is also desirable. IE next + async is desirable when you want control over iteration but don't care about control over asynchrony. Analogously, there must be a hypothetical API that's control over asynchrony but compiler generated iteration - a polling generator(??). I think there could be a case to be made that users do want to be able to drop down into fine control over one aspect of their control flow but not the other, but then that should be an additional, third (and fourth?) option in addition to full control or full ease of use, you can't get rid of the full-control option, which is poll_next.