Tracking issue for RFC 2033: Experimentally add coroutines to Rust

[aturon]

RFC.

This is an experimental RFC, which means that we have enough confidence in the overall direction that we're willing to land an early implementation to gain experience. However, a complete RFC will be required before any stabilization.

This issue tracks the initial implementation.

related issues

[alexcrichton]

cc #43076, an initial implementation

[mitranim]

Copied from #43076:

---

I'm using this branch for stream-heavy data processing. By streams I mean iterators with blocking FS calls. Because Generator is missing an Iterator or IntoIterator implementation, you must call your own wrapper. Zoxc kindly provided an example, but it's quite unergonomic. Consider:

Python:

def my_iter(iter):
    for value in iter:
        yield value

Rust with generators:

fn my_iter<A, I: Iterator<Item=A>>(iter: I) -> impl Iterator<Item=A> {
    gen_to_iter(move || {
        for value in iter {
            yield value;
        }
    })
}

Two extra steps: inner closure + wrapper, and, worse, you have to write the wrapper yourself. We should be able to do better.

TL:DR: There should be a built-in solution for Generator→Iterator.

[silene]

I was a bit surprised that, during the RFC discussion, links to the C++ world seemed to reference documents dating back from 2015. There have been some progress since then. The latest draft TS for coroutines in C++ is n4680. I guess the content of that draft TS will be discussed again when the complete RFC for Rust's coroutines is worded, so here are some of the salient points.

First, it envisions coroutines in a way similar to what this experimental RFC proposes, that is, they are stackless state machines. A function is a coroutine if and only if its body contains the co_await keyword somewhere (or co_yield which is just syntactic sugar for co_await, or co_return). Any occurrence of co_await in the body marks a suspension point where control is returned to the caller.

The object passed to co_await should provide three methods. The first one tells the state machine whether the suspension should be skipped and the coroutine immediately resumed (kind of a degenerate case). The second method is executed before returning control to the caller; it is meant to be used for chaining asynchronous tasks, handling recursive calls, etc. The third method is executed once the coroutine is resumed, e.g. to construct the value returned by the co_await expression. When implementing most generators, these three methods would have trivial bodies, respectively { return false; }, {}, and {}.

Various customization mechanisms are also provided. They tell how to construct the object received by the caller, how to allocate the local variables of the state machine, what to do at the start of the coroutine (e.g. immediately suspend), what to do at the end, what do to in case of an unhandled exception, what to do with the value passed to co_yield or co_return (how yielded values are passed back to the caller is completely controlled by the code).

[arielb1]

One subtle point that came up is how we handle the partially-empty boxes created inside of box statements with respect to OIBITs/borrows.

For example, if we have something like:

fn foo(...) -> Foo<...> {}
fn bar(...) -> Bar<...> {}
box (foo(...), yield, bar(...))

Then at the yield point, the generator obviously contains a live Foo<...> for OIBIT and borrow purposes. It also contains a semi-empty Box<(Foo<...>, (), Bar<...>)>, and we have to decide whether we should have that mean that it is to be treated like it contains a Box, just the Foo<...>, or something else.

[masonk]

I might be missing something in the RFC, but based on the definition of resume in the Generator struct, and the given examples, it looks like these generators don't have two way communication. Ideally this language construct would allow us to yield values out and resume values into the generator.

Here's an example of implementing the async/await pattern using coroutines in ES6. The generator yields Promises and the coroutine resumes the generator with the unwrapped value of a Promise each time the Promise completes. There is no way this pattern could have been implemented without the two-way communication.

Rust has a problem here because what's the type of resume? In the ES6 example, the generator always yields out some kind of Promise and is always resumed with the unwrapped value of the Promise. However the contained type changes on each line. In other words, first it yields a Promise<X> and is resumed with an X, and then it yields a Promise<Y> and is resumed with a Y. I can imagine various ways of declaring that this generator first yields a Wrapper<X> and then a Wrapper<Y>, and expects to be resumed with an X and then a Y, but I can't imagine how the compiler will prove that this is what happens when the code runs.

TL;DR:
yield value is the less interesting half. It has the potential to be a much more ergonomic way to build an Iterator, but nothing more.

let resumedValue = yield value; is the fun half. It's what turns on the unique flow control possibilities of coroutines.

(Here are some more very interesting ideas for how to use two-way coroutines.)

[mikeyhew]

@arielb1

Then at the yield point, the generator obviously contains a live Foo<...> for OIBIT and borrow purposes. It also contains a semi-empty Box<(Foo<...>, (), Bar<...>)>, and we have to decide whether we should have that mean that it is to be treated like it contains a Box, just the Foo<...>, or something else.

I don't know what you mean by "OIBIT". But at the yield point, you do not have a Box<(Foo<...>, (), Bar<...>)> yet. You have a <Box<(Foo<...>, (), Bar<...>)> as Boxed>::Place and a Foo<...> that would need to be dropped if the generator were dropped before resuming.

[clarfonthey]

Looking at the API, it doesn't seem very ergonomic/idiomatic that you have to check if resume returns Yielded or Complete every single iteration. What makes the most sense is two methods:

fn resume(&mut self) -> Option<Self::Yield>;
fn await_done(self) -> Self::Return;

Note that this would technically require adding an additional state to closure-based generators which holds the return value, instead of immediately returning it. This would make futures and iterators more ergonomic, though.

I also think that explicitly clarifying that dropping a Generator does not exhaust it, stopping it entirely. This makes sense if we view the generator as a channel: resume requests a value from the channel, await_done waits until the channel is closed and returns a final state, and drop simply closes the channel.

[janhohenheim]

Has there been any progress regarding the generator -> iterator conversion? If not, is there any active discussion about it somewhere? It would be useful to link it.
@Nemikolh and @uHOOCCOOHu, I'm curious about why you disagree with @clarcharr's suggestion. Care to share your thoughts?

[phaux]

Has there been any progress regarding the generator -> iterator conversion? If not, is there any active discussion about it somewhere?

https://internals.rust-lang.org/t/pre-rfc-generator-integration-with-for-loops/6625

[Flupp]

I was looking at the current Generator-API and immediately felt uneasy when I read

If Complete is returned then the generator has completely finished with the value provided. It is invalid for the generator to be resumed again.

Instead of relying on the programmer to not resume after completion, I would strongly prefer if this was ensured by the compiler. This is easily possible by using slightly different types:

pub enum GeneratorState<S, Y, R> {
    Yielded(S, Y),
    Complete(R),
}

pub trait Generator where Self: std::marker::Sized {
    type Yield;
    type Return;
    fn resume(self) -> GeneratorState<Self, Self::Yield, Self::Return>;
}

(see this rust playground for a small usage example)

The current API documentation also states:

This function may panic if it is called after the Complete variant has been returned previously. While generator literals in the language are guaranteed to panic on resuming after Complete, this is not guaranteed for all implementations of the Generator trait.

So you might not immediately notice a resume-after-completion at runtime even when it actually occurs. A panic on resume-after-completion needs additional checks to be performed by resume, which would not be necessary with the above idea.

In fact, the same idea was already brought up in a different context, however, the focus of this discussion was not on type safety.

I assume there are good reasons for the current API. Nevertheless I think it is worth (re)considering the above idea to prevent resume-after-completion. This protects the programmer from a class of mistakes similar to use-after-free, which is already successfully prevented by rust.

[Nemo157]

I too would have preferred a similar construction for the compile time safety. Unfortunately, that construction doesn't work with immovable generators, once they have been resumed they can't ever be passed by value. I can't think of a way to encode that constraint in a similar way for pinned references, it seems you need some kind of affine reference that you can pass in and recieve back in the GeneratorState::Yielded variant rather than the current lifetime scoped Pin reference.

[clarfonthey]

A resume/await_done version seems much more ergonomic than moving the generator every time resume is called. And plus, this would prevent all of @withoutboats' work on pinning from actually being applied.

[rpjohnst]

Note that Iterator has a similar constraint- it's not really a big deal, it doesn't affect safety, and the vast majority of users of the trait don't even have to worry about it.

[haudan]

Question regarding the current experimental implementation: Can the yield and return types of generators (move-like syntax) be annotated? I would like to do the following:

use std::hash::Hash;

// Somehow add annotations so that `generator` implements
// `Generator<Yield = Box<Hash>, Return = ()>`.
// As of now, `Box<i32>` gets deduced for the Yield type.
let mut generator = || {
    yield Box::new(123i32);
    yield Box::new("hello");
};

[Nemo157]

I was hopeful that let mut generator: impl Generator<Yield = Box<Debug>> = || { ... }; might allow this, but testing with

fn foo() -> impl Generator<Yield = Box<Debug + 'static>> {
    || {
        yield Box::new(123i32);
        yield Box::new("hello");
    }
}

it seems the associated types of the return value aren't used to infer the types for the yield expression; this could be different once let _: impl Trait is implemented, but I wouldn't expect it to be.

(Note that Hash can't be used as a trait object because its methods have generic type parameters which must go through monomorphization).

One terrible way to do this is to place an unreachable yield at the start of the generator declaring its yield and return types, e.g.:

let mut generator = || {
    if false { yield { return () } as Box<Debug> };
    yield Box::new(123i32);
    yield Box::new("hello");
};

EDIT: The more I look at yield { return () } as Box<Debug> the more I wonder how long till Cthulu truly owns me.

[haudan]

Yeah, I was hoping as well impl Trait would do the trick, but couldn't get it to work either. Your if false { yield { return () } as Box<Debug> }; hack does indeed work, though after seeing that, I don't think I will be able to sleep for tonight.

I guess the only way is to introduce more syntax to annotate the types?

[Diggsey]

Will the Generator::resume() method be changed to use Pin<Self> and be safe, or is the idea to add a new SafeGenerator trait?

[Nemo157]

I assumed that it would be changed, and I happened to be looking at the Pin RFC just now and noticed that it agrees, but it is blocked on object safety of arbitrary self types (which is currently an open RFC):

Once the arbitrary_self_types feature becomes object safe, we will make three changes to the generator API:

1. We will change the resume method to take self by self: Pin<Self> instead of &mut self.
2. We will implement !Unpin for the anonymous type of an immovable generator.
3. We will make it safe to define an immovable generator.

The third point has actually happened already, but it doesn't help much since that required making Generator::resume unsafe.

[zesterer]

On my wish-list is for the resume type to be a GAT rather than a type parameter (I'm not even sure why it's a type parameter right now? I don't see any value in a type implementing Generator multiple times with distinct resume types...), allowing us to pass resume values with distinct lifetimes back into the generator. For example:

pub trait Generator {
    type Yield;
    type Return;
    type Resume<'a>;

    fn resume<'a>(self: Pin<&mut Self>, arg: Self::Resume<'a>) -> GeneratorState<Self::Yield, Self::Return>;
}

On the generator side, the value you get out of an invocation of yield(_) would be a value that lives until the next yield. For example:

let resumed_value = yield(1);
// <-- Legal to use `resumed_value` here
let resumed_value2 = yield(2);
// <-- Illegal to use `resumed_value` here, the second `yield` invalidates the first resume value

This seems of questionable value, except for the fact that this design allows you to repeatedly pass some sort of long-lived state/context into a generator for it to operate upon. For example:

struct Log(Vec<String>);

// (I realise that `for<'a> &'a mut Log` isn't valid syntax since Rust doesn't support lifetime HRTs yet)
fn generator_that_logs() -> impl Generator<Resume = for<'a> &'a mut Log, Yield = i32> {
    let log = yield(1);
    log.0.push("Just generated '1'".into());
    let log = yield(2);
    log.0.push("Just generated '2'".into());
}

// Example usage

let mut gen = std::pin::pin!(generator_that_logs());

let mut log = Log(Vec::new());
assert_eq!(gen.resume(&mut log), GeneratorState::Yielded(1));
assert_eq!(gen.resume(&mut log), GeneratorState::Yielded(2));

assert_eq!(log.0, vec![
    "Just generated '1'".into(),
    "Just generated '2'".into(),
]);

Being able to communicate with a generator using some extra context where said context has a lifetime bound to the caller of resume makes generators immensely more useful for many applications, particularly those with deferred execution patterns that need to do some useful work on the context between invocations of resume. The only ways to do this today require a lot of unsafe (which can't be hidden from the user) and have extremely questionable soundness.

[real-felix]

Hey, I used to having core::iter::from_generator in a project, but after upgrading the compiler version, it tells me that:

unknown feature iter_from_generator

What should I do now, to get an iterator from a generator?

[Kobzol]

Generator were renamed to coroutines recently (#116958). The feature is now called iter_from_coroutine.

[real-felix]

Generator were renamed to coroutines recently (#116958). The feature is now called iter_from_coroutine.

What a quick answer! Thanks a lot. Does it change anything to the actual feature? Can I write an iterator directly with a generator syntax, or must I still use core::iter::from_coroutine + closure coroutine/generator?

[Kobzol]

I have no idea, but I think that this PR just renamed stuff, and didn't change any functionality.

[initial-algebra]

On my wish-list is for the resume type to be a GAT rather than a type parameter (I'm not even sure why it's a type parameter right now? I don't see any value in a type implementing Generator multiple times with distinct resume types...), allowing us to pass resume values with distinct lifetimes back into the generator.

I can't agree enough. There's no excuse to add yet another core trait that's been handicapped due to the lack of GATs. While we're at it, Yield should also be a GAT. For example, that would enable coroutines to be used for LendingIterator (as Iterator is one of those aforementioned handicapped traits) implementations.

EDIT: It's unnecessary for the resume type to be a GAT, as long as you can use for<...> as in the issue @Nemo157 brought up, but it's crucial that Yield be a GAT so it can mention the lifetime on self.

[Kixunil]

I started experimenting with coroutines and it looks like they implement lifetimes incorrectly? https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=a7a1af3f2e2259aaa7df29350763e27f

I would expect it to impl<'a, 'b> Coroutine<&'a mut &'b [u8], Yield=Result<(), ()>, Return=Result<(), ()>> for {coroutine@src/main.rs:14:27: 14:46} but it seems to use some specific lifetime (how's that even possible?!) instead of all lifetimes.

[Nemo157]

@Kixunil #68923

[tezlm]

One thing I'd like to see implemented is a way to strongly type yield and return types together.

enum Yield {
  Foo,
  Bar,
}

enum Resume {
  FooResponse,
  BarResponse,
}

type BadState<C> = CoroutineState<C, Yield, Resume>;

yield Yield::Foo could be resumed with Resume::BarResponse, which is pretty bad! Maybe resuming a coroutine could return a new coroutine with the same state but different Yield/Return types.

[SuitespaceDev]

We recently did some exploratory work to find out if Rust could support our streaming architecture - and hit several of the roadblocks mentioned in this thread.

Thus, I am here to toss in our two cents.

At a very high level generators, coroutines and streams should be somewhat complimentary concepts, regardless of language. Especially I mean that a calling scope loops/iterates/traverses until end of some incoming output, seamlessly forwarding underlying yields.

foo() Scope

• Single yield "Header: \n"
• nested loop over foo_content()
  • yield some_content
  • yield bar(some_content)
    • various operations..
    • yield some_processed_content
    • yield some_optional_content

Ultimately the yield of some_processed_content and some_optional_content should end up becoming the yield of upstream functions when the developer says so. One could think of the total call stack as a tree and each yield as a node on such a tree.

Then, resume/pause/etc are simply a form of iterating cursor, keeping up with what has been traversed so far.

Cheers

[antonsmetanin]

I made a simple test of coroutines for my use case, but there seems to be a problem with lifetimes of values passed into resume():
https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=0b35977241bf770cb2a782e2360f0dcf

Isn't borrow supposed to end after the call?

[fogti]

@antonsmetanin it's even worse: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=491f8437083e51ec5c0bf3aedf7e5d04; the values passed into are used through all generator invocations.

[rpjohnst]

There's an existing issue about giving coroutines the ability to borrow the argument to resume: #68923. And also some recent discussion that happened on a different issue: rust-lang/rfcs#3513 (comment)

[zesterer]

Today I had a go at using coroutines to write a high-performance VM that supports reentrant code (effect handlers and the like). The idea is to produce a chain of TCO-ed dynamic calls like threaded code, except with each instruction being a little coroutine instead of just a regular closure.

It's so close to working very well and producing near-optimal code (something I was very shocked at), but unfortunately the codegen is being nuked from high by the panic-if-resumed-after-return check, resulting in extra branches and stack spilling.

I know that this is way outside the bounds of what most people are probably going to be using coroutines for, but I'm very much wishing for an unsafe version of Coroutine::resume, perhaps called Coroutine::resume_unchecked, that elides the check, with the consequence that calling the method after the coroutine has returned is UB. Having this really would make coroutines zero-cost in the most literal sense of the phrase.

[Sunshine40]

It's sad we can't impl<C: Coroutine<Return = ()> + Unpin> IntoIterator for C just because it might overlap with impl<I: Iterator> IntoIterator for I.

In Python it's easy to define a "yielding" generator function that actually returns an iterator when called:

def yield1to3():
    yield 1
    yield 2
    yield 3

In Rust,

#[coroutine] || {
    yield 1;
    yield 2;
    yield 3;
}

is not a closure, it's the coroutine (generator) itself, but it can't be used as an iterator without explicit conversion.

[cynecx]

@Sunshine40 You'd want actual generators:

#![feature(gen_blocks)]

fn main() {
    for v in gen { yield 1; yield 2 } {
        println!("{v}");
    }
}

https://play.rust-lang.org/?version=nightly&mode=debug&edition=2024&gist=d86514b466bd8051a17f2fdea5bbe67d

[Kixunil]

@zesterer for a while I was thinking of writing an article about this for Future but I thought it's a micro-optimization, so I've been putting it off. If your experience proves otherwise I might actually try doing it sooner. Just note that for perfect zero-cost just having resume_unchecked is not enough because the mere presence of resume necessitates storing some flag to know whether the coroutine (future) panicked. It needs a completely new trait and code that handles it which I believe can be added backwards-compatibly. (impl<T: Coroutine> UnafeCoroutine for T, then bounding by UnsafeCoroutine and having a helper that fuses UnsafeCoroutine into Coroutine for those cases where you have to.) It also needs an additional method that signals to the coroutine you're giving up and dropping it before it was finished.

[TheApproach]

Fascinated by the conversation at large but especially between @zesterer and @Kixunil here
I'm very familiar with coroutines in general but also quire n00b at Rust so please forgive if this ends up being nonsensical haha.

That said; I wonder if it could be handled by the mere behavior of Coroutine or alternatively, instead of making an unsafe version, there is a RepeatingCoroutine or looping or some such.

i.e., is it actually required that Coroutine panic on resume if trying to run after return just yields () or a zero-size struct? Alternatively, or perhaps following such a yield, resetting the iteration. Then the upstream function could decide what to do and a panic-capable coroutine could still be derived from it.

I am mixing logic from different languages here so, again, perhaps this makes no sense to however Coroutines are being implemented. Glad to see the progress so far from userland though. thx

[NobodyXu]

i.e., is it actually required that Coroutine panic on resume if trying to run after return just yields () or a zero-size struct?

That would change the return type, now all of them would be injected with an Option or ()

Alternatively, or perhaps following such a yield, resetting the iteration.

It's only possible when the coroutine is pure (no I/O), and the parameters for it are either only used by immutable reference, or copyable/clonable.

And it has performance implications: These parameters have to be saved somewhere, and copying/cloning isn't free.

Then the upstream function could decide what to do and a panic-capable coroutine could still be derived from it.

IMO it is possible that rust has a FusedCoroutine, similar to FuseFuture, which always return a certain variant (e.g. yield in coroutine, Pending in FusedFuture) and has a method to test if it has returned.

[TheApproach]

@NobodyXu ah yes that makes perfect sense of course. I suppose the hope was that the Rust internals had some magic type wrangling to hide what my mind wants to think of as "forwarding null". I see that would not be the case.

Per resetting, that does make sense as well. Does that then imply that, when pure, @zesterer 's UnsafeCoroutine could be both trivial and safe?

Per upstream handling of safety, very nice to hear that.
From the sound of it FusedCoroutine would achieve what I was intending. thx

[NobodyXu]

Per resetting, that does make sense as well. Does that then imply that, when pure, @zesterer 's UnsafeCoroutine could be both trivial and safe?

I think adding a new unsafe method is definitely doable and achievable, though ideally compiler should optimize out the panic landing code.

[Kixunil]

Does that then imply that, when pure, @zesterer 's UnsafeCoroutine could be both trivial and safe?

The reason for unsafe is actually Rust's moves. If you want to yield a network connection from a coroutine you can only do that once, so you either have to keep track of it at runtime or compile time. Runtime is slower and the compiler currently can't check it without introducing other costs (e.g. moving stuff around a lot), so only unsafe remains for performance-critical things.

There are tricks to deal with the unsafety though. The code is generated by the compiler, so the implementation doesn't need unsafe and calling it us usually done by some kind of executor which can be an easy place to audit unsafe. Further if we get &move references (AKA stack box - basically Box that has a lifetime because it's backed by stack; main feature being the ability to move out of it without runtime checks) we can design a safe wrapper with fn resume(&move self) -> CoroutineState<(Y, &move Self), R> which enforces correctness with tiny additional cost (having to move the pointer out). It should still be cheaper overall than forcing all coroutines to handle the state because it doesn't multiply over multiple layers of coroutines (this may be more visible in async code where people call async functions from async functions, each call creating an additonal layer with all the panic handling).

@NobodyXu the compiler can't optimize it across dynamic dispatch but a special trait can. I'm not suggesting that the cost of dynamic dispatch is small enough for this to matter, just stating the fact.

[NobodyXu]

fn resume(&move self) -> CoroutineState<(Y, &move Self), R>

I wish Future::poll also uses &move self to avoid panic landing.

the compiler can't optimize it across dynamic dispatch but a special trait can.

That's true, adding an unsafe method or use &mut self could avoid the panic overhead.

Or maybe using a .fused() could avoid these panic overhead, at the cost of using Option<C> in layout and one additional if, it's definitely cheaper than panic.

[Kixunil]

@NobodyXu note that I've since realized that &move self is unsound because of the pin requirement to run destructor before overwriting the value and you can leak &move T which would prevent running the destructor. What is actually needed is PartialMove<Pin<&mut Self>> where PartialMove is a smart pointer that calls a special method (I chose the name cancel) which drops the parts that can be moved out but keeps whatever cannot, so that the pinned destructor can still run.

Also I've found a way to return a zero-sized token instead of PartialMove<Pin<&mut Self>> so that the whole trait could be safe and zero-cost! It just needs a different signature. However I came up with all these things when designing a state machine that's neither Future nor Coroutine but it's sufficiently similar that the same ideas can be applied. It's just that I have to rewrite it in terms of Future to explain it. (I've already started writing down the previously mentioned things about Future just haven't finished.)

Or maybe using a .fused() could avoid these panic overhead, at the cost of using Option<C> in layout and one additional if, it's definitely cheaper than panic.

The performance cost is the same - it's one branch and one byte in memory. Less panics probably means smaller binary but hardly anyone cares about it.