Coroutines

[vadimcn]

Rendered

[eddyb]

@thestinger confused me a bit about the subject, but my last findings suggest Python is misusing the word "coroutines", which traditionally are a superset of "generators" (the difference being the ability to yield through multiple stack frames).

[vadimcn]

True, that's why I called them "shallow" coroutines.

[eddyb]

Can't we just extend Iterator<T> instead?
Since generators that don't get input from yield are actually iterators.

[eddyb]

I would much rather see all the tools required for implementing this (in a syntax extension, for example), than baking it into the language (like closures are today, and will be before you can implement Fn traits manually).

[vadimcn]

I think you'd need some kind of "unsafe goto", which is ignored by borrow/lifetime checker passes for this to be possible as a syntax extension. Not likely, in my estimation...

[eddyb]

Why? When you can do a state machine transform and you get code that's easier to optimize.
I have a sketch for a syntactic (no type information) state machine transform which only really needs anonymous return types to work.

[vadimcn]

@eddyb, I would love to see how that transform looks like!

[eevee]

Based on my experience with Python:

• Not using the iterator interface for advancing coros is a huge mistake. One of the most common uses of them in Python is as an easy way to implement iteration on a custom type.
• Likewise, not allowing them top-level makes them needlessly cumbersome; I expect most of these would be wrapped in factory functions that return them, which seems like something the compiler could trivially do for me.
• In my experience, generators that make use of sending (which is extremely powerful but also extremely uncommon) take completely different types for their initial arguments and the sent arguments. Or, rather, such generators may not expect to immediately receive a sent argument, whereas your design basically enforces it with the first call. contextmanager is a good example, though it really doesn't apply to Rust at the moment.

[vadimcn]

Not using the iterator interface for advancing coros is a huge mistake. One of the most common uses of them in Python is as an easy way to implement iteration on a custom type.

Assuming the coroutine->iterator "adapter" trait is provided by the standard library (as it should), your collection iterator implementation will look like this: fn get_iter()->Iterator<T> { coro {... yield x; ... } }, which is entirely reasonable in my book, in terms of boilerplate.
On the other hand, supporting the "final" return value makes coroutines much more versatile than just creation of iterators.

Likewise, not allowing them top-level makes them needlessly cumbersome; I expect most of these would be wrapped in factory functions that return them, which seems like something the compiler could trivially do for me.

We could support that, but what would that save us vs the above? A pair of braces? (presumably, you'd still have to say coro somewhere in the function signature).

In my experience, generators that make use of sending (which is extremely powerful but also extremely uncommon) take completely different types for their initial arguments and the sent arguments. Or, rather, such generators may not expect to immediately receive a sent argument, whereas your design basically enforces it with the first call. contextmanager is a good example, though it really doesn't apply to Rust at the moment.

Quite the opposite, actually. This is where being wrapped in an outer function comes in handy: The initial set of arguments go to the outer function, which may perform some pre-processing, before constructing and returning the coroutine closure.
Unlike Python generators, the proposed design avoids the awkward requirement of having to pass None to the first invocation of send(). (How would that even work in a statically compiled language? By requiring coroutine's parameters to be wrapped in an Option<>? Yuck!)

[alexcrichton]

Thanks for the RFC! The team has decided that while desirable, this is not needed for 1.0 at this time. I imagine that a library such as this will work best with as little language support as possible, so this may be best done as a new library rather than a language change per-se.

Closing.

[vadimcn]

:( Did the team have concrete suggestions of how this could be done as a library?

[alexcrichton]

We believe that it's possible to add support for this in a backwards-compatible fashion, especially because yield is a reserved keyword. This will need a large amount of library support, regardless, and I'd think that this could be prototyped entirely as a library before adding language sugar for coroutines.

[flying-sheep]

how should that be possible?

yield means “pause execution here, until the iterator gets sent a signal to continue”.

this needs first-class support of pauseable and resumable code.

[pnkfelix]

I think what Alex means is that a library in stdlib could provide the functionality.

This is analogous to how setjmp is provided as a library function in C. Or if you prefer, call-with-current-continuation in Scheme. It simply may not require much or any primitive language change. One might want nicer sugar for the feature, but we have yield reserved so that seems okay.

(Caveat: I have not given any consideration to how the borrow-checking rules could be impacted by such a feature. I am assuming it's a solvable problem at worst.)

[flying-sheep]

ok, this is sadly far over my head. the implications and techniques necessary to create a data structure wrapping a function’s environment and stack is too deep to consider working on.

i hope some heavy contributor takes this up.