Discussion for Coroutine Theory

[lewissbaker]

Please add comments to this issue for discussion of the article Coroutine Theory.

[natbraun]

Excellent article, thanks! Quick question: could you explain if/how one can customize the way coroutines allocate memory on the heap? May be using an allocator of some sort... Thanks!

[lewissbaker]

The Coroutines TS provides the ability to customise the allocation strategy by overloading operator new on the promise type of the coroutine.

I'll cover this in a bit more detail in my next post. Stay tuned!

[skgbanga]

Great article, thanks!

In the case, the compiler can verify that the co-routine doesn't outlive the caller (e.g. cases like python generators in loop), does the activation frame of the co-routine remain on the stack? (and the stack part of the co-routine is not destroyed? or maybe some of it is destroyed and some of it is not.)

[lewissbaker]

@skgbanga Where the nested coroutine-frame is placed in the caller's activation frame depends on the whether or not the caller is a coroutine or a normal function and if it's a coroutine on whether or not the lifetime of the nested coroutine spans a suspend-point in the caller.

If the caller is a coroutine and the lifetime of the nested coroutine spans a suspend-point then the compiler is going to have to put it in the coroutine-frame, if it doesn't span a suspend-point then the nested coroutine frame may be placed either in the stack-frame or the coroutine-frame (compiler-dependant).

If the caller is a regular function then it must be placed on the stack-frame if allocation is elided.

Placing it within the stack-frame in these cases should be safe since the lifetime of the nested coroutine-frame does not outlive the lifetime of the enclosing stack-frame.

[skgbanga]

Makes sense. I am looking forward to an example in the series where the allocation is elided. e.g consider this (func is a normal function, and coor is a co-routine)

func():

• allocate some stuff on stack - Stage 1
• h = coor()
• let's say coor() is suspended at this stage
• allocate more stuff on stack - Stage 2
• h.resume()

I am quite interested to know where does the compiler put the Stage 2 variables on the stack? after the co-routine's activation frame? What if co-routine wants to increase its stack?

Let me know if I am not super clear in what I am describing.

[danghvu]

@lewissbaker. Thanks for the post and your cool work, I have 2 questions:

• You mentioned there is a heap allocation for activation frame, what is the typical size that is allocated ? That is, what is the memory usage of a coroutine ?
• When you have more than 1 coroutine on the flight, do you have a queue to store the pending coroutine handler ? For example if there are more than one "executor" / thread in the system -- they can poll and make progress on the pending coroutine ? Or is the coroutine only resumes if it is explicitly "resume" via a call ?

[lewissbaker]

@danghvu Great questions!

What is the memory usage of a coroutine?

The size of the coroutine frame is highly dependent on the coroutine body, type of coroutine as well as the compiler and optimisation flags so it's difficult to give an exact answer here. However, we can put a lower-bound on the size of the coroutine frame.

The coroutine frame needs to store a number of things:

• The promise object. The size of this will vary depending on the type of coroutine. It can be empty, but typically holds at least a couple of pointers.
• Copies of the parameters to the coroutine. The size required for these will depend on the number and sizes of parameters passed to the coroutine. Note, however, that the compiler is allowed to avoid storing a copy of a parameter in the coroutine frame if the parameter has a trivial destructor and is never referenced after the first suspend-point.
• Some representation of the suspend-point / resumption-address so that a suspended coroutine knows where to resume execution. MSVC currently uses 16 bytes for this, Clang uses slightly more. Future compiler optimisations may be able to reduce this further.
• The state of any local variables or temporaries whose lifetime spans one or more suspend-points within the coroutine. A compiler should be able to reuse storage within the coroutine frame for different variables whose lifetimes don't overlap. As a lower-bound you can look at each suspend-point and sum the sizes of local variables whose lifetime spans that suspend-point and then take the maximum of those sums.
• Compilers may also store additional state in the coroutine frame, such as saved register values.

Clang is currently more effective at trimming down coroutine frame size than MSVC.

Is the coroutine only resumes if it is explicitly "resume" via a call?

Yes, that's right.

A coroutine suspends when it co_awaits something. That something is then responsible for scheduling the resumption of the coroutine when the result is ready (eg. when an I/O operation completes). It resumes the coroutine by calling coroutine_handle<>::resume() on the handle that represents the suspended coroutine frame that is awaiting the operation.

The executor resumes the awaiting coroutine by calling coroutine_handle::resume() when the operation completes in much the same way as the executor would execute the callback for callback-based async notification approaches.

[danghvu]

Thank you for the prompt reply!
I want to follow up on question 2, I can see now that a coroutine has to be resumed when a coroutine_handle::resume called.
Part of my question also asks about how a coroutine_handle is kept when it is suspended, can you explain this a bit ? As I understand, the handler will be returned to the parent, but only there -- the user is responsible for passing around this handler (to executor) if needed ?

[lewissbaker]

@danghvu The coroutine_handle of the suspended coroutine is passed to the await_suspend() call of the awaitable object. The implementation of that method can store the coroutine handle wherever is most appropriate for the operation being awaited.

For example, the await_suspend method for a cppcoro::task object will store the coroutine handle of the awaiting coroutine in the promise object of the task's coroutine, whereas the cppcoro::file_read_operation object stores the coroutine handle in read-operation object itself and then passes a pointer to the read-operation to the executor.

If you watch @GorNishanov's recent CppCon talk on coroutine interactions with Networking TS he shows how the coroutine handle can be stored in a lambda that is passed into the underlying callback mechanism of the networking TS.

[breathe67]

Great article, clear explanation of the mechanics of stackless coroutines, thanks.

From what I understand, if you call a coroutine and give it a pointer parameter to data that 's on the stack of the caller (e.g. coroutine( int *integer_on_call_stack) ) , it's quite possible the coroutine frame will point to invalid data if it's resumed on a different stack frame or thread. I guess you just have to be aware of this ?

[lewissbaker]

@breathe67 Yes, that's right. If you pass a pointer or reference to a value owned by the caller into a coroutine then you need to be aware that the coroutine will continue to hold a reference to that data after the initial call returns.

This can be safe if you make sure you co_await the returned task/future before the data being referenced by the coroutine goes out of scope, but it's definitely a potential gotcha you need to be aware of when writing asynchronous coroutine code.

[ioquatix]

I think you should link to this video: https://www.youtube.com/watch?v=_fu0gx-xseY

[JimViebke]

Would you consider adding a code example?

[ronaldpetty]

Well done!

[lewissbaker]

@ioquatix I've been thinking about just making a separate post listing some useful coroutine resources. I'll make sure to include that link in there.

@JimViebke I'll try to include more code examples into future posts. In the mean-time, check out some code examples on the cppcoro README: https://github.com/lewissbaker/cppcoro

[crusader-mike]

Good article. Few notes:

1. making coroutine call to be the same as function call (probably) won't work -- simply because coroutine "personal" frame allocation may fail with std::bad_alloc. This never happens to normal function call.

2. I would argue that it makes sense to move "coroutine frame" allocation to the caller -- in this way we could avoid expense of moving/copying call parameters to coroutine frame. Compiler can generate code that creates parameters right where they should be (in coroutine frame). Even more -- when compiling stackless coroutine compiler can figure out which parameters need to be on coroutine frame (and which ones can stay on stack frame) and put this information into coroutine signature.

3. if compiler decides to use stack for coroutine frame -- frame allocation will never throw. It makes sense to allow end user to assert this expectation. I.e. some keyword that states "I expect this coroutine frame to be allocated in local storage, please emit compiler error if it is not the case". This is important in case when coroutine gets called from noexcept function (or function that is not expected to throw std::bad_alloc).

4. what is going to happen if some destructor throws during coroutine Destroy operation?

5. In light of points 1-4 hiding coroutine frame allocation from user might be not a very good idea. May be it makes sense to make "frame allocation" a separate explicit step in which user can choose himself where frame is going to be allocated (heap or stack).

[crusader-mike]

6. Also, following that line of thinking -- there is no need for co_await keyword at all. You can make function declaration different from coroutine (but leave invocation syntax same/similar) -- in this way I don't need to change client code if I decide to change my coroutine into a function... and I wouldn't need to propagate co_awaits up the call tree. And, since function and coroutine will be two different kinds of object -- any mismatches will be picked up by linker. You will still need a new keyword for suspend.

[breathe67]

@crusader-mike However it would be harder to look at a function and realize it suspends rather than executes normally. You might often be looking at normal non-waiting functions in your (or someone else's) codebase and wondering if any of them actually suspend.

[crusader-mike]

@breathe67 this can be addressed in many ways -- naming conventions (coro_ prefix), syntax highlighting (editor can notice that foo() has type coroutine), slightly different call syntax or extra "frame" parameter (see point #5). Besides, calling coroutine doesn't guarantee that it will suspend at all. After all coroutine is a generalized form of function... It's invocation doesn't have to be different (or incompatible).

One more point:
7. Exception specification (which nowadays boils down to presence or absence of noexcept) -- what to do with it? With function it applies to entire execution, with coroutines your execution is split into small chunks -- some of them can throw, some won't. It would be nice to be able to specify that this particular invocation is noexcept.

[lewissbaker]

@crusader-mike Thanks for the feedback and the great questions.

1. It's true that calls to a coroutine function may throw std::bad_alloc. This would be no diffierent to, say, calling a function that takes a std::string by value or that needs to allocate a std::promise in order to return a std::future. While this does mean that by default, coroutines should not be declared noexcept, there are mechanisms available to allow customising the return-value in case the coroutine frame allocation fails, rather than throwing std::bad_alloc (see response to 7. below).

The advantage of making coroutine functions look exactly like ordinary functions from the call-site is that it allows you to change the implementation of that function to delegate creation of the coroutine to some other function without needing to change the signature. eg.

// Change from this...
task<> foo()
{
  co_await bar();
}

// To this, without any signature change.
// note that foo() is no longer a coroutine, but it's still a function that returns a task<>
task<> foo()
{
  auto x = some_non_async_function();
  task<> result = make_task<void>(when_all(bar(), baz(x)));
  return result;
}

2. The general direction that compilers are moving is that the coroutine function would be compiled to a small stub that simply allocated and constructed the coroutine frame, moving parameters as required. The main body of the coroutine would be outlined to a separate function. This stub would be inlinable by the compiler and thus would allow it to do things like allocation elision and copy elision when constructing the coroutine frame.

The Coroutines TS already has provisions for allowing you to hook in custom allocators for the coroutine frame which would allow the caller to specify the allocation strategy. The main limitation here is that you cannot know the coroutine frame allocation size at compile time as the frame size can vary based on backend compiler optimisations. This makes it difficult to allocate the memory for the coroutine on the stack, since you don't know how big to make the buffer. The compiler is in a much better position to be able to determine the size of the allocations and also whether the allocations can be elided.

3. This is an interesting avenue for research. Perhaps some kind of annotation at the call-site to get the compiler to assert at compile time that the allocation is elided would be useful?

4. I expect that you'd end up with behaviour that is much the same as if you had a heap-allocated class with several members and one of those members throws from its destructor during destruction of the containing class. ie. it will probably leak memory/resources

5. The allocation mechanism for coroutines is customisable. You can provide an overloaded operator new on the promise type for the coroutine. This operator can optionally be given the parameters passed to the coroutine function so should allow you to pass in a custom allocator object to use for allocating memory needed for the coroutine frame.

6. There was an alternative proposal a couple of years ago for a coroutine design that was similar to what you suggest here. There were some compelling arguments against that design. See the following papers for background:

• P0114R0 - Resumable Expressions
• N4398 - A unified syntax for stackless and stackful coroutines
• P0073R2 - On unifying the coroutines and resumable functions proposals
• P0171R0 - Response to: Resumable Expressions P0114R0

There are also use-cases where I want to call a coroutine but not actually block the caller while calling it (eg. so that I can execute two coroutines concurrently). e.g. See cppcoro::when_all() usage.

7. The design of coroutines should accommodate writing noexcept coroutines through use of the promise_type::get_return_object_on_allocation_failure() function which is called if the coroutine frame allocation fails. There are other avenues that can be pursued for declaring that the coroutine body itself won't throw. eg. returning task_nothrow instead of task - the task_nothrow type could provide an await_ready() method that is declared noexcept and a promise_type::unhandled_exception() method that just calls std::terminate().

[crusader-mike]

@lewissbaker I spent few days reading coroutine-related material and watching presentations. Wow... complicated topic.

Let me summarize my mental model first (please let me know if I am way off the mark somewhere):

• stackful coroutines -- code (represented as tree of function calls) gets executed on a separate stack with special commands to switch/return to another stack. These places are 'suspension points' -- when we reach them, we probably (but not necessarily) prepared a value somewhere for others to consume or reached certain state. Protocol that defines when result becomes available is defined by coroutine implementation (some sort of a contract between coroutine and it's caller(s)).

  • function attributes (e.g. noexcept) work as usual since nothing happened to the function itself -- in it's own context it is executed as usual and whatever semantic given attribute enforces still applies to entire function

  • what we do here is we use a stack to turn "linear" program (written as tree of function calls) into a state machine (at run-time) -- similar to how preemptive OS turns a program into state machine (that switches its state with each asm instruction). With exception that in coroutine switch completes only when we reach next suspension point

• stackless coroutine -- a way to create aforementioned state machine (at compile time) by analyzing your code. It is kind of "inlined coroutine". Result of this magic is an object that gets constructed, repeatedly called into and eventually destroyed. Conceptually it has the same properties, but on code level it is fundamentally different -- to switch it state we call "switch-to-next-state()" method (aka resume) and it returns back to the caller (because it is a function. once we went through "inlining" magic corotine is gone -- we have just an almost normal C++ object).

  • this object needs to be allocated and constructed somewhere

  • co_await/etc keywords are just a way to hide boilerplate code and mark suspension points for compiler -- it'll use them to break your original code into parts

  • as result compiler need to see into every function your coroutine may call if it (the callee) contains a suspension point. Or, in other words, if it takes your coroutine and expands every inline function call -- all suspension points (your coroutine can have) should be inside of resulting function body.

  • with certain coroutines current state can be "data-is-not-ready-yet" (for example when it wants to read 1kb of data from the socket but they arrive byte-by-byte), so the client (caller) need to spin event loop, which will drive state switching (via callback installed during construction) until state becomes "data-is-ready"

  • which means if we are to hide coroutine behind single interface (i.e. func(a,b,c)) we need a way to distinguish if given call returns on first suspension point or on switching to "data-is-ready" state. This would explain necessity of a co_await keyword -- because that call always returns after reaching next suspension point. If you don't want to wait -- you call it as a function and then you have an option to spin event loop/etc until coroutine state reaches what you need it to reach. If you do want to wait for "data-is-ready" state -- solution is to use co_await and turn your caller into coroutine (thus delegating necessity of spinning event loop elsewhere)

I think this should cover (at a high level) most of what I've learned... Now it is clear that there is a lot of depth to this and I am just only scratching the surface -- you probably went through this line of thinking many times long ago. Therefore, I suggest looking at my comments as an attempt to learn, not criticize -- if my notes contain nothing new, please point me out why I am wrong. If you think given idea is interesting -- you are welcome to steal it.

Now to original points:

points 1-3,5: it seems that coroutine construction has the same semantics as object construction -- why not use the same approach? (after all it is an object) Smth like this:

for(auto x: my_coro1(a,b,c).start()) ... ;   // in-stack

co_await (new my_coro2(a,b,c))->start();     // in-heap

• it can be massaged (maybe with additional language features) to look nice
• it gives user explicit control over frame allocations
• need some mechanism for coro to recognize if it is ok to kill itself on final suspend (hidden flag in constructor? like we do in virtual destructors already?)
• you can have more fun on language level -- how about providing extra argument into resume() which will be available in coroutine as result of co_yield? :-) Smth like this:

void whatever_you_have_I_ll_double_it(int x) {
    for(;;) x = co_yield 2*x;
}

As you have pointed out for smth like this to work related information (frame size, required arguments, etc) need to be already available at compile time. Maybe another compilation step where coroutines are converted to objects and co_* keywords -- to related boilerplate? Or a new ABI-lvl object (coroutine) that carries it's frame size along with signature -- so that stack allocation (or parameter passed to operator new) could get and use it at run time? We may assume that all coroutine parameters will go to coroutine frame for simplicity (they probably will in most cases anyway). Or making coroutines similar to templates in sense that it's definition has to be visible in all translation units that call them.

point 4: problem here is that in couroutine Destroy() can be called in multiple non-obvious places -- i.e. it is hard to look at the code and tell if it is going to kill your program or not. Well... It is always hard in case of throwing destructor, but now it will be even harder :-)

point 6: Here is what I have in mind:

• we need a keyword to distinguish between two types of calls -- "couroutine" call (wait until data-is-ready state) and "normal" call (wait until suspend point). So we use co_await to mark first one
• why not flip this around -- if coroutine is clearly marked in it's declaration, we know that it is a "coroutine" call, why not:
  • use a keyword for non-waiting call (which will have noop effect on normal function call)
  • forbid non-inline(!) function from making a "coroutine" call
• ... in this way if I decide to change my coroutine back into a function -- I simply change it's declaration, not it's call sites

point 7: since coroutine body gets split into chunks (with addition of constructor) -- it is probably makes sense to forbid (or change semantics) of noexcept on couroutine. I think automatic noexcept deduction can be useful here.

More notes:

8. the way coroutine is implemented right now prevents possibility of RVO. Well, at least no one mentioned it in any source I've read. On language level it should be possible though -- address to uninitialized memory can be passed to coroutine in co_await and used to construct returned object inside of coroutine code.

9. I don't like automatic heap elision -- it is too similar to copy elision and will probably go through the same painful evolution (30 years before people realized that is absolutely has to be mandatory):

   • it is hard to tell if it is possible by looking at the code
   • you can't rely on it happening (or force it to happen)
   • it makes it even harder to predict required stack size -- idea is that you can create reliable C++ program by analyzing code and figuring out max stack size (and pre-allocate it at start) becomes less viable

[crusader-mike]

Or making coroutines similar to templates in sense that it's definition has to be visible in all translation units that call them.

I am positively sure this idea is worth consideration... Basically, you write a coroutine, then on translation step (similar to templates) compiler will convert it into object (with associated facilities) and it can be used by the rest of the code in the similar way template instantiations get used.

coroutine<int a, string b>
generator<int> coo(float c) { ... }  // compiler will complain if 'c' "crosses" any suspension point

int main()
{
    coo<1,"abc"> mycoo;
    for(auto x: mycoo(0.5))  ... ;
}

[dfct]

@crusader-mike, re:

We may assume that all coroutine parameters will go to coroutine frame for simplicity (they probably will in most cases anyway).

as well as @lewissbaker, re:

Copies of the parameters to the coroutine. The size required for these will depend on the number and sizes of parameters passed to the coroutine. Note, however, that the compiler is allowed to avoid storing a copy of a parameter in the coroutine frame if the parameter has a trivial destructor and is never referenced after the first suspend-point.

Are there clear rules for this behavior, or is a bit compiler-specific, or..?

I'm running into weird issues where, for reasons I can't seem to nail down, my function parameters will sometimes survive past the first suspend point without issue, but other times require me to copy or move them into a local variable first in order to access them later.

[ioquatix]

@dfct what platform are you running on and can you show me the assembly of the transfer function?

[dfct]

@ioquatix I'm using macOS & Apple's Xcode-bundled version of clang. I've been trying to extract enough code to have a reproducible test case, and can post the code & assembly then.

It has proven a bit challenging due to time constraints & Xcode's unintentional lack of support for indexing & debugging coroutine code. Working with what I've got..

edit: Here's an example: https://wandbox.org/permlink/lQFoC2noGQPaZxTj

[dfct]

@ioquatix I've been playing with this a bit more, and I should note that using the asio executor is /not/ required to reproduce the issue, though it does seem to make it significantly more apparent.. This code without asio reliably fails within a few runs for me: https://wandbox.org/permlink/lz45rYsmcLz5pub9
(Interestingly, it does not fail on wandbox, unlink my first link ^)

[ioquatix]

What's the failure?

Have you tried using asan, ubsan and tsan?

[dfct]

@ioquatix In the coroutine that does not explicitly move the function parameter to a local variable before calling co_await, the function parameter is /sometimes/ invalid when the coroutine resumes. In the first example above, which reproduces on wandbox, you can see it print incorrect values e.g. "1946159296" instead of "1" after the co_await. (Similar for the second link, though that one doesn't reproduce well on wandbox.)

I'm not familiar with the use of those sanitizers, I'm afraid...

RE: the two quotes above of @lewissbaker & @crusader-mike, it seems like I shouldn't need to explicitly move the function parameter to a local variable for it to persist past a co_await call.

[ioquatix]

x64 uses registers for passing arguments.

If you are not careful, those registers get clobbered. It will depend on the exact semantics of co_await.

The first resume loads the registers with the arguments.

The co_await call should push registers on the stack, then when the call completes, it pops them off again. If the code is not careful, the arguments would be clobbered. This can happen if co_await itself is passing arguments.

[ioquatix]

I would suggest stepping through with the debugger and after resuming the co_await, check the values of the registers. On x64 it's RCX, RDX, R8, and R9 for the first four integer/pointer arguments.

[ioquatix]

Can you disassemble the function so we can see the code inserted by co_await? It will be clear if it's preserving the registers. Just make a simple function, compile -O0 and then show us the disassembly.

[dfct]

Like this? https://godbolt.org/g/cGy2j6

That code reliably prints gibberish for instead of '1' for test_obj.val. Here's the same executing code on wandbox: https://wandbox.org/permlink/XpEmgfWHhcmFoIma

Start
Coro: Val after co_await: 1275070656	
Finish

[ioquatix]

Excellent. Can you make a simple example (MVP) that still fails?

[ioquatix]

It's interesting that with -O2, it doesn't seem to fail.

[dfct]

Are you sure? For me when I add -O2 to the last example ( https://wandbox.org/permlink/XpEmgfWHhcmFoIma ), it just goes from printing gibberish to printing 0. But test_obj.val should be (and is before co_await) 1.

I'm not sure how boil the code down further; when I removed asio and launched lambdas in new threads instead it stopped reliably failing on wandbox (though it still fails locally). Not sure what else to try, I'm a bit of a novice here..

For what it may be worth, I just tried running that wandbox code ^ on an arm64 iPad via a basic app, and the same issue occurred.

Thank you for spending so much of your time investigating this with me, ioquatix! Very much appreciated.. :)

[ioquatix]

Oh, my bad, I thought 0 was the correct output, and it was producing it reliably :p

I think the next step, without digging into the assembly too much, is to use the sanitisers to check the code at run time. It sounds like something is going wrong with the reference and it's getting blown away.

[crusader-mike]

I drifted away from coroutine topic in last few months, but aren't you are supposed to wait for your Coro to complete somehow before leaving scope of your lambda (and destroying your objects)?

	auto first_test = test_obj(1);
	boost::asio::post(io, [obj = std::move(first_test)]() mutable {
		Coro(std::move(obj));
        std::this_thread::sleep_for(chrono::seconds(5));   // <---
	});

[ioquatix]

That would make sense - if the Coro goes out of scope, it would be a problem :p

[ioquatix]

I've been working on a C library for very low level (stackful) coroutines. If you want something a bit less magic, you could take a look at it: https://github.com/kurocha/coroutine

[ioquatix]

It's still a work in progress though. There are other similar libraries, but none that have the same coroutine transfer arguments/return.

[crusader-mike]

in this case it is obj destructor that causes problems :)

https://github.com/kurocha/coroutine

Thank you. I'll have a look.

[dfct]

That's super interesting, sleeping there does seem to fix it. If you've std::move'd the object, though, why would leaving the previous scope matter? Shouldn't the object exist in the coroutine frame, which outlives the scope..?

[crusader-mike]

because std::move doesn't move anything -- it enables "move" (which is just a fancy copy constructor). Your Coro takes test_obj**&&**, not test_obj

[dfct]

omg. THANK YOU! I completely misunderstood what std::move and && were doing for me there. Sure enough, removing && from Coro(test_obj&& t) results in the behavior I was expecting. Can't believe this entire saga has boiled down to two misplaced ampersands...

[ioquatix]

Hehe, C++. A language we all love and/to hate :)

[rkfg]

To allocate space for a new activation frame, you just increment this register by the frame-size. To free space for an activation frame, you just decrement this register by the frame-size.
Isn't the opposite true? Stack (usually, AFAIK) grows "backwards" so you decrement the pointer to allocate space and increment to free it.

[lewissbaker]

Stack (usually, AFAIK) grows "backwards" so you decrement the pointer to allocate space and increment to free it.

Yes, you are right. On most architectures the stack grows backwards.
The point was mainly just that allocating the stack-frame is cheap.

[yuanzhubi]

Hi, can the article explain how the "exception propagation" replace the "resume" behavior? Thanks

//eg.
try{
co_awiat f();
} catch(...){
}
//and if g resume with exception, how does it propagate? With explicit dealer? Or implicit dealer codes generated and called?
co_awiat g();

[rturrado]

Hi there!

• Some bullet sentences are missing the final dot. E.g.:

  • Destroying any parameter objects
  • Freeing memory used by the activation-frame
  • Ensuring any values held in registers are written to the coroutine frame

• Minor typo in The 'Return' Operation section: it should say "responsibilities" instead of "repsonsibilities".

• I think this long sentence had better off be rephrased:

  "This ability to execute logic after the coroutine enters the ‘suspended’ state allows the coroutine to be scheduled for
  resumption without the need for synchronisation that would otherwise be required if the coroutine was scheduled for
  resumption prior to entering the ‘suspended’ state due to the potential for suspension and resumption of the coroutine to
  race."

Thanks!