Coroutines vs Callbacks

Coroutines vs Callbacks (what the compiler proves for you)

A companion to Coroutines as State Machines and Folly to Coroutine Migration. The state-machine page shows what the compiler builds; the migration page shows what changed in our I/O path. This one answers the narrower question an engineer asks the first time they see a .thenValue chain collapse into linear co_await and shrug "same thing, nicer syntax": is the coroutine version actually more correct, or just prettier? The answer is that it deletes whole classes of bug by construction — and it's worth knowing exactly which ones, because the bugs it doesn't delete are precisely the ones still biting us (the commit-thread hop, the sgs_keepalive rule, the ublk war story).

The one rule that explains everything

A callback is a hand-written continuation; a coroutine is the compiler writing that same continuation for you.

When an async op can't return its result now, "what to do next" can't stay an implicit return address on the stack — by the time the result arrives the stack is gone (we unwound back to the reactor). So "what to do next" must be made explicit and heap-resident: some code (a lambda / function pointer) plus its captured state (a control block). That is all a .thenValue is, and all a folly::Promise consumer is. A coroutine is the exact same machine, generated: the frame is the control block, the resume index is the lambda, co_await is the split point (see Coroutines as State Machines for the lowering).

So this is not "callbacks vs something fundamentally different." It is hand-assembled continuations vs compiler-generated continuations — identical control flow. The whole difference is what the type system and compiler can now discharge for you that you previously had to get right by hand, at every single call site.

Two rungs of "callback" — be precise about what Folly already fixed

"Callback" isn't one thing. Our before-code lived on the upper rung, so don't strawman it:

Rung	Form	What it manages for you	What's still on you
0 — raw completion callback	start_async(..., [&](size_t n){ r = n; baton.post(); }), or C-style void* + fn ptr	nothing	control-block lifetime, the type cast, "fire exactly once", error plumbing
1 — future / promise (Folly)	Future<T> / .thenValue / folly::Promise	a real type (Try<T>), refcounted Core lifetime, an error channel (.thenError)	RAII / scope / exceptions across the await; composition is still nested lambdas; a Core per stage
2 — coroutine	co_await in an async_result<T>	all of the above plus structured programming across the await	a small, new lifetime rule (see Limits)

Folly futures already climbed most of rung 1 — they are typed, lifetime-managed, error-carrying callbacks. The point below is not "Folly was raw void*." It's that even a polished rung-1 callback is still CPS: the continuation is still a separate heap object in a separate lambda scope, so the three things that make code reasonable — RAII, exceptions, lexical scope — still cannot cross the await. Coroutines close that last gap.

What moves from per-call-site to compiler-or-core

Here is the load-bearing claim. The dangerous obligations of CPS — free the control block exactly once, resume exactly once, forward the error, agree on the type, clean up on every path — are, in callback code, discharged at every async call site. In coroutine code they are discharged once: by the type system, by RAII, or inside a handful of awaitables.

Obligation	Callback — you, at every site	Coroutine — discharged once by
Free the control block exactly once	intrusive refcount / manual delete (vol_interface_req + intrusive_ptr_release)	the frame, freed by the machinery at completion
Agree on the payload type	void* cast (rung 0) / Try<T> unwrap (rung 1)	await_resume() -> T — a compile error if it disagrees
Invoke the continuation exactly once	you, on every path including errors and early-outs	the await_suspend/resume protocol, inside the awaitable
Forward the error	every .thenValue / .thenError must remember to check-and-forward	normal control flow: co_return std::unexpected(...) / throw
Clean up on every path	manual — no RAII across the await	destructors run on frame unwind (vol_io_guard)
Sequence N steps	nested lambdas (callback hell), a Core each	linear co_await; one frame, more case labels

So the surface area where you can get these wrong drops from O(async call sites) to O(awaitable implementations) — and you write awaitables rarely (sisl::async::value_awaitable, when_all, the task promise) but use them everywhere. Prove "resume exactly once" once, inside value_awaitable::complete, and it holds compositionally for every co_await req->promise_ in the tree. That is the real reduction, and it is structural, not visual: the compiler is now the thing maintaining invariants that used to be a comment and a code review.

Structured programming comes back across the await

This is the part that is more than surface area — it's why the coroutine version is easier to reason about, not just to read. Callbacks destroy the stack at each suspension, so they destroy the three tools you reason with; coroutines restore all three across the co_await:

• RAII. vol_io_guard releases the volume's in-flight count when the frame unwinds — on the happy path, an early co_return, or an exception — across every suspension in between. You cannot hold a scope guard across a .thenValue: the enclosing scope exited the moment you registered the continuation. That is precisely why the before-code needed a refcounted vol_interface_req with explicit add-ref/release instead of a stack guard — the refcount was a manual stand-in for the RAII the callback model couldn't give it.

• Exceptions / errors as ordinary control flow. The after-code forwards errors on the normal return path:

  if (auto const wr = co_await rd()->async_write(new_blkids, data_sgs, nullptr); !wr)
      co_return std::unexpected(std::errc::io_error);     // just a return — nothing to thread

  The before-code threaded every error through a nested .thenValue lambda, each of which had to remember to check-and-forward; drop one and the error silently vanishes. With co_await a dropped error is a dropped co_return — far harder to write by accident.

• Local sequential reasoning. The stretch between two co_awaits reads and reasons exactly like synchronous code; you can apply ordinary "this, then that" logic to it. A callback graph has no "next statement" to reason about — only continuations registered elsewhere.

Limits: what coroutines do NOT buy you (where our real bugs live)

Coroutines structure control flow. They prove nothing about the interleaving of shared state, and they quietly add one new lifetime rule. Every one of these is a live concern in HomeBlocks:

1. A new lifetime obligation: dangling captures. Coloring traded "free the control block" for "don't reference a temp across a co_await." That is the entire reason for the sgs_keepalive rule — a lazily-run task captures the sg_list by reference, so it must be heap-owned until when_all runs it, or it dangles. Still a lifetime bug; just a more local and increasingly compiler-checkable one (-Wdangling-class analysis) than a control-block free scattered across error paths.

2. Detached handles reopen the manual hazard. The guarantees hold for the structured path. The moment you detail::detach a fire-and-forget task you are back to manual lifetime — and because tasks are lazy, dropping one instead of detaching it is a silent no-op, the dual of forgetting to fire a callback.

3. Concurrency is still unstructured — and migration slightly widens this surface. co_await is a yield barrier; only the code between two awaits is atomic on a reactor. Hold no invariant across a co_await that another coroutine could violate while you're parked. Coroutines cannot prevent the ublk war story (index b-tree corruption from driving reactor-affine I/O on a foreign thread) or the commit-thread serialization point — those are shared-state/affinity bugs, orthogonal to control-flow structure. Worse, sticky affinity makes it easy to pile a fat tail onto the commit thread after co_await req->promise_, because it reads like ordinary local code. The thing the syntax hides — which thread am I on now — is exactly the thing it can't prove for you.

Bottom line. Coroutines provably shrink the lifetime / sequencing / type / error-propagation surface — by factoring those obligations into the type system, RAII, and a few awaitables — and restore local reasoning across the await. They do not shrink the shared-state / threading surface; that is still on you, and it is where our hardest bugs are. Trust the compiler on control flow; keep reviewing affinity by hand.

Side by side

	Hand-written callback (Folly .thenValue / Promise)	C++ coroutine (sisl::async)
Continuation is…	a heap object + a lambda you write	the frame + resume index the compiler writes
Control-block lifetime	manual refcount / delete, every path	machinery frees the frame at completion
Payload type checked?	void* cast (raw) or Try<T> unwrap	await_resume() -> T, compile-time
"Resume exactly once" lives	at every call site	inside the awaitable (write once)
Error propagation	check-and-forward in each lambda	co_return std::unexpected / throw
RAII / scope across the await	no — scope exited at registration	yes — destructors run on frame unwind
Compose N steps	nested lambdas, a Core each	linear co_await, one frame
Surface area for the above	O(call sites)	O(awaitable implementations)
Shared-state / affinity safety	on you	still on you

Back to HomeBlocks

Read the Folly to Coroutine Migration write path with this lens and the diff stops being cosmetic:

• the refcounted vol_interface_req → io_req on the frame + vol_io_guard is "control-block lifetime → RAII across the await";
• the nested .thenValue / .thenError → linear co_await + co_return std::unexpected is "error plumbing → ordinary control flow";
• the sgs_keepalive heap-own is the one obligation the trade added, not removed;
• and the commit-thread hop + ublk war story are the surface coroutines don't touch — control flow got structured, affinity did not.