You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Cancellation is ordinary control flow, but std::stop_token makes every operation pay for thread safety
#486
Cancellation in an asynchronous application is ordinary control flow, not a rare event. It has two paths, and both run in the steady state of a normal program.
Registration
Every operation awaited under a live stop token registers a std::stop_callback when it has to wait and removes it when it completes, whether or not anything is ever cancelled. It is safe to say that in a well-designed cancellable asynchronous chain, registration happens on every operation that waits:
co_await sock.read_some(buf); // registers a stop_callback, removes it on completionco_awaitcorosio::delay(1s); // registers another, removes it on completion
Cancellation
It is pretty common to write operations where cancellation happens on every paths:
An operation with a deadline cancels either the timer when it finishes in time or the operation itself when it does not:
co_awaitcorosio::timeout(sock.read_some(buf), 30s); // requests a stop to cancel the timer
A race cancels every branch but the winner:
co_awaitcapy::when_any(reader(sock), writer(sock)); // requests a stop to cancel the loser
capy carries cancellation as a std::stop_token in await_context. A stop token is built for a request that may come from any thread at any moment, so registration and cancellation both synchronize with other threads, even when everything runs on one executor. That is a high price for two things that happen this often.
Measurements
One P-core of an i7-13700, clang 24 -O3, libstdc++ 16, Asio 1.38.2. "In situ" is a coroutine parked on async_event (on a corosio::io_context) or on asio::channel (on an asio::io_context).
per operation
capy
Asio
registration, primitive alone
28.6 ns
3.7–4.7 ns
registration, in situ (armed minus unarmed)
24.6–25.3 ns
3.1–3.4 ns
cancellation, primitive alone (fired minus not fired)
94 ns
0.4–0.8 ns
cancellation, in situ (cancelled minus completed)
92–95 ns
0.3 ns
Each timeout() call pays for a new std::stop_source (one heap allocation), two or three registrations (the timer wait, the inner operation, and the parent token when it is live) and one cancellation.
Where the cost comes from
Registration. Constructing and destroying a std::stop_callback is 6 locked RMWs in libstdc++: the state's refcount up and down, lock and unlock to link, lock and unlock to unlink.
Cancellation.request_stop() locks the state, runs the callback, then releases a binary_semaphore so that a stop_callback being destroyed on another thread can wait for it.
Completion, even without a live token.async_event::set() does claimed_.exchange() for every waiter, to arbitrate against a stop callback running on another thread.
Allocation. A stop source keeps its state in a separate allocation made with global new, so every timeout() and every when_any pays for one. Asio's equivalents also allocate per call, but from its per-thread recycling cache.
Why Asio's figures are lower
A cancellation_slot is not thread-safe. It is written, cleared and emitted only on the operation's executor, and a requester on another thread posts the emit there (co_spawn does this itself). Registration is a plain store and cancellation is an indirect call. Only a request from another thread pays for crossing threads.
Reproduction
bench.cpp
#include<boost/asio.hpp>
#include<boost/capy.hpp>
#include<boost/corosio.hpp>namespacecapy= boost::capy;
namespacecorosio= boost::corosio;
namespaceasio= boost::asio;
constexpr std::size_t N = 200000; // calls per batchconstexprint R = 11; // batches, the median is reported// Mean ns per call over N calls of fdoublebatch(auto& f)
{
autoconst t0 = std::chrono::steady_clock::now();
for(std::size_t i = 0; i < N; ++i)
f();
std::chrono::duration<double, std::nano> const d =
std::chrono::steady_clock::now() - t0;
return d.count() / N;
}
doublemedian(std::vector<double> v)
{
std::sort(v.begin(), v.end());
return v[v.size() / 2];
}
doublens_per(auto f)
{
std::vector<double> v;
for(int r = 0; r < R; ++r)
v.push_back(batch(f));
returnmedian(v);
}
// ns per call of a minus b; batches alternate so drift cancelsdoublens_delta(auto a, auto b)
{
std::vector<double> v;
for(int r = 0; r < R; ++r)
v.push_back(batch(a) - batch(b));
returnmedian(v);
}
voidrow(charconst* name, double ns)
{
std::printf(" %-52s %5.1f ns\n", name, ns);
}
//------------------------------------------------------------------------------// capy// Parks once per ns_delta step, R * N times, then finishes.
capy::task<>
waiter(capy::async_event& ev)
{
for(std::size_t i = 0; i < R * N; ++i)
{
(void)co_await ev.wait();
ev.clear();
}
}
capy::task<>
wait_once(capy::async_event& ev)
{
(void)co_await ev.wait();
}
//------------------------------------------------------------------------------// asiostructnoop_handler
{
voidoperator()(asio::cancellation_type_t)
{
}
};
// Out of line, so clear() and emit() cannot see which handler emplace()// installed and devirtualize the calls.[[gnu::noinline]]voidslot_emplace(asio::cancellation_slot slot)
{
slot.emplace<noop_handler>();
}
[[gnu::noinline]]voidslot_clear(asio::cancellation_slot slot)
{
slot.clear();
}
using channel = asio::channel<void(boost::system::error_code)>;
// Parks once per ns_delta step, R * N times, then finishes.
asio::awaitable<void>
receiver(channel& ch)
{
for(std::size_t i = 0; i < R * N; ++i)
co_await ch.async_receive();
}
// as_tuple: a cancelled receive returns its error, like capy, instead of// throwing it
asio::awaitable<void>
receive_once(channel& ch)
{
co_await ch.async_receive(asio::as_tuple);
}
//------------------------------------------------------------------------------intmain()
{
// libstdc++'s request_stop() skips a futex wake per callback it runs// until the process has started a thread, as any real one hasstd::thread([] {}).join();
std::puts("isolated");
{
std::stop_source live;
std::stop_token tok = live.get_token();
row("capy stop_callback register+unlink, live source",
ns_per([&] { std::stop_callback cb(tok, [] {}); }));
// A stop_source fires only once, so every call takes a fresh oneauto watch = [](bool fire)
{
std::stop_source src;
std::stop_callback cb(src.get_token(), [] {});
if(fire)
src.request_stop();
};
row("capy request_stop, one callback",
ns_delta([&] { watch(true); }, [&] { watch(false); }));
}
{
// Timed inside run(), where asio's per-thread handler cache is// active, as it is for an operation started by a coroutine.
asio::io_context ioc(1);
asio::post(
ioc,
[]
{
asio::cancellation_signal sig;
asio::cancellation_slot slot = sig.slot();
auto install = [&](bool fire)
{
slot_emplace(slot);
if(fire)
sig.emit(asio::cancellation_type::terminal);
slot_clear(slot);
};
row("asio slot.emplace + clear, handler memory recycled",
ns_per([&] { install(false); }));
row("asio emit, one handler",
ns_delta([&] { install(true); }, [&] { install(false); }));
});
ioc.run();
}
std::puts("in situ");
{
corosio::io_context ioc(1);
std::stop_source src;
capy::async_event armed, unarmed;
capy::run_async(ioc.get_executor(), src.get_token())(waiter(armed));
capy::run_async(ioc.get_executor())(waiter(unarmed));
ioc.poll();
auto wake = [&](capy::async_event& ev)
{
ev.set();
ioc.poll();
};
row("capy per-op registration overhead",
ns_delta([&] { wake(armed); }, [&] { wake(unarmed); }));
}
{
// A fired stop_source stays stopped, so each cycle launches a task// with a fresh one. The task ends after its one wait either way; a// work guard keeps the context from stopping between cycles.
corosio::io_context ioc(1);
auto work = capy::make_work_guard(ioc.get_executor());
capy::async_event ev;
auto cycle = [&](bool cancel)
{
std::stop_source src;
capy::run_async(ioc.get_executor(), src.get_token())(wait_once(ev));
ioc.poll();
if(cancel)
src.request_stop();
else
ev.set();
ioc.poll();
ev.clear();
};
row("capy per-op cancellation overhead",
ns_delta([&] { cycle(true); }, [&] { cycle(false); }));
}
{
asio::io_context ioc(1);
asio::cancellation_signal sig;
channel armed(ioc), unarmed(ioc);
asio::co_spawn(
ioc,
receiver(armed),
asio::bind_cancellation_slot(sig.slot(), asio::detached));
asio::co_spawn(ioc, receiver(unarmed), asio::detached);
ioc.poll();
auto wake = [&](channel& ch)
{
ch.try_send(boost::system::error_code());
ioc.poll();
};
row("asio per-op registration overhead",
ns_delta([&] { wake(armed); }, [&] { wake(unarmed); }));
}
{
// One task per cycle as for capy: a cancelled coroutine's state// stays cancelled too.
asio::io_context ioc(1);
auto work = asio::make_work_guard(ioc); // tasks end between cycles
asio::cancellation_signal sig;
channel ch(ioc);
auto cycle = [&](bool cancel)
{
asio::co_spawn(
ioc,
receive_once(ch),
asio::bind_cancellation_slot(sig.slot(), asio::detached));
ioc.poll();
if(cancel)
sig.emit(asio::cancellation_type::terminal);
else
ch.try_send(boost::system::error_code());
ioc.poll();
};
row("asio per-op cancellation overhead",
ns_delta([&] { cycle(true); }, [&] { cycle(false); }));
}
}
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Cancellation in an asynchronous application is ordinary control flow, not a rare event. It has two paths, and both run in the steady state of a normal program.
Registration
Every operation awaited under a live stop token registers a
std::stop_callbackwhen it has to wait and removes it when it completes, whether or not anything is ever cancelled. It is safe to say that in a well-designed cancellable asynchronous chain, registration happens on every operation that waits:Cancellation
It is pretty common to write operations where cancellation happens on every paths:
An operation with a deadline cancels either the timer when it finishes in time or the operation itself when it does not:
A race cancels every branch but the winner:
capy carries cancellation as a
std::stop_tokeninawait_context. A stop token is built for a request that may come from any thread at any moment, so registration and cancellation both synchronize with other threads, even when everything runs on one executor. That is a high price for two things that happen this often.Measurements
One P-core of an i7-13700, clang 24
-O3, libstdc++ 16, Asio 1.38.2. "In situ" is a coroutine parked onasync_event(on acorosio::io_context) or onasio::channel(on anasio::io_context).Each
timeout()call pays for a newstd::stop_source(one heap allocation), two or three registrations (the timer wait, the inner operation, and the parent token when it is live) and one cancellation.Where the cost comes from
std::stop_callbackis 6 locked RMWs in libstdc++: the state's refcount up and down, lock and unlock to link, lock and unlock to unlink.request_stop()locks the state, runs the callback, then releases abinary_semaphoreso that astop_callbackbeing destroyed on another thread can wait for it.async_event::set()doesclaimed_.exchange()for every waiter, to arbitrate against a stop callback running on another thread.new, so everytimeout()and everywhen_anypays for one. Asio's equivalents also allocate per call, but from its per-thread recycling cache.Why Asio's figures are lower
A
cancellation_slotis not thread-safe. It is written, cleared and emitted only on the operation's executor, and a requester on another thread posts the emit there (co_spawndoes this itself). Registration is a plain store and cancellation is an indirect call. Only a request from another thread pays for crossing threads.Reproduction
bench.cpp
All reactions