Core Concepts

This page explains the fundamental concepts behind Elio's design.

Coroutines and Tasks

Elio uses C++20 coroutines as the foundation for async programming. A coroutine is a function that can suspend and resume execution.

The `task<T>` Type

coro::task<T> is the primary coroutine type in Elio:

#include <elio/coro/task.hpp>

// A task that returns an int
coro::task<int> compute() {
    co_return 42;
}

// A task that returns nothing
coro::task<void> do_work() {
    int result = co_await compute();
    ELIO_LOG_INFO("Result: {}", result);
    co_return;
}

task<T> is a move-only, single-shot lazy owner. Moving it transfers the unstarted coroutine frame and leaves the source empty, which allows lazy tasks to be stored in containers or passed through move-only callables. The task object does not represent running work: once ownership is transferred to the scheduler, use the join_handle<T> returned by spawn() when the work must be observed. Moving a task never migrates a running coroutine or pending I/O.

Awaiting Tasks

Use co_await to wait for a task to complete:

coro::task<void> example() {
    // Sequential execution
    int a = co_await compute();
    int b = co_await compute();
    
    ELIO_LOG_INFO("Sum: {}", a + b);
    co_return;
}

Scheduler

The scheduler manages coroutine execution across multiple threads.

Using async_main (Recommended)

The simplest way to run async code is with ELIO_ASYNC_MAIN:

#include <elio/elio.hpp>

coro::task<int> async_main(int argc, char* argv[]) {
    // Access command line arguments
    if (argc > 1) {
        std::cout << "Argument: " << argv[1] << std::endl;
    }

    // Your async code here
    co_return 0;
}

ELIO_ASYNC_MAIN(async_main)

For async_main functions returning void:

coro::task<void> async_main(int argc, char* argv[]) {
    co_await do_work();
    co_return;
}

ELIO_ASYNC_MAIN(async_main)  // Same macro handles void return type

For simple programs without command line arguments:

coro::task<int> async_main() {
    co_return 0;
}

ELIO_ASYNC_MAIN(async_main)  // Same macro handles no-argument signature

Note: ELIO_ASYNC_MAIN uses compile-time dispatch to detect the callable's arity and return type, supporting all four combinations: task<int>(argc,argv), task<void>(argc,argv), task<int>(), task<void>().

Using elio::run()

For more control, use elio::run() directly:

int main(int argc, char* argv[]) {
    // Run with default thread count (hardware concurrency)
    return elio::run(async_main, argc, argv);
}

// Or use run_config for full control
int main(int argc, char* argv[]) {
    elio::run_config config;
    config.num_threads = 4;  // 4 worker threads
    
    return elio::run(config, async_main, argc, argv);
}

Manual Scheduler Control

For advanced use cases, you can manage the scheduler manually:

#include <elio/runtime/scheduler.hpp>

// Create with N worker threads
runtime::scheduler sched(4);

// Start the scheduler
sched.start();

// Spawn tasks (pass callable, not invoked task)
sched.go(my_coroutine);

// Shutdown when done
sched.shutdown();

Task Spawning

Elio provides several ways to spawn concurrent tasks:

Fire-and-Forget with `elio::go()`

Use elio::go() to spawn a task that runs independently:

coro::task<void> background_work() {
    // This runs in the background
    co_return;
}

coro::task<void> main_task() {
    // Spawn and continue immediately (don't wait)
    elio::go(background_work);
    
    // Lambda with captures is also safe
    int value = 42;
    elio::go([value]() -> coro::task<void> {
        // 'value' is safely copied into the coroutine frame
        co_return;
    });
    
    // Continue with other work...
    co_return;
}

Joinable Tasks with `elio::spawn()`

Use elio::spawn() to get a join_handle that lets you await the result later:

coro::task<int> compute(int x) {
    co_return x * 2;
}

coro::task<void> parallel_example() {
    // Spawn multiple tasks concurrently
    auto h1 = elio::spawn(compute, 10);
    auto h2 = elio::spawn(compute, 20);
    auto h3 = elio::spawn(compute, 30);
    
    // All three run in parallel
    // Now wait for results
    int a = co_await h1;  // 20
    int b = co_await h2;  // 40
    int c = co_await h3;  // 60
    
    ELIO_LOG_INFO("Sum: {}", a + b + c);  // 120
    co_return;
}

Affinity Spawn with `elio::go_to()`

Use elio::go_to() to spawn a task with affinity to a specific worker thread. The task is placed on the target worker's queue with affinity set before it first resumes. If a steal attempt later observes the task on another queue, the scheduler requeues it to the affinity worker instead of executing it on the wrong worker.

For deterministic placement, pass a worker id in [0, scheduler.num_threads()). Out-of-range ids are not rejected, but they are treated as scheduler fallback behavior rather than a precise pinning contract.

coro::task<void> connection_handler(int fd) {
    // This entire task runs on the designated worker
    co_return;
}

coro::task<void> accept_loop() {
    int worker = 0;
    while (auto fd = co_await accept(...)) {
        // Round-robin connections across workers
        elio::go_to(worker % num_workers, connection_handler, fd);
        ++worker;
    }
}

This is useful when you need cache locality or want to partition work across workers at spawn time, without the brief migration window that go() + set_affinity() would have.

Checking Completion

You can check if a spawned task is done without blocking:

coro::task<void> poll_example() {
    auto handle = elio::spawn(slow_operation);
    
    while (!handle.is_ready()) {
        // Do other work while waiting
        co_await do_something_else();
    }
    
    auto result = co_await handle;
    co_return;
}

Work Stealing

Elio uses a work-stealing scheduler for load balancing. Each worker thread has a local queue, and idle workers steal tasks from busy workers.

Efficient Idle Waiting

When workers have no tasks to execute, they enter an efficient sleep state instead of busy-waiting:

Unified wake mechanism: Each worker's I/O backend contains an eventfd that external threads can write to, waking the worker from its I/O poll
Configurable wait strategy: Workers can spin before blocking (for low-latency) or block immediately (for CPU efficiency)
Automatic wake-up: When tasks are scheduled to a worker, it is automatically woken via eventfd write to the I/O backend
IO integration: Workers poll the IO backend (io_uring/epoll) before sleeping

This design ensures:

Near-zero CPU usage when idle with default blocking strategy (< 1%)
Configurable spin-before-block for low-latency workloads
Fast wake-up latency when new work arrives
Efficient coordination between task scheduling and IO polling

Thread Affinity

Tasks can request a specific worker thread using the affinity API. The scheduler respects affinity during stealing: a stolen affinity task is requeued to its bound worker instead of being executed by the thief.

#include <elio/runtime/affinity.hpp>

coro::task<void> affinity_example() {
    // Bind to worker 2 and migrate there immediately
    co_await elio::runtime::set_affinity(2);

    // Bind to the current worker (prevent migration)
    co_await elio::runtime::bind_to_current_worker();

    // Remove affinity (allow free migration again)
    co_await elio::runtime::clear_affinity();

    co_return;
}

All three functions are also available in the elio namespace as convenience aliases (elio::set_affinity, elio::bind_to_current_worker, elio::clear_affinity).

set_affinity accepts an optional second parameter migrate (default true). When true, the coroutine is immediately rescheduled on the target worker. When false, the affinity is recorded but the coroutine continues on its current worker until its next suspension point.

Design Rationale

Why work-stealing. A centralized task queue becomes a contention bottleneck under high concurrency. Work-stealing distributes scheduling decisions: each worker thread operates on its own local deque, and idle workers steal from busy ones. This provides automatic load balancing without requiring a central coordinator. The owner-side LIFO order also has a cache-locality benefit -- the most recently pushed task is the one most likely to have hot data in the current core's cache.

Why Chase-Lev deque. The Chase-Lev algorithm gives the owner thread lock-free push and pop operations with no atomic read-modify-write on the fast path. Contention only occurs when the deque has a single element and both the owner and a thief attempt to take it simultaneously. The deque also supports dynamic resizing by replacing the underlying circular buffer, which avoids the need to pre-allocate a fixed upper bound.

Why MPSC inbox for external submissions. Cross-thread task submissions (e.g., spawning a task onto a specific worker from another thread) go through a bounded MPSC ring buffer rather than directly into the Chase-Lev deque. This separation keeps the deque's invariants simple -- only the owner ever pushes -- and the bounded capacity with cache-line aligned slots (alignas(64)) eliminates false sharing between producers and the consumer. If a sustained burst fills the ring after bounded retries, submissions spill into a locked per-worker overflow queue. This rare slow path preserves worker affinity and borrowed coroutine ownership instead of resuming on the submitting thread.

Vthreads And The Virtual Stack

An Elio vthread is the logical asynchronous execution represented by a root task and its chain of awaited coroutine tasks. It is not an operating system thread and it is not a coroutine-frame allocation arena. A vthread may suspend and resume on scheduler workers, and may migrate between workers when its affinity and I/O ownership permit that movement.

C++20 stackless coroutines do not maintain a call stack in the traditional sense. When a coroutine suspends, the compiler-generated frame is stored on the heap, but the chain of callers that led to that suspension point is lost. This makes debugging difficult -- tools like gdb bt show the scheduler's dispatch loop rather than the logical call chain of coroutines.

Elio reconstructs this information through a virtual stack: an intrusive linked list of promise_base objects connected by parent_ pointers. The current_frame_ thread-local identifies the frame executing on a thread. A lazy task<T> does not remain installed merely because its frame is owned; when it is awaited, its parent_ is rebound to the actual awaiting coroutine for that execution chain. Scheduler, synchronization, I/O, and affinity-migration resume paths preserve that parent and install the resumed frame for the duration of the resume call. Generator iteration similarly binds the producer to its active consumer and restores the consumer context before transferring a yielded value or completion. Initial scheduler ownership handoff, including targeted spawn, is separate and detaches construction-time ancestry.

The overhead is minimal -- one pointer per coroutine frame, updated when logical ancestry is established.

What it enables

elio-pstack: A CLI tool that attaches to a running process (or reads a coredump) and walks the virtual stack chains to print coroutine backtraces, similar to pstack for threads.
Debugger extensions: elio-gdb.py and elio-lldb.py use the same frame linkage to implement elio bt (backtrace) and elio list (list active coroutines).
Exception propagation: When a coroutine throws, unhandled_exception() captures it in the promise. The parent coroutine can then rethrow the exception when it co_awaits the child's result, propagating errors up the logical call chain.

Frame metadata

Each promise_base also carries debug metadata:

frame_magic_: A constant (0x454C494F46524D45, ASCII "ELIOFRME") that debuggers use to validate that a memory region is a live coroutine frame.
debug_id_: A lazily-allocated unique ID (batch-allocated per thread to avoid global contention).
debug_location_: Source file, function name, and line number.
debug_state_: Current state (created, running, suspended, completed, failed).

Coroutine Frame Allocation

coro::task frames use the implementation's standard coroutine heap allocation path in every build. Elio does not pool task frames or require nested frames to be destroyed in LIFO order. A frame may be destroyed on a different worker from the one that created or last resumed it, provided ownership is transferred and the frame is destroyed exactly once.

Frame allocation is independent of the vthread abstraction described above. Vthreads still maintain a logical virtual stack through the promise_base parent chain; that chain tracks coroutine ancestry but does not own or allocate the coroutine frames.

Sanitizer compatibility

AddressSanitizer and ThreadSanitizer observe the same task-frame allocation and deallocation path as normal builds. There is no sanitizer-specific allocator mode, so sanitizer coverage matches production frame lifetime behavior.

I/O Context

The I/O context manages async I/O operations. Each worker thread has its own I/O context for lock-free operation. In most cases, you don't need to interact with io_context directly - the library automatically uses the per-thread io_context.

Accessing the Current Context

#include <elio/io/io_awaitables.hpp>

// Get the current thread's I/O context (from within a coroutine)
auto& ctx = io::current_io_context();

I/O Backends

Elio supports two I/O backends:

Backend	Kernel Version	Performance
io_uring	Linux 5.1+	Best
epoll	Any Linux	Good

The backend is selected automatically at compile time based on availability.

Awaitables

Awaitables are types that can be used with co_await. Elio provides several built-in awaitables:

I/O Awaitables

// Read from a socket
auto result = co_await stream.read(buffer, size);

// Write to a socket
auto result = co_await stream.write(data, len);

// Accept a connection
auto stream = co_await listener.accept();

Timer Awaitables

#include <elio/time/timer.hpp>

// Sleep for a duration
co_await time::sleep_for(std::chrono::seconds(1));

// Sleep until a time point
co_await time::sleep_until(deadline);

Synchronization Primitives

Elio provides coroutine-aware synchronization primitives.

Mutex

#include <elio/sync/primitives.hpp>

sync::mutex mtx;

coro::task<void> critical_section() {
    co_await mtx.lock();
    // Protected code here
    mtx.unlock();
    co_return;
}

Shared Mutex (Read-Write Lock)

shared_mutex allows multiple concurrent readers or a single exclusive writer:

sync::shared_mutex rwlock;

// Multiple readers can run concurrently
coro::task<void> reader() {
    co_await rwlock.lock_shared();
    sync::shared_lock_guard guard(rwlock);  // RAII unlock
    // Read shared data
    co_return;
}

// Writers get exclusive access
coro::task<void> writer() {
    co_await rwlock.lock();
    sync::unique_lock_guard guard(rwlock);  // RAII unlock
    // Modify shared data
    co_return;
}

Event

event is a one-shot signaling primitive. One or more coroutines wait for the event to be set:

sync::event evt;

coro::task<void> waiter() {
    co_await evt.wait();
    // Event was set
    co_return;
}

coro::task<void> signaler() {
    // Wake all waiters
    evt.set();
    co_return;
}

Channel

channel<T> provides a multi-producer multi-consumer queue for passing values between coroutines. By default, channel<T> creates a rendezvous (synchronous) channel where send blocks until a matching recv is ready:

// Rendezvous channel (default): sender waits for receiver
sync::channel<int> ch;

// Bounded channel: buffers up to 16 items before back-pressure
sync::channel<int> bch(16);

// Unbounded channel: no back-pressure, may grow indefinitely
auto uch = sync::channel<int>::unbounded();

coro::task<void> producer() {
    co_await ch.send(42);
    co_return;
}

coro::task<void> consumer() {
    auto value = co_await ch.recv();
    // value == 42
    co_return;
}

Semaphore

sync::semaphore sem(10);  // Max 10 concurrent

coro::task<void> limited_work() {
    co_await sem.acquire();
    // Do work
    sem.release();
    co_return;
}

Spinlock

spinlock is a lightweight, non-suspending lock for very short critical sections. It uses a TTAS (Test-and-Test-and-Set) algorithm with CPU pause instructions for efficient spinning:

sync::spinlock sl;

coro::task<void> quick_update() {
    sl.lock();          // Spins until acquired (does not suspend coroutine)
    // Very short critical section
    sl.unlock();
    co_return;
}

Use spinlock_guard for RAII-style locking:

sync::spinlock sl;

coro::task<void> safe_update() {
    sync::spinlock_guard guard(sl);  // Locks on construction
    // Critical section
    co_return;
    // Automatically unlocked on destruction
}

When to use spinlock vs mutex: Use spinlock when the critical section is very short (a few assignments or pointer swaps) and contention is low. Use mutex when the critical section might suspend (e.g., performing I/O) or when contention is high — mutex suspends the coroutine instead of busy-waiting, allowing other coroutines to run.

Cancellation Safety

Coroutine-aware synchronization primitives (event, mutex, semaphore, condition_variable, channel, shared_mutex) embed waiter nodes in the suspended coroutine frame. If a frame is destroyed while its waiter is still linked, the awaiter's destructor removes that node from the primitive's internal waiter list.

This lifetime cleanup is not itself a cancellation API. with_timeout() requests cooperative cancellation when its timer wins; it does not forcibly destroy the losing child task. The child stops promptly only when it passes the supplied cancel_token to an operation that supports cancellation.

In particular, event::wait() has no cancel_token overload. Wrapping it in with_timeout() does not cancel or destroy the event waiter when the timeout wins; that child remains suspended and linked until the event is set.

The event and every object captured by that child must therefore outlive the pending wait. They must remain alive until the child has resumed past event::wait(), including the interval after set() dequeues the waiter but before the scheduler resumes it.

Key guarantees:

Destroying a frame whose waiter is still linked removes that waiter from the primitive's list
No manual waiter-list cleanup is required; unlinking happens in the awaiter's destructor
Timeout cancellation remains cooperative and requires a token-aware operation

Condition Variable

condition_variable allows coroutines to wait for a condition to become true. It works with mutex, spinlock, or in an unlocked mode:

With mutex (recommended for most cases):

sync::mutex mtx;
sync::condition_variable cv;
bool ready = false;

coro::task<void> waiter() {
    co_await mtx.lock();
    while (!ready) {
        // Single co_await: wait() returns task<void>
        co_await cv.wait(mtx);
    }
    mtx.unlock();
    co_return;
}

coro::task<void> notifier() {
    co_await mtx.lock();
    ready = true;
    mtx.unlock();
    cv.notify_one();  // Wake one waiter
    co_return;
}

With spinlock:

sync::spinlock sl;
sync::condition_variable cv;
bool ready = false;

coro::task<void> waiter() {
    sl.lock();
    while (!ready) {
        co_await cv.wait(sl);  // Single co_await (spinlock re-lock is synchronous)
    }
    sl.unlock();
    co_return;
}

Unlocked mode (for single-worker scenarios where all coroutines run on the same thread):

sync::condition_variable cv;
bool ready = false;

coro::task<void> waiter() {
    while (!ready) {
        co_await cv.wait_unlocked();
    }
    co_return;
}

notify_one() wakes exactly one waiting coroutine; notify_all() wakes all of them.

Cancellation

Elio provides a cooperative cancellation mechanism for async operations using cancel_source and cancel_token.

Basic Usage

#include <elio/coro/cancel_token.hpp>

coro::task<void> cancellable_work(coro::cancel_token token) {
    while (!token.is_cancelled()) {
        // Do work...
        
        // Cancellable sleep - returns early if cancelled
        auto result = co_await time::sleep_for(100ms, token);
        if (result == coro::cancel_result::cancelled) {
            break;
        }
    }
    co_return;
}

coro::task<void> controller() {
    coro::cancel_source source;
    
    // Start work with a token
    elio::go(cancellable_work, source.get_token());
    
    // Wait some time
    co_await time::sleep_for(5s);
    
    // Cancel the operation
    source.cancel();
    co_return;
}

How It Works

cancel_source - Creates and controls cancellation state
cancel_token - Lightweight handle passed to operations
Operations periodically check token.is_cancelled()
Calling source.cancel() triggers all registered callbacks

Cancellable Operations

Many Elio operations support cancellation:

// Sleep with cancellation
auto result = co_await time::sleep_for(1s, token);

// HTTP request with cancellation
auto response = co_await client.get(url, token);

// RPC call with cancellation
auto result = co_await rpc_client->call<Method>(req, timeout, token);

// WebSocket connect with cancellation
bool connected = co_await ws_client.connect(ws_url, token);

// WebSocket receive with cancellation
auto msg = co_await ws_client.receive(token);

// SSE connect with cancellation
bool listening = co_await sse_client.connect(sse_url, token);

// SSE event receive with cancellation
auto event = co_await sse_client.receive(token);

For client networking operations, cancellation is wired into pending I/O or the operation's protocol. Cancelling the token passed to an RPC call returns rpc_error::cancelled locally and sends a best-effort RPC cancel frame when the request was already sent; context-aware server handlers observe that through rpc_context::cancel_token. Cancelling the token passed to an HTTP request, WebSocket connect(), or SSE connect() can abort TCP connect, TLS handshake, request write, and response header/body reads. Cancelling the token passed to WebSocket or SSE receive() can abort a blocked frame/event read. These operations report cancellation via their normal failure return and set errno to ECANCELED where applicable.

Implementing Cancellable Operations

Register callbacks to respond to cancellation:

coro::task<void> custom_operation(coro::cancel_token token) {
    // Register a callback
    auto reg = token.on_cancel([&]() {
        // Cleanup or signal early exit
    });
    
    // Do work...
    
    // Registration automatically unregisters on destruction
    co_return;
}

Error Handling

Elio uses io_result for I/O operation results. io_result contains an int32_t result field (bytes transferred or negative errno) and a uint32_t flags field. Use result.success() to check for errors, result.bytes_transferred() for byte count, and result.error_code() for the errno value. Factory methods like tcp_listener::bind() use std::optional.

coro::task<void> handle_errors() {
    auto result = co_await stream.read(buffer, size);
    
    if (result.result > 0) {
        // Success - result.result contains bytes read
    } else if (result.result == 0) {
        // EOF - connection closed
    } else {
        // Error - result.result is negative errno
        ELIO_LOG_ERROR("Read error: {}", strerror(-result.result));
    }
    co_return;
}

For factory methods like tcp_listener::bind(), check for std::nullopt and use errno to get the error code:

auto listener = tcp_listener::bind(ipv4_address(port));
if (!listener) {
    ELIO_LOG_ERROR("Bind failed: {}", strerror(errno));
    co_return;
}

Logging

Elio includes a built-in logging system:

#include <elio/log/macros.hpp>

ELIO_LOG_DEBUG("Debug message");
ELIO_LOG_INFO("Info: value={}", 42);
ELIO_LOG_WARNING("Warning!");
ELIO_LOG_ERROR("Error: {}", strerror(errno));

// Set log level
log::logger::instance().set_level(log::level::debug);

Next Steps

Learn about Networking for TCP and HTTP
See Examples for complete code samples

Core Concepts

Core Concepts

Coroutines and Tasks

The task<T> Type

Awaiting Tasks

Scheduler

Using async_main (Recommended)

Using elio::run()

Manual Scheduler Control

Task Spawning

Fire-and-Forget with elio::go()

Joinable Tasks with elio::spawn()

Affinity Spawn with elio::go_to()

Checking Completion

Work Stealing

Efficient Idle Waiting

Thread Affinity

Design Rationale

Vthreads And The Virtual Stack

What it enables

Frame metadata

Coroutine Frame Allocation

Sanitizer compatibility

I/O Context

Accessing the Current Context

I/O Backends

Awaitables

I/O Awaitables

Timer Awaitables

Synchronization Primitives

Mutex

Shared Mutex (Read-Write Lock)

Event

Channel

Semaphore

Spinlock

Cancellation Safety

Condition Variable

Cancellation

Basic Usage

How It Works

Cancellable Operations

Implementing Cancellable Operations

Error Handling

Logging

Next Steps

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

The `task<T>` Type

Fire-and-Forget with `elio::go()`

Joinable Tasks with `elio::spawn()`

Affinity Spawn with `elio::go_to()`