Skip to content

Core Concepts

Jump to bottom
github-actions[bot] edited this page Jun 23, 2026 · 34 revisions

Core Concepts

This page explains the fundamental concepts behind Elio's design.

Coroutines and Tasks

Elio uses C++20 coroutines as the foundation for async programming. A coroutine is a function that can suspend and resume execution.

The task<T> Type

coro::task<T> is the primary coroutine type in Elio:

#include <elio/coro/task.hpp>

// A task that returns an int
coro::task<int> compute() {
    co_return 42;
}

// A task that returns nothing
coro::task<void> do_work() {
    int result = co_await compute();
    ELIO_LOG_INFO("Result: {}", result);
    co_return;
}

Awaiting Tasks

Use co_await to wait for a task to complete:

coro::task<void> example() {
    // Sequential execution
    int a = co_await compute();
    int b = co_await compute();
    
    ELIO_LOG_INFO("Sum: {}", a + b);
    co_return;
}

Scheduler

The scheduler manages coroutine execution across multiple threads.

Using async_main (Recommended)

The simplest way to run async code is with ELIO_ASYNC_MAIN:

#include <elio/elio.hpp>

coro::task<int> async_main(int argc, char* argv[]) {
    // Access command line arguments
    if (argc > 1) {
        std::cout << "Argument: " << argv[1] << std::endl;
    }

    // Your async code here
    co_return 0;
}

ELIO_ASYNC_MAIN(async_main)

For async_main functions returning void:

coro::task<void> async_main(int argc, char* argv[]) {
    co_await do_work();
    co_return;
}

ELIO_ASYNC_MAIN(async_main)  // Same macro handles void return type

For simple programs without command line arguments:

coro::task<int> async_main() {
    co_return 0;
}

ELIO_ASYNC_MAIN(async_main)  // Same macro handles no-argument signature

Note: ELIO_ASYNC_MAIN uses compile-time dispatch to detect the callable's arity and return type, supporting all four combinations: task<int>(argc,argv), task<void>(argc,argv), task<int>(), task<void>().

Using elio::run()

For more control, use elio::run() directly:

int main(int argc, char* argv[]) {
    // Run with default thread count (hardware concurrency)
    return elio::run(async_main, argc, argv);
}

// Or use run_config for full control
int main(int argc, char* argv[]) {
    elio::run_config config;
    config.num_threads = 4;  // 4 worker threads
    
    return elio::run(config, async_main, argc, argv);
}

Manual Scheduler Control

For advanced use cases, you can manage the scheduler manually:

#include <elio/runtime/scheduler.hpp>

// Create with N worker threads
runtime::scheduler sched(4);

// Start the scheduler
sched.start();

// Spawn tasks (pass callable, not invoked task)
sched.go(my_coroutine);

// Shutdown when done
sched.shutdown();

Task Spawning

Elio provides several ways to spawn concurrent tasks:

Fire-and-Forget with elio::go()

Use elio::go() to spawn a task that runs independently:

coro::task<void> background_work() {
    // This runs in the background
    co_return;
}

coro::task<void> main_task() {
    // Spawn and continue immediately (don't wait)
    elio::go(background_work);
    
    // Lambda with captures is also safe
    int value = 42;
    elio::go([value]() -> coro::task<void> {
        // 'value' is safely copied into the coroutine frame
        co_return;
    });
    
    // Continue with other work...
    co_return;
}

Joinable Tasks with elio::spawn()

Use elio::spawn() to get a join_handle that lets you await the result later:

coro::task<int> compute(int x) {
    co_return x * 2;
}

coro::task<void> parallel_example() {
    // Spawn multiple tasks concurrently
    auto h1 = elio::spawn(compute, 10);
    auto h2 = elio::spawn(compute, 20);
    auto h3 = elio::spawn(compute, 30);
    
    // All three run in parallel
    // Now wait for results
    int a = co_await h1;  // 20
    int b = co_await h2;  // 40
    int c = co_await h3;  // 60
    
    ELIO_LOG_INFO("Sum: {}", a + b + c);  // 120
    co_return;
}

Pinned Spawn with elio::go_to()

Use elio::go_to() to spawn a task pinned to a specific worker thread. The task is placed directly on the target worker's queue with affinity set — it cannot be stolen by other workers.

coro::task<void> connection_handler(int fd) {
    // This entire task runs on the designated worker
    co_return;
}

coro::task<void> accept_loop() {
    int worker = 0;
    while (auto fd = co_await accept(...)) {
        // Round-robin connections across workers
        elio::go_to(worker % num_workers, connection_handler, fd);
        ++worker;
    }
}

This is useful when you need cache locality or want to partition work across workers at spawn time, without the brief migration window that go() + set_affinity() would have.

Checking Completion

You can check if a spawned task is done without blocking:

coro::task<void> poll_example() {
    auto handle = elio::spawn(slow_operation);
    
    while (!handle.is_ready()) {
        // Do other work while waiting
        co_await do_something_else();
    }
    
    auto result = co_await handle;
    co_return;
}

Work Stealing

Elio uses a work-stealing scheduler for load balancing. Each worker thread has a local queue, and idle workers steal tasks from busy workers.

Efficient Idle Waiting

When workers have no tasks to execute, they enter an efficient sleep state instead of busy-waiting:

  • Unified wake mechanism: Each worker's I/O backend contains an eventfd that external threads can write to, waking the worker from its I/O poll
  • Configurable wait strategy: Workers can spin before blocking (for low-latency) or block immediately (for CPU efficiency)
  • Automatic wake-up: When tasks are scheduled to a worker, it is automatically woken via eventfd write to the I/O backend
  • IO integration: Workers poll the IO backend (io_uring/epoll) before sleeping

This design ensures:

  • Near-zero CPU usage when idle with default blocking strategy (< 1%)
  • Configurable spin-before-block for low-latency workloads
  • Fast wake-up latency when new work arrives
  • Efficient coordination between task scheduling and IO polling

Thread Affinity

Tasks can be pinned to specific worker threads using the affinity API. A pinned task will not be stolen by other workers.

#include <elio/runtime/affinity.hpp>

coro::task<void> affinity_example() {
    // Bind to worker 2 and migrate there immediately
    co_await elio::runtime::set_affinity(2);

    // Bind to the current worker (prevent migration)
    co_await elio::runtime::bind_to_current_worker();

    // Remove affinity (allow free migration again)
    co_await elio::runtime::clear_affinity();

    co_return;
}

All three functions are also available in the elio namespace as convenience aliases (elio::set_affinity, elio::bind_to_current_worker, elio::clear_affinity).

set_affinity accepts an optional second parameter migrate (default true). When true, the coroutine is immediately rescheduled on the target worker. When false, the affinity is recorded but the coroutine continues on its current worker until its next suspension point.

Design Rationale

Why work-stealing. A centralized task queue becomes a contention bottleneck under high concurrency. Work-stealing distributes scheduling decisions: each worker thread operates on its own local deque, and idle workers steal from busy ones. This provides automatic load balancing without requiring a central coordinator. The owner-side LIFO order also has a cache-locality benefit -- the most recently pushed task is the one most likely to have hot data in the current core's cache.

Why Chase-Lev deque. The Chase-Lev algorithm gives the owner thread lock-free push and pop operations with no atomic read-modify-write on the fast path. Contention only occurs when the deque has a single element and both the owner and a thief attempt to take it simultaneously. The deque also supports dynamic resizing by replacing the underlying circular buffer, which avoids the need to pre-allocate a fixed upper bound.

Why MPSC inbox for external submissions. Cross-thread task submissions (e.g., spawning a task onto a specific worker from another thread) go through a bounded MPSC ring buffer rather than directly into the Chase-Lev deque. This separation keeps the deque's invariants simple -- only the owner ever pushes -- and the bounded capacity with cache-line aligned slots (alignas(64)) eliminates false sharing between producers and the consumer.

Virtual Stack

C++20 stackless coroutines do not maintain a call stack in the traditional sense. When a coroutine suspends, the compiler-generated frame is stored on the heap, but the chain of callers that led to that suspension point is lost. This makes debugging difficult -- tools like gdb bt show the scheduler's dispatch loop rather than the logical call chain of coroutines.

Elio reconstructs this information through a virtual stack: an intrusive linked list of promise_base objects connected by parent_ pointers. Each promise_base constructor links itself to the current frame via the current_frame_ thread-local, and the destructor restores the previous frame. This gives every coroutine a pointer to the coroutine that co_awaited it.

The overhead is minimal -- one pointer per coroutine frame, set during construction and cleared during destruction.

What it enables

  • elio-pstack: A CLI tool that attaches to a running process (or reads a coredump) and walks the virtual stack chains to print coroutine backtraces, similar to pstack for threads.
  • Debugger extensions: elio-gdb.py and elio_lldb.py use the same frame linkage to implement elio bt (backtrace) and elio list (list active coroutines).
  • Exception propagation: When a coroutine throws, unhandled_exception() captures it in the promise. The parent coroutine can then rethrow the exception when it co_awaits the child's result, propagating errors up the logical call chain.

Frame metadata

Each promise_base also carries debug metadata:

  • frame_magic_: A constant (0x454C494F46524D45, ASCII "ELIOFRME") that debuggers use to validate that a memory region is a live coroutine frame.
  • debug_id_: A lazily-allocated unique ID (batch-allocated per thread to avoid global contention).
  • debug_location_: Source file, function name, and line number.
  • debug_state_: Current state (created, running, suspended, completed, failed).

Coroutine Frame Allocation

Coroutine frames are heap-allocated by default using ::operator new/delete. When the optional vthread_stack allocator is enabled (gated by ELIO_ENABLE_VTHREAD_STACK), each vthread maintains its own segmented bump-pointer stack allocator. Coroutine frames are allocated in LIFO order within segments. When a segment is exhausted, a new segment is allocated and linked.

Sanitizer compatibility

Under AddressSanitizer or ThreadSanitizer, the custom allocator is bypassed entirely -- all allocations go directly through ::operator new and ::operator delete. This ensures that sanitizers can accurately detect leaks, use-after-free, and data races without being confused by the pooling layer. The sanitizer fallback is automatic via ELIO_SANITIZER_ACTIVE preprocessor detection.

I/O Context

The I/O context manages async I/O operations. Each worker thread has its own I/O context for lock-free operation. In most cases, you don't need to interact with io_context directly - the library automatically uses the per-thread io_context.

Accessing the Current Context

#include <elio/io/io_awaitables.hpp>

// Get the current thread's I/O context (from within a coroutine)
auto& ctx = io::current_io_context();

I/O Backends

Elio supports two I/O backends:

Backend Kernel Version Performance
io_uring Linux 5.1+ Best
epoll Any Linux Good

The backend is selected automatically at compile time based on availability.

Awaitables

Awaitables are types that can be used with co_await. Elio provides several built-in awaitables:

I/O Awaitables

// Read from a socket
auto result = co_await stream.read(buffer, size);

// Write to a socket
auto result = co_await stream.write(data, len);

// Accept a connection
auto stream = co_await listener.accept();

Timer Awaitables

#include <elio/time/timer.hpp>

// Sleep for a duration
co_await time::sleep_for(std::chrono::seconds(1));

// Sleep until a time point
co_await time::sleep_until(deadline);

Synchronization Primitives

Elio provides coroutine-aware synchronization primitives.

Mutex

#include <elio/sync/primitives.hpp>

sync::mutex mtx;

coro::task<void> critical_section() {
    co_await mtx.lock();
    // Protected code here
    mtx.unlock();
    co_return;
}

Shared Mutex (Read-Write Lock)

shared_mutex allows multiple concurrent readers or a single exclusive writer:

sync::shared_mutex rwlock;

// Multiple readers can run concurrently
coro::task<void> reader() {
    co_await rwlock.lock_shared();
    sync::shared_lock_guard guard(rwlock);  // RAII unlock
    // Read shared data
    co_return;
}

// Writers get exclusive access
coro::task<void> writer() {
    co_await rwlock.lock();
    sync::unique_lock_guard guard(rwlock);  // RAII unlock
    // Modify shared data
    co_return;
}

Event

event is a one-shot signaling primitive. One or more coroutines wait for the event to be set:

sync::event evt;

coro::task<void> waiter() {
    co_await evt.wait();
    // Event was set
    co_return;
}

coro::task<void> signaler() {
    // Wake all waiters
    evt.set();
    co_return;
}

Channel

channel<T> provides a multi-producer multi-consumer queue for passing values between coroutines. By default, channel<T> creates a rendezvous (synchronous) channel where send blocks until a matching recv is ready:

// Rendezvous channel (default): sender waits for receiver
sync::channel<int> ch;

// Bounded channel: buffers up to 16 items before back-pressure
sync::channel<int> bch(16);

// Unbounded channel: no back-pressure, may grow indefinitely
auto uch = sync::channel<int>::unbounded();

coro::task<void> producer() {
    co_await ch.send(42);
    co_return;
}

coro::task<void> consumer() {
    auto value = co_await ch.recv();
    // value == 42
    co_return;
}

Semaphore

sync::semaphore sem(10);  // Max 10 concurrent

coro::task<void> limited_work() {
    co_await sem.acquire();
    // Do work
    sem.release();
    co_return;
}

Spinlock

spinlock is a lightweight, non-suspending lock for very short critical sections. It uses a TTAS (Test-and-Test-and-Set) algorithm with CPU pause instructions for efficient spinning:

sync::spinlock sl;

coro::task<void> quick_update() {
    sl.lock();          // Spins until acquired (does not suspend coroutine)
    // Very short critical section
    sl.unlock();
    co_return;
}

Use spinlock_guard for RAII-style locking:

sync::spinlock sl;

coro::task<void> safe_update() {
    sync::spinlock_guard guard(sl);  // Locks on construction
    // Critical section
    co_return;
    // Automatically unlocked on destruction
}

When to use spinlock vs mutex: Use spinlock when the critical section is very short (a few assignments or pointer swaps) and contention is low. Use mutex when the critical section might suspend (e.g., performing I/O) or when contention is high — mutex suspends the coroutine instead of busy-waiting, allowing other coroutines to run.

Cancellation Safety

All coroutine-aware synchronization primitives (event, mutex, semaphore, condition_variable, channel, shared_mutex) are cancellation-safe: if a waiting coroutine is destroyed (due to cancellation, timeout, or forced termination) before it is woken, it is automatically unlinked from the primitive's internal waiter list. The primitive can then safely call its wake function (set(), unlock(), release(), notify_one(), send(), etc.) without risk of use-after-free.

This is implemented via an intrusive linked list: each waiter node is embedded in the coroutine frame and registers itself with the primitive on suspension. If the coroutine is destroyed before being woken, the waiter's destructor acquires the primitive's mutex and removes itself from the list — ensuring the primitive never holds a dangling handle.

sync::event evt;

// This is safe even if the waiter times out and is destroyed:
coro::task<void> waiter_with_timeout() {
    auto result = co_await with_timeout(5s, [](cancel_token tok) -> task<void> {
        co_await evt.wait();
    });
    if (!result) {  // timed out
        co_return;  // Waiter destroyed — event::set() remains safe
    }
}

Key guarantees:

  • Destroying a waiting coroutine does not invalidate the primitive's waiter list
  • Calling set()/unlock()/release()/notify_one()/notify_all() after a waiter is destroyed is safe — the destroyed waiter is simply skipped
  • No manual cleanup is required — unlinking happens automatically in the waiter's destructor

Condition Variable

condition_variable allows coroutines to wait for a condition to become true. It works with mutex, spinlock, or in an unlocked mode:

With mutex (recommended for most cases):

sync::mutex mtx;
sync::condition_variable cv;
bool ready = false;

coro::task<void> waiter() {
    co_await mtx.lock();
    while (!ready) {
        // Single co_await: wait() returns task<void>
        co_await cv.wait(mtx);
    }
    mtx.unlock();
    co_return;
}

coro::task<void> notifier() {
    co_await mtx.lock();
    ready = true;
    mtx.unlock();
    cv.notify_one();  // Wake one waiter
    co_return;
}

With spinlock:

sync::spinlock sl;
sync::condition_variable cv;
bool ready = false;

coro::task<void> waiter() {
    sl.lock();
    while (!ready) {
        co_await cv.wait(sl);  // Single co_await (spinlock re-lock is synchronous)
    }
    sl.unlock();
    co_return;
}

Unlocked mode (for single-worker scenarios where all coroutines run on the same thread):

sync::condition_variable cv;
bool ready = false;

coro::task<void> waiter() {
    while (!ready) {
        co_await cv.wait_unlocked();
    }
    co_return;
}

notify_one() wakes exactly one waiting coroutine; notify_all() wakes all of them.

Cancellation

Elio provides a cooperative cancellation mechanism for async operations using cancel_source and cancel_token.

Basic Usage

#include <elio/coro/cancel_token.hpp>

coro::task<void> cancellable_work(coro::cancel_token token) {
    while (!token.is_cancelled()) {
        // Do work...
        
        // Cancellable sleep - returns early if cancelled
        auto result = co_await time::sleep_for(100ms, token);
        if (result == coro::cancel_result::cancelled) {
            break;
        }
    }
    co_return;
}

coro::task<void> controller() {
    coro::cancel_source source;
    
    // Start work with a token
    elio::go(cancellable_work, source.get_token());
    
    // Wait some time
    co_await time::sleep_for(5s);
    
    // Cancel the operation
    source.cancel();
    co_return;
}

How It Works

  1. cancel_source - Creates and controls cancellation state
  2. cancel_token - Lightweight handle passed to operations
  3. Operations periodically check token.is_cancelled()
  4. Calling source.cancel() triggers all registered callbacks

Cancellable Operations

Many Elio operations support cancellation:

// Sleep with cancellation
auto result = co_await time::sleep_for(1s, token);

// HTTP request with cancellation
auto response = co_await client.get(url, token);

// RPC call with cancellation
auto result = co_await rpc_client->call<Method>(req, timeout, token);

// WebSocket receive with cancellation
auto msg = co_await ws_client.receive(token);

// SSE event receive with cancellation
auto event = co_await sse_client.receive(token);

Implementing Cancellable Operations

Register callbacks to respond to cancellation:

coro::task<void> custom_operation(coro::cancel_token token) {
    // Register a callback
    auto reg = token.on_cancel([&]() {
        // Cleanup or signal early exit
    });
    
    // Do work...
    
    // Registration automatically unregisters on destruction
    co_return;
}

Error Handling

Elio uses io_result for I/O operation results. io_result contains an int32_t result field (bytes transferred or negative errno) and a uint32_t flags field. Use result.success() to check for errors, result.bytes_transferred() for byte count, and result.error_code() for the errno value. Factory methods like tcp_listener::bind() use std::optional.

coro::task<void> handle_errors() {
    auto result = co_await stream.read(buffer, size);
    
    if (result.result > 0) {
        // Success - result.result contains bytes read
    } else if (result.result == 0) {
        // EOF - connection closed
    } else {
        // Error - result.result is negative errno
        ELIO_LOG_ERROR("Read error: {}", strerror(-result.result));
    }
    co_return;
}

For factory methods like tcp_listener::bind(), check for std::nullopt and use errno to get the error code:

auto listener = tcp_listener::bind(ipv4_address(port));
if (!listener) {
    ELIO_LOG_ERROR("Bind failed: {}", strerror(errno));
    co_return;
}

Logging

Elio includes a built-in logging system:

#include <elio/log/macros.hpp>

ELIO_LOG_DEBUG("Debug message");
ELIO_LOG_INFO("Info: value={}", 42);
ELIO_LOG_WARNING("Warning!");
ELIO_LOG_ERROR("Error: {}", strerror(errno));

// Set log level
log::logger::instance().set_level(log::level::debug);

Next Steps

Clone this wiki locally