# Performance Tuning Guide

This guide covers performance optimization techniques for Elio applications.

## Overview

Elio is designed for high performance through:
- Lock-free data structures (Chase-Lev deque)
- Work-stealing scheduler
- Efficient I/O backends (io_uring, epoll)
- Standard coroutine frame allocation with unrestricted destruction order
- Minimal synchronization overhead

## Actual Performance Numbers

### Scheduling Benchmarks

| Operation | Typical | Best Case | Notes |
|-----------|---------|-----------|-------|
| Task Spawn | Benchmark-dependent | - | Standard heap frames |
| Context Switch | ~230 ns | ~212 ns | Suspend and resume |
| Yield | ~30 ns | ~16 ns | Per 1000 vthreads |
| MPSC push | ~5 ns | - | Cross-thread scheduling |
| Chase-Lev push | ~13 ns | - | Local queue operation |

### I/O Benchmarks

| Scenario | Latency | Throughput |
|----------|---------|------------|
| Single-thread file read | 1.46 μs/read | 685K IOPS |
| 4-thread concurrent read | 0.93 μs/read | 1.07M IOPS |

### Scalability

CPU-bound workload with 100K iterations per task:

| Threads | Throughput | Speedup |
|---------|-----------|---------|
| 1 | ~18K tasks/sec | 1.0x |
| 2 | ~33K tasks/sec | 1.9x |
| 4 | ~56K tasks/sec | 3.2x |
| 8 | ~86K tasks/sec | 4.9x |

Scaling efficiency depends on workload characteristics. Tasks with more computation relative to scheduling overhead will show better scaling.

### Wake-up Mechanism

Elio uses an **eventfd embedded in each worker's I/O backend** (epoll/io_uring) for cross-thread notifications. This provides a single unified wait point — both I/O completions and task wake-ups unblock the same `poll()` call, eliminating the latency gap that exists with separate wait mechanisms. The eventfd counter deduplicates wakes, making unconditional wake safe and minimizing scheduling overhead.

## Built-in Optimizations

### Unconditional Wake

Workers track their idle state. Task submissions always trigger wake syscalls on cross-thread submit; the eventfd counter deduplicates, making unconditional wake safe:

```cpp
// In worker_thread::schedule()
if (inbox_->push(handle.address())) {
    // Always wake on cross-thread submit. The eventfd dedupes via its
    // counter — calling wake() on a busy worker just bumps the counter.
    wake();
}
```

The previous lazy wake optimization was removed due to a race condition that caused 10ms tail latency on weak hardware.

The MPSC ring remains the normal submission path. If it stays full after
bounded retries, the worker accepts the handle through a locked overflow queue.
That slow path trades burst-only synchronization for correctness: a borrowed
resume handle stays assigned to its worker rather than running on the producer
thread or being destroyed.

### Unified Wake Mechanism

Each worker's I/O backend (epoll or io_uring) contains an embedded `eventfd`. When a task is submitted to a worker from another thread, the submitter writes to that worker's eventfd. Because the eventfd is registered with the same epoll/io_uring instance that handles I/O completions, both I/O events and task wake-ups unblock the same `poll()` call.

This unified design has two key benefits:

1. **Single wait point.** A worker blocked on I/O poll is immediately woken by a cross-thread task submission. There is no separate condition variable or futex that could introduce a latency gap between "I/O ready" and "task ready" paths.

2. **Safe unconditional wake.** The eventfd counter deduplicates wakes — calling `wake()` on a busy worker just bumps the counter, which is consumed on the next poll return. This eliminates the race condition that existed with the previous lazy wake optimization.

The result is that cross-thread scheduling latency equals one `eventfd_write` plus one `epoll_wait`/`io_uring_enter` return — typically under 5 microseconds.

### Wait Strategy

Elio supports configurable wait strategies to balance latency vs CPU usage:

```cpp
#include <elio/runtime/scheduler.hpp>
#include <elio/runtime/wait_strategy.hpp>

using namespace elio::runtime;

// Pure blocking (default) - lowest CPU usage
scheduler sched(4, wait_strategy::blocking());

// Hybrid spin-then-block - good for low-latency workloads
// Spins for 1000 iterations with yield, then blocks on I/O poll
scheduler sched(4, wait_strategy::hybrid(1000));

// Aggressive spinning - ultra-low latency (uses pause instruction)
scheduler sched(4, wait_strategy::spinning(1000));

// Custom strategy
wait_strategy custom{
    .spin_iterations = 500,   // Spin count before blocking
    .spin_yield = true       // Yield during spin (friendlier to other threads)
};
scheduler sched(4, custom);
```

**Strategy Selection Guide:**

| Strategy | CPU Usage | Wake Latency | Use Case |
|----------|-----------|--------------|----------|
| `blocking()` | Lowest | ~1-10 μs | General workloads (default) |
| `hybrid(N)` | Low-Medium | ~1-5 μs | Latency-sensitive with mixed load |
| `spinning(N)` | High | ~100-500 ns | Ultra-low latency, dedicated CPUs |
| `aggressive(N)` | Medium-High | ~100-1000 ns | Low latency, shared CPUs |

The `spin_yield` flag controls whether the spin phase uses `std::this_thread::yield()` (true) or the CPU pause instruction (false). Yielding is friendlier to other threads but slightly slower.

**Runtime Configuration:**

```cpp
// Change per-worker strategy at runtime
auto* worker = sched.get_worker(0);
worker->set_wait_strategy(wait_strategy::spinning(2000));
```

### io_uring Batch Submit

I/O operations are automatically batched:

```cpp
// In io_uring_backend::poll()
// Auto-submit any pending operations before waiting
if (io_uring_sq_ready(&ring_) > 0) {
    io_uring_submit(&ring_);
}
```

This reduces the number of `io_uring_submit` syscalls by batching multiple operations.

### Lazy Debug ID Allocation

Debug IDs for coroutines are only allocated when actually accessed, reducing creation overhead in production:

```cpp
// debug_id_ initialized to 0, allocated on first id() call
// Only available when ELIO_ENABLE_DEBUG_METADATA is enabled
uint64_t id() noexcept {
    if (debug_id_ == 0) {
        debug_id_ = id_allocator::allocate();
    }
    return debug_id_;
}
```

### Optimized Yield Path

Yielding skips affinity checks and scheduler lookups for better performance:

```cpp
// In yield_awaitable::await_suspend()
auto* worker = runtime::worker_thread::current();
if (worker) {
    // Fast path: directly schedule to local queue
    worker->schedule_local(awaiter);
    return;
}
// Slow path only when no current worker
```

## Scheduler Tuning

### Thread Count

```cpp
#include <elio/runtime/scheduler.hpp>

// Default: matches hardware concurrency
scheduler sched;

// Custom thread count
scheduler sched(8);  // 8 worker threads

// For I/O-bound workloads, consider more threads than cores
scheduler sched(std::thread::hardware_concurrency() * 2);

// For CPU-bound workloads, match core count
scheduler sched(std::thread::hardware_concurrency());
```

### Dynamic Thread Adjustment

The scheduler supports changing the worker thread count at runtime:

```cpp
// Adjust thread count at runtime
sched.set_thread_count(8);  // Grow to 8 workers
sched.set_thread_count(2);  // Shrink to 2 workers
// Note: set_thread_count handles starting/stopping workers dynamically
```

For automatic scaling, use the **Autoscaler** component. It monitors queue length and automatically scales worker threads based on configurable thresholds:

```cpp
#include <elio/runtime/autoscaler.hpp>

elio::runtime::autoscaler_config config;
config.overload_threshold = 20;    // Scale up when queue > 20
config.idle_threshold = 5;       // Scale down when queue < 5
config.idle_delay = std::chrono::seconds(30);
config.min_workers = 2;
config.max_workers = 16;

elio::runtime::autoscaler<runtime::scheduler,
    elio::runtime::on_overload<elio::runtime::scale_up<elio::runtime::null>>,
    elio::runtime::on_idle<elio::runtime::scale_down<elio::runtime::null>>,
    elio::runtime::on_block<elio::runtime::log>
> autoscaler(config);

autoscaler.start(&sched);
```

This is useful for adapting to load changes automatically (see [Scheduler Statistics](#scheduler-statistics)).

### Thread Affinity

Pin coroutines to specific workers for cache locality:

```cpp
#include <elio/runtime/affinity.hpp>

coro::task<void> cache_sensitive_work() {
    // Bind to current worker for cache locality
    co_await elio::bind_to_current_worker();

    // All subsequent work stays on this worker
    process_data();
}

// Or set affinity to a specific worker
coro::task<void> pinned_work() {
    co_await elio::set_affinity(2);  // Bind to worker 2 and migrate there
    co_await elio::set_affinity(2, false);  // Bind without migrating

    // Later, allow free migration again
    co_await elio::clear_affinity();
}
```

`set_affinity` is an awaitable. When called with `migrate=true` (the default), the coroutine is immediately rescheduled on the target worker. With `migrate=false`, the affinity is recorded but migration is deferred until the next scheduling point. `clear_affinity` removes caller affinity so the task can be stolen when no stronger runtime ownership constraint exists. An active worker-local I/O pin always wins; neither API migrates a pending backend operation.

#### Spawn-time Pinning with `go_to()`

When you know the target worker at spawn time, `go_to()` is more efficient than `go()` + `set_affinity()`:

```cpp
// Preferred: affinity is set before first resume, no migration window
elio::go_to(2, cache_sensitive_work);

// Alternative: task may briefly run on another worker before migrating
elio::go([]() -> coro::task<void> {
    co_await elio::set_affinity(2);
    // ...
    co_return;
});
```

`go_to()` sets the worker affinity before scheduling the task and initially enqueues it to the target worker when that worker is available. If a later steal attempt observes the task on another queue, the scheduler requeues it to the affinity worker instead of executing it on the wrong worker. This avoids the brief scheduling window where the task could execute on the wrong worker before `set_affinity` takes effect.

Use a worker id in `[0, scheduler.num_threads())` when exact placement matters.
Out-of-range ids are not rejected, but they are fallback behavior rather than a
stable pinning contract, especially while the pool is being resized.

## I/O Backend Selection

### io_uring vs epoll

Elio auto-detects the best available backend:

```cpp
#include <elio/io/io_context.hpp>

// Auto-detect (prefers io_uring)
io::io_context ctx;

// Force specific backend
io::io_context ctx(io::io_context::backend_type::io_uring);
io::io_context ctx(io::io_context::backend_type::epoll);

// Check active backend
std::cout << "Backend: " << ctx.get_backend_name() << std::endl;
```

These directly constructed contexts are standalone. Drive and serialize them
from their owning thread; mutating one from a scheduler worker is rejected.
Scheduler coroutines should use
`io::current_io_context()`; a pending operation is pinned to that worker and
context generation until completion, so it does not incur a central-reactor
hop or migrate between backends.

**Why io_uring is preferred:**
- **Submission batching.** Multiple I/O operations can be queued in the submission ring before a single `io_uring_enter` syscall, amortizing syscall overhead across many operations.
- **Completion batching.** Completions accumulate in the completion ring and can be reaped in bulk without per-operation syscalls, unlike epoll where each I/O still requires a separate `read`/`write`/`accept` call after readiness notification.
- **Registered resources.** File descriptors and buffers can be pre-registered with the kernel, reducing per-operation kernel crossing cost by avoiding repeated `fget`/`fput` and page table walks.
- **Native async semantics.** Operations are inherently asynchronous — submit and forget until completion — which aligns naturally with coroutine suspension and resumption. There is no "readiness" vs "completion" mismatch as with epoll.

**epoll fallback:**
- Works on older kernels (pre-5.1)
- Lower memory overhead (no shared ring buffers)
- Adequate for moderate workloads where per-operation syscall cost is not the bottleneck

### io_uring Kernel Requirements

For best io_uring performance:
- Linux 5.1+: Basic io_uring
- Linux 5.6+: Full features
- Linux 5.11+: Multi-shot accept

## Memory Management

### Coroutine Frame Allocation

`coro::task` frames use the standard coroutine heap allocation path. There is no
per-vthread bump allocator or LIFO destruction requirement. Keeping frames small
still reduces allocation traffic and cache pressure, but callers should not
depend on allocator locality or on the worker that eventually frees a frame.

### Avoiding Allocations

Keep coroutine frames small to reduce allocation and cache cost:

```cpp
// Bad: Large array increases every coroutine frame allocation
coro::task<void> large_frame() {
    char buffer[8192];  // Increases every frame allocation
    co_await read_data(buffer);
}

// Good: Allocate separately
coro::task<void> small_frame() {
    auto buffer = std::make_unique<char[]>(8192);
    co_await read_data(buffer.get());
}
```

## Synchronization Primitives

### Mutex Performance

Elio's mutex uses atomic fast-path for uncontended cases:

```cpp
#include <elio/sync/primitives.hpp>

sync::mutex mtx;

// Fast path: atomic CAS (~10ns)
// Slow path: suspend and queue (~100ns + context switch)

coro::task<void> critical_section() {
    co_await mtx.lock();
    // ... critical section ...
    mtx.unlock();
}

// Use try_lock to avoid blocking
if (mtx.try_lock()) {
    // Got lock immediately
    mtx.unlock();
} else {
    // Skip or retry later
}
```

### Reader-Writer Lock

For read-heavy workloads:

```cpp
sync::shared_mutex rw_mtx;

// Multiple concurrent readers (atomic counter, no blocking)
coro::task<void> reader() {
    co_await rw_mtx.lock_shared();
    auto data = read_data();
    rw_mtx.unlock_shared();
}

// Exclusive writers
coro::task<void> writer() {
    co_await rw_mtx.lock();
    write_data();
    rw_mtx.unlock();
}
```

### Channel Selection

Choose appropriate channel type:

```cpp
// Rendezvous channel: synchronous hand-off, no buffering (default)
sync::channel<int> ch;       // or equivalently: sync::channel<int> ch(0);

// Bounded channel: back-pressure, bounded memory
sync::channel<int> bch(100);

// Unbounded channel: faster but can grow indefinitely
auto uch = sync::channel<int>::unbounded();

// Low-level bounded MPMC ring: non-blocking try_push/try_pop
// Requires #include <elio/sync/lockfree_ring.hpp>
sync::LockfreeMPMCRing<int> ring(1024);
int value = 42;
bool pushed = ring.try_push(value);
auto popped = ring.try_pop();
```

## Network Performance

### Connection Pooling

HTTP client uses connection pooling by default:

```cpp
http::client_config config;
config.max_connections_per_host = 10;
config.pool_idle_timeout = std::chrono::seconds(60);

http::client client(config);  // No io_context parameter
```

### Buffer Sizes

Tune read buffer sizes for your workload:

```cpp
http::client_config config;
config.read_buffer_size = 16384;  // 16KB (default: 8KB)

// For large payloads
config.read_buffer_size = 65536;  // 64KB
```

### TCP Settings

Configure TCP options for performance:

```cpp
// Enable TCP_NODELAY for latency-sensitive applications
net::tcp_stream stream = /* ... */;
stream.set_no_delay(true);  // Note: underscore in method name

// Buffer sizes are set via tcp_options at connection time, not on the stream
net::tcp_options opts;
opts.recv_buffer = 65536;
opts.send_buffer = 65536;
```

## Profiling and Monitoring

### Scheduler Statistics

The scheduler exposes individual metric accessors rather than a single stats struct:

```cpp
// Available scheduler metrics
size_t total = sched.total_tasks_executed();     // Total across all workers
size_t w0 = sched.worker_tasks_executed(0);      // Worker 0's count
size_t pending = sched.pending_tasks();          // Currently pending tasks
size_t threads = sched.num_threads();            // Current thread count
```

These are lightweight atomic reads suitable for periodic monitoring in production. Combine with `set_thread_count` to implement your own adaptive scaling.

### Logging Overhead

Debug logging has overhead; disable in production:

```cpp
// Set at compile time
// cmake -DELIO_ENABLE_DEBUG_METADATA=OFF ..

// Or at runtime
elio::log::logger::instance().set_level(elio::log::level::warning);
```

### Coroutine Stack Tracing

Use virtual stack for debugging without significant overhead:

```cpp
// Enable in debug builds only
#if ELIO_ENABLE_DEBUG_METADATA
    auto* frame = coro::promise_base::current_frame();
    auto stack = coro::dump_virtual_stack();
    for (const auto& entry : stack) {
        fmt::print("{}\n", entry);
    }
#endif
```

## Benchmarking Tips

### Warm-up

```cpp
// Warm up allocators and caches
for (int i = 0; i < 1000; i++) {
    elio::go(warmup_task);
}
sched.wait_for_idle();

// Now measure
auto start = std::chrono::steady_clock::now();
// ... actual benchmark ...
auto end = std::chrono::steady_clock::now();
```

### Avoid Measurement Overhead

```cpp
// Bad: timing inside hot loop
for (int i = 0; i < 1000000; i++) {
    auto start = now();  // Overhead!
    do_work();
    auto end = now();
    record(end - start);
}

// Good: time the whole batch
auto start = now();
for (int i = 0; i < 1000000; i++) {
    do_work();
}
auto end = now();
auto avg = (end - start) / 1000000;
```

### Use Release Builds

Always benchmark with optimizations:

```bash
cmake -DCMAKE_BUILD_TYPE=Release ..
cmake --build .
```

## Common Performance Issues

### Problem: High Latency Spikes

**Causes:**
- Work stealing delays
- GC pauses in other processes
- Kernel scheduling

**Solutions:**
- Pin critical tasks to workers
- Use CPU affinity for scheduler threads
- Consider real-time scheduling

### Problem: Low Throughput

**Causes:**
- Lock contention
- Inefficient I/O batching
- Small buffer sizes

**Solutions:**
- Profile lock contention
- Use io_uring for batching
- Increase buffer sizes

### Problem: High Memory Usage

**Causes:**
- Unbounded channels
- Large coroutine frames
- Connection pool growth

**Solutions:**
- Use bounded channels
- Allocate large buffers separately
- Limit connection pool size

## Running Benchmarks

Elio includes several benchmark tools:

```bash
cmake --build build

# Quick benchmark - measures spawn, context switch, yield
./build/examples/quick_benchmark

# Microbenchmarks - individual operation timing
./build/examples/microbench

# I/O benchmark - file read throughput
./build/examples/io_benchmark

# Full benchmark suite
./build/examples/benchmark

# Scalability test - multi-thread scaling
./build/examples/scalability_test
```

### Interpreting Results

Benchmark results can vary significantly (min/max differ by 2-7x) due to:
- CPU frequency scaling
- System load
- Cache state
- Memory allocation patterns

Run benchmarks multiple times and use minimum values for best-case analysis.

## Quick Reference

| Scenario | Recommendation |
|----------|----------------|
| I/O-bound | 2x core count threads |
| CPU-bound | 1x core count threads |
| Latency-critical | Pin to workers, io_uring |
| Throughput-critical | Large buffers, batching |
| Memory-constrained | Bounded channels, small pools |
| Read-heavy sync | Use shared_mutex |

## See Also

- [Core Concepts](Core-Concepts.md)
- [Debugging Guide](Debugging.md)
- [HTTP/2 Guide](HTTP2-Guide.md)