Skip to content

v3.2.0

Choose a tag to compare

@mbelhadi mbelhadi released this 20 Sep 23:26
· 391 commits to main since this release

Changed

  • The embedded event loop's ev.h defaults are libev's again (qev 5.1.0): a translation unit
    that includes <qb/ev/ev.h> without the generated ev_config.h the qb::ev target carries
    now gets libev's full watcher set rather than qb's reduced profile. qb's own builds and every
    consumer of the qb::ev target are unchanged: the configuration header is included first and
    decides. The Linux aio backend is compiled in only on request (QB_EV_USE_LINUXAIO).

  • The coroutine layer no longer queues in std::deques: qb::growable_ring buffers the stream's
    chunks, the channel's values and every parked waiter (Huly QB-215).
    MSVC's STL packs
    sizeof(T) <= 8 ? 2 : 1 elements per deque block (16, 8 and 4 for 1-, 2- and 4-byte types), so a
    std::deque<E> of 64-byte events cost a heap allocation and a free per chunk and a
    std::deque<std::coroutine_handle<>> one every two parks, where libstdc++ packs 512 bytes per
    block. Found by the ask-cost probe: its stream mode ran at 69 ns per chunk on Windows against
    31 for a bare push while Linux sat at 24 vs 24 -- a factor two on one platform over identical code
    is a data structure, not codegen -- and confirmed by qb's own benchmarks, where the sync
    primitives with 64-512 parked waiters and the channel's try-send/try-recv ran 1.3-2x behind g++
    while the deque-free cells sat at the ordinary MSVC ratio. qb::growable_ring<T>
    (<qb/system/container/growable_ring.h>) is a power-of-two ring over storage aligned for T,
    doubled when full with the elements moved, one allocation per doubling and none per element,
    deque-shaped (emplace_back, pop_front, front, operator[], iterators, erase(it),
    erase_at) and destroying what it still holds; ask_stream's chunk buffer, channel<T>'s
    value buffer and its sender / receiver / selector lists, and the waiter lists of semaphore,
    async_mutex, async_rw_lock and async_event use it (barrier and async_latch already
    kept a std::vector). Measured control 6712ef30 vs candidate, one quiet session per host,
    same-length executable paths: on Windows/MSVC 19.51 the probe's stream round trip 72.3 -> 36.3
    ns per chunk (a bare push 32), async_mutex -17 to -25 % and async_rw_lock write -16 to -22 %
    from 8 parked coroutines up, channel::try_send / try_recv -16 to -25 %, channel send/recv
    -30 % at 64 messages, the semaphore -5 to +2 %; the ask path and the push path unchanged on the
    real harness (bank-transaction 162.7 -> 163.4 and 110.8 -> 109.0 ns per unit, ping-pong
    30.2 -> 29.9, distributions overlapping). On WSL2 g++-14, where the deque was already amortised,
    the probe's stream mode is flat (24.7 -> 24.7 ns), channel::try_send / try_recv -27 to -30 %
    from 1024 messages up (past 64 elements a deque walks its block map on every push and pop, a ring
    bumps a pointer) and every sync primitive sits within the +-3 % band an untouched std::vector
    cell draws on the same binary.
    growable-ring pins the ring: order across wrap-around and growth, one live element per held
    item, 64-byte slots, the ordered erase.

  • qb::ask no longer allocates a coroutine frame: it returns an awaitable that lives in the
    awaiting frame (Huly QB-214, the "ask frame" loss point of QB-212).
    co_await qb::ask(...)
    used to enter a task<E> whose whole body was "build the awaiter, stamp, push, co_await it": a
    pooled frame allocated and freed per ask, the initial suspend and the transfer into it, a
    co_return moving the 64-byte reply into the promise's variant, the final suspend and the
    transfer back, a second move out of the variant. Both forms of qb::ask now return the
    exchange itself as an awaitable -- qb::ask_operation<E>, qb::ask_emplace_operation<E, Args...>
    -- a prvalue built into the caller's frame that holds the context, the target, the request (or
    the constructor arguments) and the timeout, engages the ask_awaiter in place at
    await_suspend() and hands the reply over with one move. Nothing is sent until it is awaited
    (the laziness of task is kept) and a cancelled scope sends nothing, throwing cancelled_error
    at once. Source-compatible: the operation converts implicitly to task<E> (rvalues), so
    task<E> t = qb::ask(...) and calls.emplace_back(qb::ask(...)) behind
    when_all(std::vector<task<E>>) keep compiling, and the variadic when_all / when_any / race
    and coro_with_timeout take it directly (a detail::to_task normalisation); only code that
    names the return type as exactly task<E> in a concept or a decltype sees the change.
    Measured control 7296ac8d vs candidate, one quiet session per host, same-length executable
    paths: the ask-cost probe's round trip 45.2 -> 35.6 ns on WSL2 g++-14 (the ask mechanics above
    a bare push/reply 21.5 -> 12.4 ns, -43 %) and 64.0 -> 48.5 ns on Windows/MSVC 19.51
    (33 -> 16 ns above push, -51 %); qb-vs-others savina/bank-transaction 136 -> 97 ns
    per unit 1c-spin (-29 %), 141 -> 99 1c-park (-30 %), 82 -> 73 2c-spin (-12 %), 84 -> 73 2c-park
    (-12 %) on WSL2 and 229 -> 161 (-30 %), 232 -> 163 (-30 %), 142 -> 106 (-25 %), 143 -> 109
    (-24 %) on Windows, 12 interleaved launches per cell, every distribution disjoint; dev/bench
    ask-roundtrip same-core 5.5 -> 5.1 ms, cross-core 14.5 -> 14.3; the ask-free anchors and every
    ask-free cell that moved in a grid pass re-censused flat on both hosts. The bank-transaction
    harness itself detected the emplace form through a concept keyed on the exact task<Deposit>
    return type and, with the candidate, fell back silently to the by-value form wrapped in a task
    (+17 % on every config): it now detects by callability and returns what qb::ask returns -- the
    very concept shape the documentation now warns against. request-operation pins every shape of
    the new type against the running engine.

  • The actor object lives in a per-thread size-class arena (Huly QB-212, point 1). After the
    3.2 registry work the actor object's own new / delete was the last heap traffic of an actor
    lifetime -- exactly one malloc per actor, counted -- and on qb-vs-others' savina/fib (57 312
    lifetimes per window) that pair was 27.8 % of the core on WSL2 g++-14 of a 124 ns lifetime, 179 ns
    on Windows/MSVC whose heap is slower still. qb::Actor now declares class-level operator new /
    operator delete (the usual sized, the aligned and the placement forms) over
    qb::allocator::thread_arena, a new thread-private allocator: 16-byte size classes up to 1 KiB
    with LIFO reuse per class, bump allocation from a 64 KiB first chunk then 2 MB slab_cache slabs
    (huge-page-backed and prefaulted on Linux, kept mapped process-wide when a thread gives them
    back), constant-initialised thread-local state so the hot path pays no TLS init guard, and a
    teardown at thread exit that returns the slabs only when nothing is live -- a thread that exits
    with a live block orphans its chunks on a reachable, counted list rather than recycle them. An
    actor is created and destroyed on one thread by construction, so the arena needs no lock, no
    atomics and no cross-thread free path. Larger or over-aligned actor types fall through to the
    global allocator; a derived class that declares its own operators keeps them (the pool hook);
    qb::allocate_actor<T> stays the construction hook. Measured against develop in one quiet
    session per host, 12 interleaved launches, fib 1c-spin: 185.3 -> 127.2 ns (-31 %) on Windows/MSVC 19.51 and
    129.2 -> 99.1 ns (-23 %) on WSL2 g++-14, the four fib cells all in that band (-22 to -35 %) and the other
    seven shapes inside their spread; dev/bench's actor cells -0.2 to -4 % against the same control
    (the grids, censuses and the bench record are under qb-vs-others' results/<host>/ qb-branch-perf-actor-arena/). Pinned by the unit thread-arena (rounding, LIFO reuse, the
    large and over-aligned fall-throughs, growth from the first chunk to slabs, the exit teardown
    with and without a live block) and the engine actor-arena (a burst of 8000 addRefActor
    lifetimes is a burst of arena blocks whose slabs return when the worker exits; an alignas(64)
    actor stays aligned and unpooled; a 2 KiB actor falls through; a class with its own operators
    keeps them).

  • A parked core meets a timer under a millisecond under a millisecond, on Linux (Huly
    QB-196).
    The park of a core that owns io watchers is ev_run(EVRUN_ONCE) capped at
    latency, and its wait went through epoll_wait, which takes whole milliseconds and which
    libev rounds UP -- so a latency of 100 µs parked a full millisecond, and a 200 µs
    qb::io::async::callback on an otherwise idle core fired at the millisecond. Measured with
    qb-vs-others' parked-timer-wake probe on WSL2 g++-14 (Linux 6.6): a 100 µs timer on a parked
    core 1010 µs late at p50, a 1 ms one 110 µs late, a 5 ms one reached through 1 ms parks 357 µs
    late, against 0.1 µs on a spinning core. qev's epoll backend now asks the kernel in nanoseconds
    where it can -- epoll_pwait2 (Linux 5.11, glibc 2.35), honoured to the thread's timer slack of
    50 µs, probed once per loop with an older kernel keeping epoll_wait -- and the same probe,
    re-run against develop in one session, says what the timer costs now (the figures are in
    qb-vs-others' docs/TUNING.md §19); the non-blocking pass keeps epoll_wait and its measured
    floor. The
    fact the issue was filed on -- a Windows park capped at the 15.6 ms system tick -- was NOT what
    the measurement found: with the box's timer resolution read at 15.625 ms, a 1 ms wepoll wait
    returned in 1.0–1.5 ms at p50 and 2.4 ms at p99, timeBeginPeriod(1) changing nothing (the
    kernel's waits are tickless; its coalescing is the +0.5–1.5 ms), and a parked core woken by a
    socket paid 65–190 µs, deeper the longer its CPU had been idle -- the CPU's idle-state exit, the
    same on qb at axis N and on develop in an interleaved A/B, so no regression since §10. Windows
    and io_uring keep the millisecond wait; CoreInitializer::setLatency and the tuning guide now
    say so. Pinned by core-park-wake
    (ASubMillisecondTimerFiresUnderAMillisecondOnACoreParkedInItsLoop, a SKIP where the libc,
    the kernel or the backend cannot express it) and by qev's test-loops (test_epoll_ns_wait,
    whose negative control -- the nanosecond path switched off -- fails it at 1058 µs).

  • ActorHandle::ready_async is event-driven (Huly QB-62). The waiting coroutine is resumed by
    the core pass that ends the child's activation -- after the child's stashed events have been
    replayed, so the child is caught up when the parent runs -- instead of polling ready()
    through a 1 ms cancellable sleep, a timer armed and disarmed a thousand times a second. A child
    whose async init FAILS (co_return false, a throw, the activation deadline, a kill while
    Activating) is reported the pass it fails, where the poll only ever reported it through the
    waiter's own timeout. The awaiter (detail::activation_awaiter) has the shape of
    ask_awaiter: an intrusive waiter node (detail::activation_waiter, linked by
    activation_wait into the core's Activating entry of the target and fired -- unlinked first --
    by whichever path ends the activation), a request_deadline in the core's own clock for the
    timeout (no ev_timer), the actor scope's embedded cancel hook for a kill of the waiter; an
    already-active child or an invalid handle answers without suspending. An Activating entry never
    disappears with a linked waiter (its destructor detaches, its move transfers the list).
    qb-core-test-system-init-ready-async: nine cases, including the resume within two passes of
    the init completing. Measured level on both hosts (qb-vs-others
    results/*/qb-branch-batch-quick-wins-3/), after one lesson: the batch's two cold bodies placed
    BEFORE the hot ones in VirtualCore.cpp and Actor.cpp read +2 % on savina/ping-pong 1c on g++
    (quartiles separated) for code the ping-pong never calls; at the end of their files, +0.1 %.

  • llm/qb.llm.md carries, with the rules an agent reads before writing code, the invariant that
    getService<T>() is the one lookup that is not phase-gated -- a service still in its async
    onInit() and one killed but not yet reaped are handed out, by design -- so push it an event
    rather than read its state (Huly QB-66).

  • __workflow__ reaches listener::current once, not three times a pass (Huly QB-199). It
    is an inline thread_local with a non-trivial constructor, so g++ routes every access through its
    TLS wrapper (the init guard, __tls_init): 2.2 % of savina/ping-pong 1c for has_work() /
    run() / nb_invoked_event() on every pass, plus the idle path's own. One reference taken at
    the top of the loop — the object lives for the thread, and the loop is the thread. Measured
    alone against develop on WSL2 g++-14 (twelve interleaved rounds, one quiet session, 60 s of quiet after the last build):
    ping-pong 1c 23.1 → 22.8 ns (−1.4 %, quartiles separated), push 24.03 → 24.04 and
    pass-cost k = 1 12.28 → 12.31 (level); every other cell level. MSVC, which initialises TLS at
    thread start, level everywhere (ping-pong 1c 31.4 / 31.5). Found by the QB-198 investigation — the dispatch loop
    prefetching the actor of an event ahead on a core serving a population, which hid the miss it
    targeted (savina/bank-transaction 1c −3.6 %) and cost everything cheaper (+6 % on a 5 ns event
    with its gate closed, +21–25 % on a hot population of 512–1 024 actors): the out-of-order engine
    already overlaps the misses of a dozen short, predictable handlers, and no cheap gate tells a
    long unpredictable one from them. Measured on the new dispatch-population probe (qb-vs-others)
    and not shipped; the batch test it produced (messaging-dispatch-batch: one batch mixing live
    destinations, an emptied slot, a broadcast, a never-assigned sid and a mid-batch kill, delivered
    in one pass) stays.

  • Per-actor lifecycle log lines are VERBOSE, not INFO. New <actor>, Delete <actor>,
    Actor(<id>) subscribed to <Event> / unsubscribed to, activating / activated and the
    destroyed-with-pending-coroutines line moved from QB_LOG_INFO to QB_LOG_VERB; the per-core
    Init Success / Stopped normally lines stay at INFO. A release build (QB_WITH_LOGGING=ON,
    default; level INFO under NDEBUG) wrote nine lines to ./qb.1.log for every actor created —
    perf put nanolog's encoder, its writer thread and the ostream behind it at the top of the
    savina/fib profile once the table growth above was fixed: 178 ms → 28 ms per repetition at one
    core (CAF 66 ms, raw-thread floor 1.7 ms on the same host), and a real application creating actors
    at that rate paid the same — shipped 3.1.0 measured in the same quiet session at its defaults
    is 159 ms per repetition on WSL2/g++-14 and 459 ms on Windows/MSVC (515 819 lines, 60.9 MB
    of qb.1.log per repetition, counted) where this line measures 7.6 and 10.5 ms.
    qb::io::log::setLevel(qb::io::log::Level::VERBOSE) shows them again.

  • The per-core actor registry is a dense vector, and the kill queue is a vector too.
    VirtualCore::ActorMap was qb::unordered_map<ActorId, std::unique_ptr<Actor>> and
    _actor_to_remove a qb::unordered_set<ActorId>; every actor lifetime paid two hashed inserts,
    two hashed erases, a node malloc/free, and every findActor / isActorAlive / pump lookup a
    hashed probe. The registry is now std::vector<std::unique_ptr<Actor>> indexed by the id's
    16-bit service slot (__actor_slot__(id) is an index plus an id compare, so a recycled slot
    never answers for its previous tenant), grown by doubling and capped at ServiceIdPool::kBits + 1,
    with _actor_count as the live count; killActor is one push_back (an actor's kill() now
    returns early once _alive is false, so an id is enqueued at most once and the queue needs no
    set semantics), and _dying_with_frame keeps the set it needs. The router's subscribe also
    drops a dynamic_cast per subscription: _registered_events is keyed by the event's type id, so
    the entry it finds IS an EventResolver<_Event> and a static_cast says the same thing without
    the RTTI walk. Measured on the same quiet Linux session (WSL2 Debian, g++ 14.2 -O3, two pinned
    cores), savina/fib n=23 — 57 313 actors created and destroyed inside one window — went from
    15.4 ms to 8.34 ms per repetition (p50 of 9, three interleaved A/B runs each within 1 %; CAF
    1.1.0 37.4 ms, raw-thread floor 3.53 ms on that session), and a static-topology shape
    (savina/chameneos) is unchanged at 10.8 ms. Suite green on Linux release, ASan and TSan (191/191).
    VirtualCore::ActorMap is no longer an example of the node-map pointer-stability contract
    (readme containers.md), because it never needed it: the actor lives behind its unique_ptr.

  • An actor's coroutine counter is allocated on its first spawn, not in its constructor.
    Actor::active_coroutines_ (the census the RAII guard decrements after the actor is gone — a
    shared_ptr<atomic<size_t>> when this landed, detail::coro_census_ref since the entry below)
    was a make_shared in every Actor::Actor() and a release in every
    destructor, whether or not the actor ever spawned a coroutine. Measured on savina/fib (one
    short-lived actor per node, none of them spawning) that pair was ~30 % of an actor's lifetime
    cost. It is now created by __ensure_coro_counter__() on the first spawn() /
    spawn_detached(), like _coro_scope already was, so the hot path is one predicted-not-taken
    null check before the fetch_add. savina/fib n=23 on Linux/g++-14 goes 8.34 ms to 7.3 ms on
    two cores and 14.9 ms to 11.3 ms on one; has_active_coroutines() and
    active_coroutine_count() answer false / 0 for a counter that was never allocated. Suite
    green on Linux release, ASan and TSan (191/191).

  • The five default events dispatch through the actor registry, not through per-type handler
    tables.
    KillEvent, SignalEvent, UnregisterCallbackEvent, PingEvent and RequireEvent
    (qb::default_events_t) cost every actor five key_table inserts at construction and, at
    removal, a walk of EVERY resolver on the core to erase them — five tables of 65 536 × 32-byte
    slots per core whose key set was exactly "the live actors", which VirtualCore::ActorMap already
    is. perf on savina/fib (57 312 actor lifetimes per repetition, one core) put registerEvent
    × 7 at ~29 % and unregisterEvents at ~12 % of a 200 ns lifetime. registerEvent<E> for a
    default E now stores a dispatch pointer in the actor itself (Actor::_default_on[k], a
    non-virtual trampoline that recasts to the registering type and checks is_alive()), and
    VirtualCore installs one DefaultEventResolver<E> per default event into its router
    (router::memh::install, new), which answers a unicast from __actor_slot__(dest) — the bounds
    check and id compare every lookup already pays — and a broadcast from a snapshot of the registry,
    so a handler that spawns actors cannot invalidate the walk. memh::unsubscribe(id) walks only the
    resolvers that own handlers (_owning), which is what makes unregisterEvents targeted; an
    installed resolver declares owns_handlers = false and subscribe<E>() on its type asserts
    rather than recasting it. Every contract the tables gave is pinned by
    qb-core-test-system-default-event-registry: a pushed default reaches the base handler of an
    actor that never mentioned it; a derived class that re-registers one REPLACES the base handler
    (the supervisor pattern); unregisterEvent<E> and qb::no_default_events both drop the event
    silently; a broadcast reaches every live actor and not one spawned by a handler during it; a
    ServiceActor is reached through the same slot. Measured in one quiet session per host, 9
    repetitions + 2 warmup, develop e814df06 → this line, p50 per repetition: savina/fib
    6.96 → 5.21 ms at two cores and 10.98 → 8.48 ms at one on WSL2 Debian/g++-14 (−25 % /
    −22 %; a second pass 5.31 / 8.62), 10.18 → 7.03 ms and 15.73 → 11.03 ms on
    Windows/MSVC 19.51 (−31 % / −30 %; second pass 7.10 / 11.48); savina/chameneos, which creates
    its 101 actors outside the window, is unchanged within its spread on both hosts (10.7 / 6.5 ms
    WSL2, 12.2–13.4 / 7.1 ms Windows). qb::no_default_events is therefore a semantic choice now,
    not a saving. New public names: qb::default_events_t, qb::default_event_index<E> (−1 for
    any other type; cvref stripped; a type DERIVED from a default event is its own event type),
    qb::is_default_event<E>. Suite green on Linux release (371/371), ASan+UBSan and TSan (192/192),
    Windows/MSVC release (188/188).

  • Google Benchmark floor is 1.9.5 (QB_BUILD_BENCHMARKS=ON only): the pinned fetch moves
    v1.9.2 → v1.9.5, and a system package is accepted only from 1.9.5 — the release that made
    benchmark::Benchmark public. The benchmark sources name that public type; until now they
    named benchmark::internal::Benchmark, which 1.9.5 deprecates, and the benchmark targets
    carried a -Wno-error=deprecated-declarations / /wd4996 exemption so that a 1.9.5 system
    package and a 1.9.2 fetch could both compile (19 warnings per build on a 1.9.5 host). The
    exemption is gone: -Werror applies to benchmark targets exactly as it does to tests. A host
    whose system package is older falls back to the fetch, as before.

  • The reference Actor::push / Pipe::push / allocated_push returns is valid until the
    handler or callback that obtained it returns
    — across any number of further pushes to the
    same core — where it used to die at the very next event queued to that destination core. The
    event pipes were one contiguous allocator::pipe per destination core: growth memcpyd
    everything already queued and compaction memmoved it in place, which is what invalidated
    the reference, and what cost a one-core burst of 1 M events 64 MB of copying plus ~26 600
    minor page faults per run re-touching each doubling (the cliff between 100 k and 300 k
    messages in qb-vs-others). With segmented_pipe nothing queued ever moves, so the reference
    holds until the engine consumes the event between handlers. Code written to the old rule is
    still correct; what remains forbidden is holding the reference across a coroutine's
    co_await or in a member. Pinned by SegmentedPipeContract.* (unit) and
    PushReferenceStability.* (through the engine, same-core, cross-core and under growth).

  • Event pipes allocate nothing until their first push, and the per-core pool holds the high
    water in 2 MB slabs
    (segment_pool::shrink() returns whole idle slabs to the cache,
    slab_cache::trim() returns the cache to the OS). Every core used to allocate 256 KB eagerly
    for each of its N + 1 pipes — 68 MiB at rest on 16 cores — and each pipe doubled on its own,
    so the peak was 2–3× the largest burst and never came back. A consumed segment is the next
    one a push grows into, still warm in cache. Slabs, not malloc, because the A/B measured
    where the residual went: with segments from the heap a fresh engine's 1 M-event one-core
    burst still took 15 640 minor page faults — every 4 KB of every segment — which on WSL2 was
    ~15 ms of a 20 ms run; a slab is faulted once per 2 MB, in the kernel, and never again for
    the life of the process.

  • The receive loop reads an event's width once, before its handler runs, and the pool
    staggers its segments.
    Segments carved on a fixed stride out of aligned slabs all start at
    the same offset within a 4 KB page, so item k of the pipe being drained and item k of the
    pipe being filled share their low twelve address bits — and a reply IS a byte copy of the
    received event into the outbound pipe at the same index. The loop then re-read
    bucket_size from the event to advance, a load trailing a store to an address the core
    cannot tell apart from it until the store commits (Intel's 4K aliasing); measured on
    savina/big at one core, that one stall per event doubled the run against the malloc-laid
    pipes it replaced (52 → 107 ms per rep, that reload alone 12% of the samples). The width is
    now read once, ahead of the handler, and segment_pool gives the i-th segment it carves a
    stagger of (i * 27) % 64 cache lines so consecutive segments — the swap pair of a core —
    never share a page offset, which also covers a handler that touches the event after
    reply(). The stride is 27 and not 1 because a one-line stagger moves the store onto the
    NEXT event's load for one-line events, measured at +8% on the same benchmark. The receive
    loop is driven by front() alone — a range that is empty exactly when the pipe is — rather
    than testing empty() before and after each range. Same-core round trips end up faster than
    before the segmented pipe (Linux/g++-14, median of 5 interleaved launches: big 52.7 → 52.2 ms,
    ping-pong 67.7 → 64.2 ms; ten-launch census, ns per message: thread-ring 37.7 → 35.6 at one
    core and 110.6 → 105.5 across two, big 21.9 → 21.4). Windows/MSVC keeps its one-core
    ping-pong level within 1% (77.2 → 78.0 ns per round trip, median of 24 interleaved launches,
    distributions overlapping) while every burst cell there is 2–4× faster.

  • The latency > 0 park handshake is race-free. Mailbox::wait() was
    cv.wait_for(lk, latency) with no predicate and notify() signalled without the mutex, so an
    enqueue landing between the consumer's empty drain and its registration on the condition
    variable was never seen and the core slept the whole latency. Measured on a two-core ping-pong
    at 300 000 messages: 246–293 lost wakeups on Linux (1 ms each) and 48–2849 on Windows, where
    MSVC's wait_for rounds up to the 15.6 ms scheduler tick. The handshake is a Dekker pair over
    a cache-line-aligned _parked flag with seq_cst fences on both sides, the producer taking
    the mutex before it notifies and the consumer waiting on has_data(). At latency 0 wait()
    returns at once and notify() is only its fence.

  • Mailbox::notify() issues its seq_cst fence in spin mode too. On x86 that fence is an
    mfence, which drains the producer's store buffer, so the event just enqueued becomes visible
    to a spinning consumer now rather than whenever the buffer flushes. Measured on a two-core
    ping-pong (interleaved A/B on a quiet host, p50 of 7 runs): Windows/MSVC 296–309 ns per
    round-trip without it and 259–263 with it, where the spinning cell had been slower than the
    parking one (256–272); Linux/g++-14 204–219 → 205–208, with the worst run falling from 295 to
    215 ns. Bulk single-producer throughput (BM_Multi_Producer_Consumer, 1M events) is unchanged
    at 18.6 M msg/s. arm64 is unmeasured.

  • A core no longer parks over its own pipe. An actor pushing to itself from a callback moved
    no counted event, so a parked core delivered that push after latency; the idle test now also
    requires the self-core pipe to be empty.

  • VirtualCore polls libev only when the loop has work, instead of on every pass once the
    core had ever touched a coroutine or registered a handler.

  • An idle latency > 0 core that owns io watchers parks INSIDE its io loop, not on the
    mailbox's condition variable.
    A parked core used to be deaf to its own loop: a readable
    socket, an accepted connection or a due qb::io::async::callback was answered only when the
    latency timeout expired — measured p50 923 µs at latency = 1 ms and 9.96 ms at 10 ms on
    Linux, 0.9–1.2 ms at 100 µs / 2.2–2.8 ms at 1 ms / 15.6 ms at 10 ms on Windows, where MSVC's
    wait_for lands on the 15.6 ms scheduler tick, for a socket that was readable within
    microseconds (qb-vs-others audit, axis N, tools/probes/parked-io-wake.cpp). With io watchers
    active the core now parks in listener::run_once_for(latency): the socket or timer ends the
    park through the backend poll, and an event from another core ends it through the mailbox's
    notify(), which calls listener::wake() when the parked side has published its loop
    (Mailbox::attach_loop() at core start, withdrawn by detach_loop() on exit; the producer
    re-reads the park state under the mailbox mutex, so the pointer it dereferences is never a
    dead core's — the Park::Loop announce is a release store and that re-read an acquire, the
    pair ThreadSanitizer asked for between eventfd() in the wake pipe's creation and its first
    write). Same cells, after: 31 µs p50 on Linux, 24–27 µs on Windows — the busy-poll floor plus
    one wake; a core that really slept 2 ms pays the OS wake on top (46–51 µs Linux, 62–135 µs
    Windows), the floor every framework pays once it sleeps (qb-vs-others docs/TUNING.md
    §8.2). A core with no io watchers keeps the
    condition-variable park, and Main::stop() is seen at the next pass on either path, so a
    loop park ends within latency of it exactly as the cv park did. Pinned by
    core-park-policy (past the spin floor the core parks inside its loop and an io timer fires
    at its own delay; a loop park is capped by the latency and sees stop) and core-park-wake (a
    readable socket ends a loop park, and a cross-core push ends both a loop park and a cv park,
    each in far less than latency, the process CPU time proving the core was parked rather
    than polling).

  • The core's pass costs less, and every event it carries costs half. Measured with a
    new qb-vs-others probe (tools/probes/pass-cost.cpp: one actor, k self-event chains, no
    tick, no clock inside the window) on i9-12900K / WSL2 g++-14, a pass carrying one event was
    14.5 ns and each further event in the same pass 8.9 ns; they are 12.9 ns and
    4.2 ns now. Five things, each kept only after the probe moved:

    • VirtualCore::__receive__ walks the self pipe in place up to a fence (segmented_pipe::mark(),
      front(fence &), pop_front(fence &)) instead of swapping a second pipe in: a handler's
      same-core push lands behind the fence and is the next pass's event, exactly as before, but
      the six loads and six stores of the swap are gone and the handler's first push no longer
      waits on the _wcur the swap had just written — the longest dependency chain of a
      one-event pass. _mono_pipe / _mono_pipe_swap are one _self_pipe.
    • __getPipe__ is one indexed load off the core (_pipe_of_core[CoreId], 2 KB per core,
      resolved once at construction) instead of the engine's core set, its dense-index table,
      the pipe vector's data pointer, then the pipe — four dependent loads on every push from the
      user's TU; segmented_pipe::allocate_back_slow is out of line (QB_NOINLINE QB_COLD), so
      the fast path of a push is a leaf that saves no callee-saved register.
    • __flush_all__ is an inline scan of the peer pipes that enters the drain
      (__flush_pipes__, the former body) only when one holds events — nothing at all on a
      one-core engine, where the drain's prologue and pipe walk were 6 % of a pass.
    • listener::has_work(), run() and run_once_for() read the loop's active and pending
      counts inline through qev's new ev_active_count_addr() / ev_pending_count_addr()
      (read-only aliases into the loop, taken once) instead of two calls into libqev plus a loop
      over the priorities on every pass; router::semh::route keeps the broadcast walk out of
      line (route_broadcast), so the unicast path — every event of a ping-pong — no longer
      carries its snapshot vector's register pressure; key_table::find bounds-checks against a
      cached slot count rather than a pointer difference divided by a 24-byte slot.
      Measured on qb-vs-others in one quiet session per host against develop c42abddf (9 + 2,
      CPUs 0,2, p50): WSL2 / g++-14 ping-pong 1c-spin 28.7 → 22.6 ns per round trip (−21 %),
      thread-ring 1c-spin 21.1 → 17.2 per hop, fork-join 2c-park 11.1 → 8.4 ns per message
      (−25 %), big 1c-park 22.5 → 18.5 (−18 %), counting 1c-spin 8.9 → 8.0, ping-pong 2c-park
      212.8 → 199.6 (−6 %) and 2c-spin 209 → 200 / 211; dev/bench BM_PINGPONG 64 actors / 1
      core 26.5 → 21.3 ns (−20 %), Mono_PingPong 63.7 → 58.5, the one-core pipeline 32.2 →
      29.9. One cell moves the other way and is recorded as such: thread-ring 2c-spin / 2c-park
      +1 to +5 % across three interleaved censuses, bisected to the flush scan alone — a
      waiting core's idle pass is ~2 ns shorter, and the 100-actor ring's cross-core hop, unlike
      the two-actor ping-pong's, is slower for it (the same sensitivity QB-180 measured: the idle
      spin pass paces the cross-core exchange). Kept, because undoing it costs 0.5 ns on every
      one-event pass and 3 ns on a four-event one; the idle-pass pacing is an axis of its own.
      Windows / MSVC 19.51, same protocol: the probe's one-event pass 20.4 → 15.6 ns (−24 %),
      ping-pong 1c-spin 41.4 → 30.6 (−26 %), thread-ring 1c-spin 22.7 → 18.5 (−19 %), big 1c
      −8 / −12 %, ping-pong 2c-park 260 → 246 / 230, thread-ring 2c +2 to +5 % (the same residual);
      dev/bench Mono_PingPong 76.6 → 66.4, the one-core pipeline 42.7 → 34.6 (−19 %), the 8 × 8
      pipeline 263.5 → 228.1 (−13 %), BM_PINGPONG 64 actors 29.4 → 26.5. Suites: WSL2 release /
      ASan+UBSan / TSan 192/192, the Windows full gate.
  • A pass with nothing for the loop to do does not run the loop (qev 5.1.0 ev_now_set,
    ev_timer_next, ev_timer_count_addr, ev_wake_pending_addr; Huly QB-190).
    Once the poll was
    on a cadence, what a non-blocking pass still paid was ev_run itself — its clock read, the timer
    heap, the pending walk, ~22 ns on g++-14 and ~31 on MSVC — on every pass of every core that held
    one far timer (a sleep, a retry, a keep-alive) or one quiet socket, to find nothing. The
    listener now asks the loop inline before calling it: a pending event, a wake an ev_async_send
    from another thread left while no pass was blocking, a poll the cadence is due to make, or a
    timer within reach — and a pass with none of the four does not enter the loop. "Within reach"
    is judged on the CPU's counter, like the cadence: the listener anchors each reading of the
    loop's clock against tsc_ticks() and estimates now as the anchor plus the counter's advance
    (a 0.1 % rate margin, 2 ms of slack, a fresh anchor every second at most), so a deadline beyond
    the estimate costs no clock read, and one within it costs the precise read — which is then
    handed to the loop (ev_now_set) so the pass that fires the timer reads no clock of its own. A
    timer is never judged against the estimate, only against a real reading; precision is the
    clock's, as before. Measured with the io-pass probe's new timer shape and its pass shape
    (one pinned core, five alternations, quiet hosts): a busy pass with a far timer 36.6 → 26.3 ns
    on WSL2 g++-14 (−28 %), a pass with a quiet socket between two polls 48.0 → 27.5 (−43 %),
    against 12.2 for a plain pass; the byte's wake latency, push, the timed ask and the no-watcher
    pass level. MSVC: MSVC. What a raw ev_timer user must know follows from it: a loop that
    runs only when something is due has a stale clock the rest of the time, so
    event::timer::start() / again() refresh the loop's clock before every arm (~17 ns, the
    ev_now_update sleep, callback and with_timeout already made; the C-level
    ev_timer_start keeps libev's contract and leaves the refresh to the caller, as the QUIC
    endpoint's arm now gets through the wrapper). Tests: tests/io/unit/async/listener-timer-gate.cpp
    (a far timer costs no loop pass — a thousand passes, zero iterations; a timer within reach fires
    on the first pass after its deadline; the loop's clocks stay consistent across supplied passes;
    a pending event, a wake from another thread and a byte at the cadence's next poll each run the
    loop; an event::timer armed after 30 ms of skipped passes fires 20 ms later where the raw C arm
    fires it early — each mechanism disabled in turn is rejected), the raw-fd fixture shared with the
    cadence test (tests/io/shared/raw_fd_fixture.h), qev's test_now_set / test_timer_next /
    test_wake_pending (floor 35 → 48). Found on the way, in qev (Huly QB-195): on Windows the
    QueryPerformanceCounter clock of QB-193 had never been compiled in — libev's "misconfiguration"
    block forces EV_USE_MONOTONIC to 0 wherever CLOCK_MONOTONIC is undefined, and MSVC defines it
    nowhere — so the loop's "monotonic" time was the precise system time, stepped by every wall-clock
    adjustment; the block exempts Windows now, and a timer pass reads 31.5 → 28.6 ns there.

  • A core polls a quiet socket on a cadence, not on every pass (qev 5.1.0 EVRUN_NOPOLL, Huly
    QB-191).
    A core that owns one socket ran the backend poll — epoll_wait(0), wepoll's IOCP wait,
    kevent — on EVERY pass, and on a quiet socket every one of those calls returned nothing: with
    the pass at its floor an io pass cost 124 ns against 12 for a plain pass on WSL2 g++-14
    (the poll ~80, the rest of ev_run ~22), 275 against 16 on MSVC 19.51 (the IOCP wait
    ~245). The listener polls on a cadence now: on every pass while the loop is HOT (the previous
    pass's poll reported a ready fd — read inline off qev's new ev_io_fed_addr(), so a burst
    stays at poll latency) and otherwise once per CoreInitializer::setIoPollInterval (1 µs by
    default; zero = every pass, the 3.1 contract), measured on the CPU's own counter
    (qb::tsc_ticks, ~5 ns, calibrated once against mono_now()). The passes in between run
    ev_run(EVRUN_NOWAIT | EVRUN_NOPOLL): timers, periodics and pending events exactly as before,
    no backend call. A blocking pass — run_once_for, a park — always polls (there the poll IS
    the wake), and the listener's own default stays "every pass", so a program that drives
    run(EVRUN_NOWAIT) itself keeps one call, one poll; a VirtualCore opts its listener in at
    thread start. Measured with the new io-pass probe (qb-vs-others, one pinned core, a loopback
    TCP pair, five alternations): a pass with a quiet socket 124 → 48.5 ns on g++ (−61 %),
    275 → 83 on MSVC (−70 %); the byte's wake latency on that socket p50 3.36 → 3.93 µs on g++
    (half the interval, as designed; p99 level) and 17.7 → 18.9 on MSVC (inside the spread; the
    same binary with the interval set to 0 reads the control on both hosts); push, the timed ask
    and the no-watcher pass level. Tests: tests/io/unit/async/listener-io-cadence.cpp (the knob;
    every pass by default; inside the interval the poll is skipped and due it runs; a delivery keeps
    the loop hot for the next pass; timers and fed events ignore the cadence; a blocking pass
    always polls — each mechanism disabled in turn is rejected; the fixture is a pipe on POSIX and
    a loopback TCP pair through wepoll on Windows, where the case waits for the kernel to report
    the byte readable before the pass it counts, since AFD does not make a loopback send readable
    synchronously), core-io-poll-interval.cpp (the initializer's value reaches the core's
    listener, zero means every pass, a millisecond is 500–2000× a microsecond in the counter's
    ticks), qev's test_nopoll (unconditional floor 32 → 35) and its wepoll twin — writing which
    found that qev's wepoll suite had never measured wepoll at all (Huly QB-194: a raw SOCKET
    as fd, every case green on fd_kill's EV_ERROR). Not done here, recorded: the EVRUN_NOPOLL pass still costs the loop's clock read
    and bookkeeping (~22 ns of the 48); skipping ev_run altogether when no timer is due needs the
    earliest deadline and the loop's clock readable inline, the next cut with QB-190.

  • A request timeout is a deadline in the core's own clock, not a libev timer (Huly QB-189).
    Every timed ask, ask_stream::next(), ping and require armed an ev_timer: an
    ev_now_update and a heap insert to arm, a heap remove to disarm, and — the part that cost —
    a referenced active watcher for the whole wait, so listener::has_work() stayed true and
    EVERY pass of the core ran ev_run for the one request in flight (22 ns a pass on g++ once
    qev's pass reached its floor, 31 on MSVC with a precise clock). The deadline is now an
    intrusive node in the awaiter, linked into the core's deadline_list (a VirtualCore
    member, sorted by deadline, a tail insert for the equal-timeout common case, O(1) unlink on
    reply, cancel or teardown), and the pass checks it: a member load when nothing is armed; when
    something is, a COARSE clock read first (CLOCK_MONOTONIC_COARSE, GetTickCount64,
    CLOCK_MONOTONIC_RAW_APPROX: the scheduler tick, ~3–5 ns) and the precise mono_now() only
    within one tick of the earliest deadline, so a timeout keeps the sub-microsecond precision it
    had and a busy core pays ~5 ns a pass for it instead of a full loop run. An idle pass hands the
    list the reading it already makes for the park policy, and a park is bounded by the earliest
    deadline (Mailbox::wait(cap) / wait(listener &, cap): the loop holds no timer for the
    request any more, so neither park may outlast it). The loop is out of the request path
    entirely — no watcher, no ev_run, has_work() false on a core whose only "watcher" is a
    pending request. Measured (ask-cost, one core, five alternations, i9-12900K): a
    co_await qb::ask<E>() with a 500 ms timeout 111.6 → 67.8 ns on WSL2 g++-14 (−39 %),
    157.1 → 90.1 on MSVC 19.51 (−43 %, and back under the 124 it read before the Windows
    clock became precise), a one-chunk timed ask_stream 173.6 → 131.3 / 354.8 → 301.9; the
    untimed ask 45.5 → 45.9 (ten launches, inside its spread), push and the one-core pass level,
    savina/bank-transaction 2c 83.0 → 80.9 / 1c 142.9 → 144.8, ping-pong 2c 159.5 → 160.3 (ten
    launches each: the per-pass gate is invisible). The list was first a thread_local read
    out-of-line, then inline: the call cost ~0.5 ns on every pass on g++ (push 23.3 → 24.4) and
    the inline TLS read ~1 ns on MSVC (push 31.6 → 33.6, four dependent loads), which is why it
    is a core member. Tests (tests/core/system/coroutine/ask-deadlines.cpp, eight): a timeout
    fires with no libev watcher active and leaves the list empty; a thousand replied asks arm
    and disarm with zero watchers; deadlines fire in deadline order, not arming order; a core
    that is never idle (an event moves on every pass) fires within two passes of the deadline,
    counted in passes because a loaded host deschedules threads; a one-nanosecond timeout fires
    on the NEXT pass and chains; a kill mid-wait cancels and disarms (a lingering actor keeps the
    core alive so a stale node would be fired into); a parking core wakes for the deadline in
    both park shapes. Each mechanism disabled in turn is rejected (the park bound: both parking
    cases read 200 ms; the busy-pass check: the busy case fires only when its ticks stop).

  • The loop's non-blocking pass at its floor, and the evpipe no longer re-enables the poll after
    the first park (qev 5.1.0, Huly QB-188).
    The ev_run(EVRUN_NOWAIT) a core with one active
    watcher runs on every pass still read the clock twice, raised libev's wake-up handshake (a
    store plus a full memory fence, there so a producer's ev_async_send would write the evpipe
    to end a sleep this pass never takes) — and counted the evpipe as a pollable fd, so the first
    Park::Loop of a core, which arms the wake and creates that pipe, put the backend poll QB-187
    had removed back on every later pass for the life of the core. A NOWAIT pass now reads the
    clock once, leaves the handshake down (the sender's flag path delivers, and a byte written
    around a park that lands after its poll is picked up by the next pass through the pending
    flags), and the loop's own pipe is not counted. qev/bench/bench-pass.c on i9-12900K: a
    NOWAIT pass over a timers-only loop 51 → 22 ns on WSL2 g++-14 (over a quiet socket
    132 → 101), 19.7 → 15.8 on MSVC. On qb's path (ask-cost): a co_await qb::ask<E>() with a
    500 ms timeout 174 → 115 ns per round trip on g++ (−34 %), 124 → 115 on MSVC, a
    one-chunk timed ask_stream 240 → 179 / 327 → 319; the untimed ask, the push and the
    one-core pass unchanged. Tests are qev's (tests/test-loops.c: a send or a signal between
    two NOWAIT passes delivered by the very next pass with nothing to poll, 400 cross-thread
    sends into NOWAIT-only and park-interleaved loops each delivered before the next, the evpipe
    not counted after a park).

  • Windows: libev's clocks were the 15.6 ms system tick, non-monotonic — every timer judged
    at that granularity (qev 5.1.0, Huly QB-193).
    ev_win32.c read GetSystemTimeAsFileTime
    (a memory read that steps every 1–15.6 ms; 2.2 measured) as the wall clock and, MSVC having
    no clock_gettime, as the monotonic clock too: an ask timeout, an io::async::callback
    delay, a sleep, a retry or the park cap could fire up to 16 ms late, a 0-delay timer armed
    and run inside one tick did not fire, and a wall-clock adjustment moved the loop's now. qev
    reads QueryPerformanceCounter (sub-µs, monotonic) and GetSystemTimePreciseAsFileTime
    now — the first only since QB-195 (below, with QB-190): libev's misconfiguration block had
    switched the monotonic clock back off on MSVC, and the precise system time stood in for it
    until then. Found by qev's loop suite the first time it ran on MSVC — two checks red on the shipped
    code — green since (39 run / 0 failed / 3 skipped). The cost is stated in the open: a
    precise read is ~16 ns where the tick was 3, so on MSVC the NOWAIT pass is 15.8 → 31 ns, a
    timer arm 5.8 → 24 and the timed ask 115 → 166; the next steps of the qev programme take
    the libev timer off the request path (QB-189) and hand the loop the reading qb already
    makes (QB-190). Linux and macOS are untouched.

  • A core with a timer no longer pays libev a syscall per pass — two defaults libev kept from
    2010, fixed in qev (5.1.0, Huly QB-187).
    Any core that owns one active watcher — a request
    timeout, a sleep, a retry, a socket — runs ev_run(EVRUN_NOWAIT) on every pass, busy or
    idle, and that call cost 297 ns whatever the loop held (qev/bench/bench-pass.c, i9-12900K
    / WSL2 g++-14): libev read its clocks through the raw clock_gettime syscall (~95 ns each,
    two per pass, one more per timer arm — a workaround for glibc < 2.17's librt that the config
    probe applied everywhere), and polled the backend even over a loop with no fd. qev now takes
    the syscall only where libc has no clock_gettime, counts its active ev_io watchers
    (ev_io_count()) and skips a poll that would not block when there is no fd. A non-blocking
    pass over a timers-only loop is 50 ns (−83 %), over a quiet socket 131 (−56 %),
    ev_now_update 17.5 (−82 %). On qb's path (tools/probes/ask-cost.cpp in qb-vs-others): a
    co_await qb::ask<E>() with a 500 ms timeout — the documented idiom, which the Savina
    grids never exercised because bank-transaction asks with no timeout — went 798 → 172 ns
    per round trip on g++ and 1108 → 124 ns on MSVC 19.51 (there the whole cost was wepoll's
    poll per pass), a one-chunk ask_stream with a timeout 860 → 238 / 973 → 330. Same-core and
    cross-core cells without a watcher do not move (they never reach the loop). The remaining
    libev cost of a timed request — one clock read and the pass bookkeeping, ~40 ns per pass —
    and the request path without a libev timer at all are the next steps of the qev programme
    (dev/plans/roadmaps/QEV_PERFORMANCE_ROADMAP.md, Huly QB-186).

  • An ask reply resumes the waiting coroutine on the spot (Huly QB-185).
    ask_awaiter::deliver_thunk — reached from resolve_ask(e) in the asker's own handler, on its
    core, in the pass that routed the reply — used to hand the parked frame to the coroutine
    scheduler (schedule_via_current: a hash-set insert for the dedup, a ready-queue push, then in
    the next pass's run() the pop, the hash-set erase and the resume), while the task<E> the
    ask returns already completed by symmetric transfer. The thunk now resumes the frame itself:
    it runs to its next suspension and control comes back into the thunk, and the awaiter it
    lives in may be gone by then (the caller's co_await destroys the task<E> once it has the
    value), which the code states and honours — nothing of the awaiter or the slot is read after
    the resume. The cancel and timeout paths keep the queue: they run from a token callback or a
    timer, not from the handler. The per-type registration flag is constinit (a plain TLS load
    instead of the init wrapper's call on every ask). Measured with the new ask-cost probe (one
    core, a coroutine looping co_await qb::ask<E>() against a reply()ing responder, beside a
    bare push/reply loop), five interleaved launches, i9-12900K: WSL2 g++-14 ask round trip
    54.0 → 46.7 ns (−14 %) — the ask machinery over the two passes it cannot avoid 29.7 → 22.4
    (−25 %) — the push round trip 24.4 → 24.3 (untouched); MSVC 19.51 81.7 → 72.0 (−12 %);
    savina/bank-transaction, the one grid cell built on ask, WSL2 2c-spin 92.4 → 84.0 ns
    per transfer (−9 %), 2c-park 93.2 → 88.3, 1c −2 / −3 %; MSVC 2c-park 154.7 → 147.4, 1c
    260.8 → 250.1 (ten-launch censuses). Measured and dropped in the same pass: draining the
    scheduler's ready coroutines right after the receive, in the same pass (for ask_stream's
    next(), ping, require, ask_all) — a null result (a one-chunk stream 107 → 105.7 ns)
    that cost ~1 ns on every pass, because a deferred wake already ran in the next pass's io phase
    BEFORE that pass's receive, so its push was handled in that same pass and the queue, not a
    pass, was the price. What the sanitizer found, and the test it left: a spawned coroutine
    that completes OUTSIDE run_ready() — now the normal shape when its last ask is answered —
    parks its frame in the scheduler's deferred-destroy list, and destroy_all_suspended()
    scrubbed such a root from that list "so the drain never frees it twice" and then skipped it
    for being done(): 192 + 72 bytes leaked per actor whose coroutine finished in the pass it
    died in (ask-roundtrip under ASan, 3 tests). The cascade frees a scrubbed root now and
    scrubs its stale ready entry with it, and listener::has_work() asks the scheduler for
    has_work() — a frame to free is work — so such a frame is freed on the next pass rather than
    at teardown. Tests: ReplyResumesTheFrameInsideTheRoutingHandler (the continuation has run
    by the time resolve_ask() returns), LastReplyCompletesTheFrameAndKillsTheActorInTheSamePass
    (the leak's shape, under leak detection), HasWorkSeesADeferredFrameAndRunReadyFreesIt, and
    DestroyAllSuspendedScrubsOwnedRootFromDeferredDestroy re-pinned to the new contract.

  • A cross-core hop no longer pays a miss on the producer's own index line: the SPSC ring's
    working indices live on private lines.
    lockfree::spsc::internal::ringbuffer kept
    write_index_ — the index the consumer polls — and cached_read_index_ — the producer's
    snapshot of the consumer's index — on ONE cache line, so every enqueue began by loading both
    from the very line the peer had just been polling. On i9-12900K a snoop of that kind leaves
    the owner's copy behind: the load was a cross-core miss on every hop, as costly as the release
    fence behind the publish (a cpu-clock profile of the two-actor send round trip put 8 544
    samples on it against 9 633 on the fence). Each side now owns two lines, each in its own
    two-line block so no spatial-prefetch partner belongs to the peer: a PRIVATE line with the
    working index and the snapshot (write_index_local_ + cached_read_index_,
    read_index_local_ + cached_write_index_) and a PUBLISHED line with nothing but the index
    the peer reads; enqueue, dequeue, consume_all, front, write_room and empty touch
    only their side's private copy and write the published line to publish — 512 bytes of
    header per ring instead of 128, against 64 KiB of slots, sizeof eight lines and alignof
    two. Two new qb-vs-others probes found it: tools/probes/xcore-hop.cpp (two actors, one per
    pinned core, a push<> or send<> round trip, and a jitter_ns option that breaks the
    PHASE LOCK of a two-actor ping-pong — a 5 ns delay in front of the flush had moved a locked
    round trip by +80 ns) and tools/probes/raw-ring.cpp (the same ring shape with no qb in the
    loop: one load of the published index line before the store, and a launch that read ~120 ns
    per round trip never gets below 166). Measured on qb-vs-others in one quiet session per host
    against develop 670e9433 (9 + 2, CPUs 0,2; ten-launch censuses on every 2c cell): WSL2 /
    g++-14 ping-pong 2c-park 211.7 → 164.7 ns per round trip (−22 %), 2c-spin 207.5 → 163.2
    (−21 %), thread-ring 2c 115 → 80 ns per hop (−31 %, both wait modes), chameneos 2c
    49.1 → 45.6 (−7 %), big 2c −4 %; dev/bench BM_Multi_PingPong_Latency 253 → 203 (−20 %),
    the 8 × 8 pipeline 322 → 264 (−18 %), BM_PINGPONG 64 actors / 8 cores 24.7 → 21.7, the
    cross-core ask −4.5 %; the xcore-hop round trip at a random phase send 240 → 198, push
    240 → 159. Windows / MSVC 19.51: ping-pong 2c-park 263.2 → 204.9 (−22 %), 2c-spin
    257.0 → 206.1 (−20 %), thread-ring 2c 138.6 → 98.9 per hop (−29 %), big 2c-park −9 %,
    chameneos 2c −5 %; dev/bench Multi_PingPong 317.8 → 245.6 (−23 %), the 8 × 8 pipeline
    252.5 → 220.2 (−13 %), the raw-spsc reference 285.6 → 222.9. Nothing else moves, and that was
    measured rather than assumed: every same-core cell (the self pipe is a segmented_pipe, not
    this ring), the one-core pass-cost probe (12.9 / 17.1 / 25.0 ns → 12.7 / 17.1 / 25.0) and
    the batched cross-core shapes — counting, fork-join, fib, bank-transaction — are level on
    fifteen-launch censuses on both hosts after their grid cells had read +2 to +11 %. The same
    campaign measured and parked the flush-order question (Huly QB-183: no cell beyond its
    spread; five flush positions read 166–237 ns locked with no relation to where the flush sat,
    and the same variant ±10 % from one binary's layout to the next), the "sequence in the slot"
    one-line hop (~3 ns per round trip over two lines on a raw ring — not worth a ring) and
    movdir64b (twice as slow: a direct store goes past the caches). Huly QB-184; qb-vs-others
    docs/TUNING.md §16.

  • Actor::time() samples the clock on demand, once per pass, keyed on the pass index,
    instead of unconditionally at the top of every pass — and a pass with no registered callback
    skips the tick phase, so it really is clock-free.
    The first form of this entry (985cbb3a)
    moved the read rather than removing it: the qb::LoopEvent handed to ICallback::on() is built
    from time(), and it was built on EVERY pass whether or not any actor would receive it, so a core
    with zero registered callbacks — every benchmark, and every server that drives itself from io and
    events — still paid one clock_gettime per pass for an event nobody received. perf on
    savina/ping-pong at one core (WSL2 / g++-14, develop 43f62afe) put __vdso_clock_gettime at
    39.6 % of the process. The snapshot copy and the LoopEvent now sit behind
    if (!_callback_list.empty()). The half that was not obvious: with the wall clock gone, a
    SPINNING core's idle pass shrank to ~20 ns of unserialized code and its cross-core exchange got
    slower for it — ping-pong and thread-ring 2c-spin +25 % (ten interleaved launches each, fully
    separated distributions) while every one-core cell took the full gain. The idle poll of the peer
    rings needs ~10–15 ns of serialized work between two reads; an lfence alone left +7 / +14 %,
    spin_loop_pause() (33 ns on i9-12900K) +9 / +20 %, a bounded tight has_data() poll was worse
    than no pacing (+30 %), and the monotonic clock read the park policy already takes on an idle
    pass — lfence; rdtsc under the vDSO, 13 ns — put both cells back on the control's figure. So
    the idle clock (_idle_since) is now stamped on idle passes in every latency mode, a
    latency-0 core included, and only the park itself stays gated on latency > 0; a busy pass reads
    no clock in any mode. Measured on WSL2 / g++-14 in one quiet session against develop
    43f62afe (qb-vs-others, 9 + 2, CPUs 0,2, p50): ping-pong 1c-spin 65.8 → 28.3 ns per round
    trip (−57 %; shipped 3.1.0 measures 97.9 in the same session), thread-ring 1c-spin 38.1 →
    17.8 ns
    per hop (−53 %), 2c-park 219 → 209 / 115 → 108, 2c-spin inside the spread (15-launch
    census: 205.6 vs 209.7 / 104.4 vs 109.3, distributions overlapping), counting level;
    dev/bench BM_Mono_PingPong_Latency 100.2 → 62.9 ns (−37 %), pipeline chain 10 actors / 1
    core 65.1 → 31.8 ns per delivery (−51 %), 8 × 8 cores 314.7 → 314.2, Multi_PingPong 257 vs
    258 over eight interleaved runs, ask round trips −3 / −4 %. Windows / MSVC figures in the
    qb-vs-others results (qb-branch-perf-loop-clock-on-demand/, both hosts).

  • spsc::ringbuffer lays its producer and consumer indices out on separate cache lines and
    each side keeps a private snapshot of the peer's index
    , so an uncontended push or pop
    touches one line and re-reads the peer only when its snapshot says full or empty. (Completed
    by the private working lines above: the snapshot and the working index sit on a line the
    peer never reads, and the published index sits alone on the line it polls.)

  • The qb::ask and coroutine hot paths allocate nothing per operation that a cancel would
    not need.
    cancellation_token::cancel() fires its callbacks in place and
    remove_on_cancel() neuters an entry rather than erasing it while the walk is live, which
    makes the guarantee every awaiter relied on — a callback never runs after remove_on_cancel
    returned, even mid-cancel
    — hold without help; the std::shared_ptr<bool> liveness flag
    that ask_awaiter, cancellation_awaiter, the semaphore park, discovery_awaiter and
    stream_next_awaiter each carried into their callback (one control block plus a closure over
    std::function's small buffer, per qb::ask, per park, per check_cancelled) is gone.
    ask_awaiter holds its token by reference — qb::ask() is its only construction site and
    the by-value context lives in the same frame — and registers its response type once per
    thread per E instead of per ask. The per-core ask table is a slot table the correlation
    id itself indexes — [core:16][generation:26][slot:22]; take pops a FIFO free list, look-up
    is an index and a generation compare, release is a push, nothing is hashed and nothing
    allocates once the table is warm — rather than an unordered_map (a stale id misses on the
    generation, and an id whose core bits are not the resolving actor's is refused before the
    table is even indexed; it went through an open-addressing map first, which perf on
    savina/bank-transaction still put at 10–12 % of the core). Take BINDS the slot in the same
    call (ask_take(owner, slot)), finish() runs once and leaves the destructor a byte to
    compare, and the awaiter takes its entry in its constructor, before the send — so the
    registry is three out-of-line calls per ask rather than five, which is where the cost was
    (perf annotate: the residue sat on prologue, epilogue and one jne, a thread_local
    guard per access on g++, no miss anywhere). Measured on savina/bank-transaction
    (qb-vs-others, same quiet session, control → candidate / candidate again, 9 + 2): the
    registry 12.8 → 4.36 % of the core, 1c-spin p50 7.70 → 7.21 / 7.25 ms and 1c-park
    7.58 → 7.30 / 7.24 on WSL2/g++-14 (≈ 9–10 ns per ask, 154 → 144 ns per transfer), 2c and
    the dev/bench round trips inside their spread; on Windows/MSVC every cell is inside its
    spread (MSVC has no per-access TLS guard to skip), and
    the coroutine scheduler's three std::unordered_set<void*> (in_flight_, owned_frames_,
    suspended_coroutines_) and its std::deque ready queue are a flat_ptr_set and a
    power-of-two ready_ring — MSVC's deque block is one 16-byte item, so every push was a
    malloc. Measured quiet-host, alternating A/B, medians of five: ask round-trip
    879–919 → 734–743 ns same-core on Windows/MSVC (WSL2/g++-14 990 → 902), ask_all over 8
    responders 7.44 → 4.18 µs (WSL2 4.86 → 3.42), spawn latency at 1000 in flight 270 → 185 µs
    (WSL2 378 → 152), 512 parked coroutines released through a semaphore / mutex / rwlock / latch
    2.7–3.7× faster on both platforms, parallel_map 1024/32 541 → 366 µs (WSL2 215 → 121),
    shared_task fan-out 512 143 → 53 µs (WSL2 49.2 → 20.4). Nothing on the per-item
    async_stream path changed and its cells move ±3–9 % in both directions across the two
    compilers.

  • Two coroutine benchmark harness defects fixed, so those cells now measure the work. Six
    actor harnesses summed a mean-RTT counter with kAvgIterations semantics they never asked
    for (record_mean_rtt_counter in shared/BenchmarkIterationSink.h is the one accumulator
    now), and the four qb-io coroutine harnesses' drain_until pumped with run_for(1ms), which
    slept a millisecond per trip (~5 ms at Windows' timer granularity) including the trip that
    had just satisfied the predicate — every cell reported 5–6 ms wall and 0 CPU time. The pump is
    run(EVRUN_NOWAIT) now; Semaphore/512 reads 36.7 µs on Windows where it read 127.

  • reply() / forward() no longer stall the copy on their own header stores. Written as
    std::swap(dest, source) + alive = 1 + send, the relocation's first wide load spanned two
    4-byte stores still in flight — measured on savina/bank-transaction, libc memmove was 4.3 % of
    the core, all of it that load. Both now go through detail::event_wire: one 16-byte header
    store, the copy, and only THEN the original's alive raised (see Fixed for why the order is
    the contract). p50 on the shape moved 8.25 / 8.08 / 5.06 / 4.93 ms (1c spin, 1c park, 2c
    spin, 2c park) to 8.19 / 8.22 / 4.54 / 4.60 ms on Linux/g++-14. The whole chain, base
    9d4aa94c → this commit, measured in ONE quiet session per host beside shipped 3.1.0
    (qb-vs-others docs/TUNING.md §12): WSL2 g++-14 9.40 / 9.44 / 5.09 / 5.02 → 8.06 / 8.80 /
    4.56 / 4.77 ms
    (3.1.0: 14.71 / 14.58 / 9.21 / 9.31), Windows MSVC 13.65 / 13.86 / 7.97 /
    7.78 → 12.91 / 12.82 / 7.57 / 7.56
    (3.1.0: 25.48 / 25.45 / 29.20 / 33.45) — −14 % / −10 %
    at 1c / 2c spin on g++, −5 % / −5 % on MSVC, and 1.8× / 2.0× (WSL2), 2.0× / 3.9× (Windows)
    over shipped 3.1.0.

  • The receive path writes nothing into an event before routing it. __receive__ and the
    flush stored alive = 0 into every arriving event; that byte store into a line the dispatch
    then loads whole was 70 % of deliver_thunk's first-header-load time on the same shape. An
    event handed to any transport now carries alive == 0 structurally — event_wire's helpers
    clear it in the header store, its copy clears it in the copy — so the store said nothing.

  • An actor's coroutine census is an intrusive, single-threaded refcount, not a
    shared_ptr<atomic>.
    The lazily-allocated counter above is detail::coro_census now
    ({active, refs}, qb/core/Actor.h) behind detail::coro_census_ref, which the spawn wrapper
    copies and releases: every spawn was a shared_ptr copy — two atomic RMWs — plus an atomic
    fetch_add/fetch_sub on the count, on a counter only ever touched from the owning core's
    thread. Same guarantee (the count outlives the actor, the last frame frees it), zero atomics.
    cancellation_token's state took the same shape: an intrusive non-atomic refs in place of
    shared_ptr, with explicit copy/move (CancellationToken.RefcountFollowsCopiesAndMoves,
    CancelSurvivesCallbackDroppingLastOtherHandle).

Added

  • An event type wider than the cross-core mailbox ring is refused at compile time (Huly
    QB-61).
    qb::detail::event_fits_ring<T> -- the sizeof-derived bucket count against
    qb::detail::max_deliverable_buckets, the ring's slot count (1023 buckets, ~64 KiB with the
    default 64-byte bucket) -- is asserted in routing_safe_type_id<T>(), the funnel every enqueue
    sink reaches, with a message naming the remedy (bulk data behind a pointer member,
    Pipe::allocated_push). Such an event could never be delivered cross-core: the flush disposed
    it and logged a CRIT, and the message was lost. The runtime drop stays for the trailing bytes
    of an allocated_push, a runtime quantity. SharedCoreCommunication::MaxRingEvents and
    VirtualCore::kMaxDeliverableBuckets are that one constant, asserted equal. Both polarities
    are pinned in event-field-shadow (a payload rounding to exactly the ceiling accepted, one
    byte past it and a 1 MiB blob refused -- sized by the ceiling, never by sizeof(qb::Event),
    because the Itanium ABI reuses the base's tail padding where MSVC does not), and the
    event-shadow negative-control battery plants the 1 MiB event at all five entry points.
  • Abandoned coroutine frames are reported in every build, and counted (Huly QB-84).
    ~CoroutineScheduler deliberately abandons frames still suspended on a watcher or a wait list
    rather than destroy them (their watchers still reference them); until now the only trace was an
    fprintf behind #ifndef NDEBUG. qb::io::async::report_abandoned_coroutine_frames -- out of
    line in qb/io/logger.cpp, the placement and policy of the detached-exception report --
    prints one WARNING line on qb::io::cerr and adds to
    qb::io::async::abandoned_coroutine_frames_total(), the process-wide tally an operator or a
    test reads back once the frames are gone. The lifecycle the framework drives itself
    (~listener, reset_coro_scheduler()) runs destroy_all_suspended() first and so DESTROYS
    parked frames rather than abandon them -- pinned in scheduler-abandoned-frames beside the
    directly-owned scheduler the report is for.
  • clang-cl is a supported Windows toolchain: the clang-cl preset, 0 warnings, the suite
    green (Huly QB-201).
    LLVM's Clang behind MSVC's command line, ABI, CRT and STL, and on the
    same host, tree and CRT it ran qb's dispatch 10–18 % faster than MSVC 19.51 (savina/ping-pong
    1c 31.6 → 27.4 ns, big −13.6 %, fib −10 %, pass-cost k = 1 −9.8 %, push −9 %; qb-vs-others
    results/desktop-b67osn6-win-msvc/qb-46-clang-cl/, QB-46 — half of the MSVC/g++ gap is codegen,
    the other half the platform). qbCompiler.cmake recognises it (CMAKE_CXX_COMPILER_ID Clang
    with CMAKE_CXX_COMPILER_FRONTEND_VARIANT MSVC) and gives it the MSVC flag set with clang's
    exemptions: through the GCC/Clang branch it took -Wall, which clang-cl reads as /Wall =
    -Weverything (17 919 warnings on this tree), plus -fPIC / -fomit-frame-pointer /
    -ffunction-sections ignored with a warning each; the event loop's strict GNU warning set is
    left out for the same reason (1 233 more on ev.c), its CRT deprecations of dup2 / close /
    getenv declined on the target; googletest's four targets carry the -Wcharacter-conversion
    exemption (gmock did not even compile: its own CMake adds -WX under clang-cl). cmake --preset clang-cl on Windows; readme/7_reference/building.md has the row.
  • qb::allocator::segmented_pipe<T> + segment_pool<T> (qb/system/allocator/segmented_pipe.h)
    — a FIFO of fixed-size segments (256 KB, from a pool the owning VirtualCore keeps) that grows
    by linking a segment behind the tail and never moves what it holds: allocate_back(n) is a
    compare and a cursor add while the tail has room, a range never straddles two segments, a
    request wider than a segment gets a dedicated exactly-sized one, and the read side walks
    front() / consume_front() / pop_front() a segment at a time, each popped segment going
    straight back to the pool. qb::VirtualPipe is now this type (was allocator::pipe<EventBucket>).
  • qb::allocator::slab_cache (qb/system/allocator/slab.h, qb/io/slab.cpp) — the
    process-wide source of the pool's memory: 2 MB slabs mapped by the platform (mmap trimmed to
    a 2 MB boundary on POSIX, VirtualAlloc on Windows), on Linux madvise(MADV_HUGEPAGE)d and
    populated in one MADV_POPULATE_WRITE pass (5.14+), carved eight standard segments to a slab,
    and kept on a free list when a pool gives them back so the next engine or core draws memory
    that is already mapped and faulted. trim() returns the cached slabs to the OS. Cold path
    only — one acquire per eight segments of growth, never per event.
  • CoreInitializer::setIdleSpin() / Main::setIdleSpin() — how long an idle latency > 0
    core keeps polling before it parks on its mailbox, a time floor measured from its first idle
    pass (default kDefaultIdleSpin, 50 µs; getIdleSpin() reads it back). Until now the floor
    was an event-count credit refilled from the previous pass, which parked a one-event-per-pass
    workload after two or three empty passes: every hop of a two-core request/response exchange
    paid an OS park + wake (measured 2–13 µs against ~300 ns of polling). setIdleSpin(0) restores
    the park-on-first-idle-pass behaviour.
  • listener::has_work() — whether a qb-io run() turn has anything to do, read from the
    loop's own ev_active_count() / ev_pending_count() plus the deferred queue and the coroutine
    scheduler's ready list. VirtualCore uses it to gate its libev pass; consumers driving a
    listener with EVRUN_NOWAIT get the same gate.
  • mpsc::ringbuffer::has_data() — does any producer ring hold an item; the predicate of the
    mailbox park.
  • listener::run_once_for(cap), arm_wake() / disarm_wake() / is_wake_armed(), wake()
    — the park half of a qb-io loop: one ev_run(EVRUN_ONCE) turn that blocks in the backend poll
    for at most cap (a one-shot timer keeps the loop alive and bounds the block) until a watcher
    fires or wake() — ev_async_send, the one libev call safe against a concurrent ev_run —
    lands from another thread. A turn that already has a deferred callback, a ready coroutine or a
    pending event does not block, because ev_run computes its wait from watcher deadlines alone.
    The embedded qev profile keeps the async family for it (QB_EV_ASYNC_ENABLE 1; six families
    compiled out instead of seven — three symbols and 24 bytes of struct ev_loop, measured).
  • Emplace forms of qb::ask and qb::ask_by (qb/core/patterns/request.h):
    co_await qb::ask<Deposit>(ctx, account, 500ms, amount, txn) constructs the request in its
    pipe slot
    and writes each field exactly once, where the by-value form built a temporary,
    moved it into the pipe (a memcpy whose first 16-byte load sits on header fields the
    constructor had just written with narrow stores — a store-forwarding stall on every ask), and
    then set the correlation id on the copy. CoroContext::push / push_to now return the built
    E& so the id can be set in place. ask_by<E>(ctx, target, dl, args...) is the deadline twin
    and fails fast on a spent budget without building anything. Measured on savina/bank-transaction
    (one ask per transfer, Linux/g++-14): Account::start's frame — 21.7 % of the core, the
    by-value build and its copy — left the top of the profile, and the shape's p50 moved from
    9.74 / 9.60 ms (1c spin/park, develop 9d4aa94c — the base this work branched from,
    NOT shipped 3.1.0, which measures 14.71 / 14.58 in the same session; the label was corrected
    after the control was re-measured beside it) to 8.25 / 8.08 ms before the wire work
    below. Tests: ActorCoroutineAsk.EmplaceAsk* (6).
  • cancellation_token::cancel_hook + token.link(hook) (qb/io/async/coroutine/cancellation.h):
    an intrusive, allocation-free cancellation registration — a {fire, ctx, prev, next} node the
    caller owns, linked into the token's state and unlinked on completion, fired (after being
    detached) before any on_cancel callback. ask_awaiter uses it in place of on_cancel, whose
    std::function + vector push was one heap allocation per ask and a linear remove_on_cancel
    per completion. link() on an already-cancelled token fires inline and returns false; a hook
    may unlink a sibling from inside its own fire. Tests: CancellationToken.Hook*,
    CancellationToken.LinkOn* (9 new).
  • qb::detail::event_wire (qb/core/Event.h): the event header as one 16-byte unit —
    swap_dest_source(e), set_dest(e, dest) and copy(dst, src, bytes), each composing the
    header in one register (SSE2, NEON, or two 64-bit words) and storing it once, with the
    liveness bit settled in the same store; copy reads the header back as exactly the bytes that
    store wrote, then moves the tail. Layout assumptions are static_asserted against qb::Event
    itself. Tests: EventWire.* (12).

Fixed

  • The embedded loop's shared object carries the qev version (src/qb/ev/CMakeLists.txt,
    byte-shared with isndev/qev): a BUILD_SHARED_LIBS=ON build derived the library VERSION as
    libtool's current.age.revision with age and revision pinned to 0, naming every release of a
    major line libqev.so.5.0.0; it is PROJECT_VERSION now, SONAME libqev.so.5.

  • qb::ask_guarded crashed under clang 15–21 at -O2/-O3 on x86-64 Linux and Intel macOS, and
    every pattern that takes its request by value was one frame layout away from the same fault (Huly
    QB-213).
    LLVM folds a coroutine parameter copy the body never writes back into the caller's
    byval argument slot, then spills that slot into the coroutine frame at the natural alignment of
    the LLVM struct type (8) while every read keeps the C++ alignment (64 for a qb::Event): the first
    aligned 16-byte load out of the slot faults whenever its frame offset is not a multiple of 16 -- a
    crash that appears or vanishes with the layout, at -O2 and above only, invisible to -O0, ASan and
    UBSan (LLVM issue 159571, fixed by pull request 159765 in LLVM 22 and backported to no earlier
    branch; GCC and MSVC build parameter copies as frame members, and under clang-cl the Windows ABI
    passes such arguments by reference). The new qb::io::async::pin_frame_copy(param) makes the copy a
    written, escaped object with a memory-operand asm barrier -- the asm emits no instruction, the copy
    costs what the spill cost plus, where a pattern hands the request to an inner coroutine, one 64-byte
    stack copy the optimiser used to forward (ask_guarded<Ping>.resume: four such copies instead of
    three), and the call is a no-op everywhere but clang on a byval ABI -- so the optimiser keeps it
    and the frame lays it out at its declared alignment; the nine request-taking pattern coroutines (ask_by, ask_retry, ask_guarded,
    both ask_all, gated_ask, ask_any, ask_quorum, ask_stream) call it on their request -- ask
    itself stopped being a coroutine in this release (Changed, QB-214) -- and
    the coroutine chapter tells a user coroutine when to do the same. Reproduced on WSL2 Debian 13 with
    clang 19.1.7: coroutine-resilience and init-patterns SIGSEGV at -O3 on a 64-byte copy from
    frame offset 360; an IR census of the resilience test found exactly that one frame copy whose
    claimed alignment its offset cannot satisfy, none after the pin, and the 202-test standalone suite
    green at -O3. qb's Linux CI gains a clang-19 lane so the oldest clang still common on developer
    machines keeps that shield honest.

  • clang-cl: every task<E> over a qb::Event crashed on its first co_await — the MSVC STL's
    coroutine_handle::from_promise() / promise() are wrong for an over-aligned promise, and qb
    now goes around them (Huly QB-200).
    The STL implements both as
    __builtin_coro_promise(ptr, 0, ...), an alignment of zero that MSVC's own compiler ignores (it
    lays every promise out at offset 16 of the frame, whatever its alignas) and that clang, which
    lays an alignas(64) promise out at offset 64, turns into a handle 48 bytes into the frame:
    resume() calls whatever bytes sit there. Every qb::Event is cache-line aligned, so every
    task<E> over an event — qb::ask's result, every request/reply coroutine — has an
    over-aligned promise, and under clang-cl 22 every ask test died with 0xC0000005 at -O0 as
    at -O2 while coroutine-basics (a task<int>) passed. Found by the QB-46 compiler A/B and
    reproduced in thirty lines with a bare alignas(64) promise (delta 16 wrong, 64 right).
    qb/io/async/coroutine/promise_access.h carries detail::handle_from_promise /
    detail::promise_of, which hand the builtin alignof(P) on clang-cl and ARE the standard calls
    everywhere else; the four from_promise and twenty-eight promise() of task.h,
    generator.h, scheduler.h and VirtualCore.cpp go through them. Test
    qb-io-test-unit-over-aligned-promise asserts the promise's alignment ≥ 64 first, then the
    handle→promise round trip and a task<Wide> awaited sixteen times; under clang-cl the whole
    suite is green (194/194).

  • QB_EV_BACKEND=iouring ran a quiet socket's pass 47× slower than epoll; it is at parity
    now, and every Linux CI job of qb runs the suite on both (qev 5.1.0, Huly QB-81).
    The env
    variable had shipped since 3.1 with no figure behind it, and the first measurement (the
    io-pass probe, one pinned core, a loopback pair) read 1345 ns against 28.5 for the pass a
    core pays over one quiet fd — a syscall storm inside the backend (the deadline timerfd armed at
    "now" on every non-blocking poll, expiring at once, drained and re-armed through
    io_uring_enter, three syscalls a cycle at 300k cycles a second), then a ready fd noticed only
    at the scheduler tick once the storm stopped (COOP_TASKRUN defers completion work to the
    next syscall, and a quiet loop made none), then a timers-only loop that never counted as
    timers-only (the backend's own timerfd watcher in iocnt, so QB-187's no-poll pass and
    QB-190's inline gate never applied). All three are in qev's changelog; on this side the
    backend is EXERCISED: tests/io/system/CMakeLists.txt's -DQB_IO_EV_TEST_BACKENDS matrix gains
    an anti-vacuity guard (event-loop-lifecycle joins the backend-sensitive set with a case that
    reads the ambient QB_EV_BACKEND — pinned, the backend must be compiled in and a fresh
    listener must be ON it, else a variant would test epoll under an io_uring name; a seccomp
    profile refusing io_uring is the realistic case), .github/workflows/cmake.yml enables
    epoll;iouring on every Linux job, and the whole suite was forced onto io_uring under
    sanitize and sanitize-thread: 385/385 each, 0 fallback. Measured against epoll on WSL2
    (kernel 6.6, g++-14, medians): a pass with one quiet socket at the default 1 µs cadence
    28.8 / 26.4 ns, polled on every pass 126.9 / 40.9, one far timer 26.5 / 26.4, wake p50
    3.98 / 4.18 µs, a parked core woken by a socket p50 41.6 / 41.3 µs, syscalls on a quiet socket
    960k/s against 1/s. Parity with a different shape — no syscall on a quiet pass, ~0.2 µs more
    per delivered event, two syscalls per park — so epoll stays the default; io_uring is the
    right choice for a core that polls on every pass. The table and the reasoning are in
    readme/6_guides/performance_tuning.md ("io_uring, measured (3.2.0)").

  • CoroutineScheduler::current()'s fallback and two awaiter defaults created libev's global
    default loop from a core thread — a data race, and a loop nobody pumps (Huly QB-197).

    CoroutineScheduler stored an ev::loop_ref it never read, and the one effect of its default
    argument ev::get_default_loop() was to CREATE the process-wide default loop from whichever
    core thread first took the current() fallback — an actor's onInit awaiting during the
    first resume of __drive_init__, before __begin_activation__ had set the listener's
    scheduler as current — so ThreadSanitizer, running the whole suite on io_uring, reported the
    read of ev_default_loop_ptr in another core's ev_loop_destroy racing that creation. The
    member and the parameter are gone: the listener constructs its scheduler bare, the fallback
    creates nothing global. timer_awaiter and socket_awaiter lose the same default argument —
    the header already warned that the default loop parked the coroutine for ever, the API doc
    lists the loop as a parameter, and every construction site injects listener::current.loop().

  • The sanitizer and coverage presets instrument the event loop at last, and its wake protocol
    is ThreadSanitizer-clean (qev 5.1.0, Huly QB-192).
    The embedded qev target never went through
    qb_apply_compiler_flags(): 548 TUs carried -fsanitize=thread and ev.c did not, so every
    sanitize / sanitize-thread / coverage run of qb had measured an uninstrumented loop
    (0 __tsan hooks in ev.c.o; 19 __tsan / 25 __asan hooks and 389 gcov counters now). Once
    it was, TSan found the loop's cross-thread wake flags exchanged through volatile accesses
    fenced by hand — correct on every target and a race by the letter of C11 — fixed in qev with the
    __atomic builtins at zero cost (codegen-identical, proven), and listener::_wake_is_pending()
    reads that flag with acquire semantics on this side. The instrumentation had fallout of its
    own, each a real defect: clang's function sanitizer rejected libev's callback dispatch
    (exempted on the two dispatch functions), gcov counters bumped from every core thread made lcov
    refuse the report (-fprofile-update=atomic), and a (void) ::read() in the shared fd fixture
    that Ubuntu's fortified glibc refuses to let a cast silence. Three presets 375/375 with ev.c
    inside, a hundred consecutive NOWAIT stress runs under TSan clean.

  • A build whose OpenSSL or zlib was not found no longer compiles as if it had been.
    qb_initialize_project_configuration() emitted QB_WITH_SSL=1 / QB_WITH_COMPRESSION=1 from the
    raw options BEFORE qbDependencies.cmake looked for the libraries, and a miss there only flipped
    the option OFF for the summary — so a host without OpenSSL that left the default ON compiled
    every TU, and exported to every consumer of the installed package, with QB_WITH_SSL=1 while
    the configure printed SSL: OFF (measured on Windows/MSVC with neither library discoverable:
    177 compile lines carrying both definitions, 0 carrying QB_HAS_SSL=1). qb's own sources gate on
    QB_HAS_* and were right; anything reading the documented QB_WITH_* name was lied to — qbm-http's
    body-uncompress-malformed test #include <zlib.h>s under it, which is a compile error in exactly
    that degraded build. The two definitions are now emitted next to the QB_HAS_* ones, after
    detection, from the value the summary prints (0 of either in the same configure); that test
    reads QB_HAS_COMPRESSION like the rest of the tree.

  • The dense dispatch table no longer reallocates on every new actor id. router::key_table
    grew with reserve(max(idx + 1, size() * 2)) on each insert — one element past what the previous
    doubling had left — so every subscription of a NEW id copied the whole table: O(n) per
    registerEvent, O(n²) per core, times one table per event type. Invisible to every static-topology
    test and benchmark (the tables reach their size once and stay there) and catastrophic to a dynamic
    one: savina/fib (57 313 actors born and dead inside one window, seven subscriptions each) measured
    43.2 s on Windows/MSVC and the equivalent on Linux against CAF's 86 ms, super-linear in the live
    count. The table now grows by CAPACITY (reserve only when idx >= capacity(), doubling it) —
    178 ms on the same run, linear again. Introduced on develop with the dense tables (axis I); never
    shipped.

  • The first non-blocking TLS connect of a process no longer reads as a failure on Windows.
    ssl::socket::n_connect(endpoint, hostname) returned the TCP connect's -1 ("in progress") and
    then minted the SSL state; the process's first SSL_CTX_new() runs OpenSSL's one-time library
    init, whose successful Win32 calls clear the thread's last-error slot — measured with a probe,
    that call and only that one turned a pending WSAEWOULDBLOCK into 0. Every caller reads
    get_last_errno() after the return to tell "in progress" from "failed":
    async::tcp::connector::run() saw 0, socket_no_error(0) is false, and the first TLS client
    connect of a process was delivered as a failure — on POSIX errno is untouched by the same
    calls, so nothing else could see it. n_connect now captures the connect's errno before the SSL
    setup and restores it before returning. Surfaced by verify-windows.ps1 on the 3.2 candidate:
    ssl-socket-loopback's NonBlockingConnectVariantsPrepareSslState failed with
    n_connect result=-1 errno=0 exactly when --gtest_shuffle drew it first (1 of 5 presets that
    run; 5/5 in isolation, 0/N after any prior TLS connect). ssl-first-connect-errno pins it: one
    test alone in its binary, so it is the process's first SSL operation on every platform whatever
    the shuffle seed.

  • A sanitized build no longer links a system GoogleTest. Under QB_SANITIZE the
    FIND_PACKAGE_ARGS system-first lookup is not offered for GoogleTest (nor Google Benchmark):
    the pinned tag is built with the same sanitizer flags as every other target. A system package
    is compiled without the sanitizer, and libc++'s container annotations then disagree between
    its TUs and ours — GTestIsInitialized() copies the argv vector GetArgvs() built inside the
    library, and AddressSanitizer reports a container-overflow that is not one. Measured on
    macOS 26.6 / AppleClang 21 with Homebrew googletest: 3 of 372 root sanitize tests aborted,
    intermittently (heap layout decides), and the same binaries pass with
    detect_container_overflow=0. QB_USE_SYSTEM_GTEST=ON still forces the system package and
    now warns under a sanitizer.

  • The start barrier yields once it has spun. Main::__wait__all__cores__ready() and the
    calling thread's wait in Main::start(true) spun on the ready counter without ever yielding,
    so an engine started with more cores than CPUs — hardware_concurrency() ignores affinity
    masks and cgroup quotas — made the last core to initialise compete for a CPU against every
    core already waiting for it. Under ThreadSanitizer that is a hang, not a delay: on a 24-vCPU
    WSL2, 23 spinning acquire loads held TSan's atomics lock in read mode continuously, the 24th
    core's fetch_add never got in, and MainLifecycle.StopMultiCoreGracefulNoError ran past its
    600 s timeout at any CPU count. Both loops spin 1024 times and then yield(); the same test
    completes in 2 s and the signal variant went from 61 s to 2 s. And then it sleeps: with
    one CPU per waiter there is nobody to yield to, yield() returns at once, and the polls'
    acquire loads still starved the last core's every atomic op behind 23 readers of TSan's
    atomics lock — the three multi-core lifecycle tests measured anywhere from 0.1 s to 30 s per
    run on the same host. After 256 yields the poll sleeps 50 µs between loads (one shared
    wait_sync_start for both loops); the tests take 100 ms, every run, and an engine whose
    cores arrive more than ~100 µs apart pays at most one quantum, once.

  • A cross-core forward() of an event already reply()ed delivered a copy the receiver never
    destroyed.
    send(Event const&)'s cross-core fast path relocates the ORIGINAL bytes into the
    peer's mailbox, and after a reply() those bytes carried alive = 1; the receive-side
    alive = 0 store masked it — until that store was removed (above), when the dispatcher skipped
    the destructor on every such copy and the payload leaked. send now refuses to relocate a
    raised flag: an already-replied event takes the local pipe, whose event_wire::copy clears it,
    and reply()/forward() raise the original only AFTER the copy is taken. Reply-then-forward
    in one handler yields two owning copies, each destroyed exactly once, same core or not —
    RelayChain.* (5), including both *ReplyThenForwardFanOutDestroysBothCopies polarities.

  • The timer_awaiter and socket_awaiter usage examples compile again. Their @code
    blocks in qb/io/async/coroutine/awaiter.h still built the awaiter without the event loop
    QB-197 made a required constructor argument, so the two examples the doc-example guard had
    gated stopped compiling (no matching constructor, measured by its compiling half on
    macOS/AppleClang 21 — the half --check-baseline cannot run). They pass
    qb::io::async::listener::current.loop() now, exactly what sleep() / wait_readable() /
    wait_for_io() inject, and each names the helper that spells it for you.