Architecture

Architecture

vanilla is a thread‑per‑core HTTP/1.1 server. Each worker is pinned to a core, owns its own state, and never shares mutable data with other workers except a handful of explicitly mutex‑guarded caches. There is no work‑stealing scheduler and no cross‑core synchronization on the hot path.

            ┌─────────── listening socket (SO_REUSEPORT) ───────────┐
            │                                                        │
        worker 0                worker 1               ...       worker N
   ┌───────────────┐      ┌───────────────┐               ┌───────────────┐
   │ epoll_wait     │      │ epoll_wait     │               │ epoll_wait     │
   │ PlainState     │      │ PlainState     │               │ PlainState     │
   │  conns[fd]     │      │  conns[fd]     │               │  conns[fd]     │
   │ AsyncReactor   │      │ AsyncReactor   │               │ AsyncReactor   │
   │  watches[fd]   │      │  watches[fd]   │               │  watches[fd]   │
   │ pg_async pool  │      │ pg_async pool  │               │ pg_async pool  │
   └───────────────┘      └───────────────┘               └───────────────┘

SO_REUSEPORT accept

Every worker bind()s the same address with SO_REUSEPORT; the kernel load‑balances incoming connections across the workers. There is no single accept thread and no shared accept lock. The one consequence to remember: two requests from the same client can land on different workers, which is why the few caches that must be coherent across requests (e.g. the crud cache that validate.sh probes for X‑Cache: MISS then HIT) live in a process‑shared, mutex‑guarded struct rather than per‑worker.

The flat, fd‑indexed connection table

Per‑connection state (ConnState) is stored in PlainState.conns, a plain array indexed by file descriptor, grown by doubling. Lookup is conns[fd] — no hashing, no per‑request allocation. Each ConnState carries a persistent read_buf (8 KiB) and write_buf (16 KiB), reused across requests on that connection.

Connection pooling (free_conns)

Under -gc none, allocating a fresh ConnState + its two buffers on every accept and freeing them on close turns into permanent allocator‑arena churn under reconnect‑heavy load. So PlainState keeps a per‑worker free‑list: close_conn resets every field of the ConnState (buffers length‑zeroed, capacity retained; the two -1 sentinels restored) and pushes it to free_conns; state_for pops from there before allocating. Memory is bounded by the worker's peak concurrent connection count, and steady‑state reconnects allocate nothing. See Memory Management under gc none.

The async runtime (opt‑in)

When ServerConfig.async_handler is set, each worker additionally owns an AsyncReactor and routes watched‑fd readiness through it. When it isn't set, a single per‑worker has_async bool skips all of it — so the synchronous hot path (pipelining, busy‑poll hybrid, large‑body streaming‑drain, sendfile, EPOLLOUT backpressure) is byte‑for‑byte unchanged when async is off.

The model is a small single‑threaded reactor with park/resume:

1. A handler that must wait on something (a DB socket, an upstream connection, a timerfd, the client becoming writable) calls ac.watch(ext_fd, interest, continuation, udata) and returns .suspend.
2. The worker registers ext_fd in its own epoll, parks the client connection (its response isn't produced yet), and goes on serving other connections.
3. When ext_fd is ready, the worker runs the continuation, which either appends the response and returns .done (send + unpark) or re‑arms a watch and returns .suspend (multi‑step chains: connect → send → recv).

AsyncReactor — watches[fd]

The reactor is the same flat, fd‑indexed, doubling‑grown layout as conns. A WatchEntry is just active/client_fd/cont/udata for the overwhelming common case of one watch per fd (a timerfd, an SSE stream, a single in‑flight query) — no allocation. Only when a second distinct client parks on an already‑active fd (a pipelined Postgres connection multiplexing several queries) does the entry promote to a queue of ParkSlots, drained in submission order. See Async Postgres and Pipelining for the FIFO, the persistent‑fd handling (FIX 3), and the queue‑buffer reuse.

The watch primitive is deliberately generic: the DB driver, reverse‑proxy/upstream calls, timers, and SSE/WebSocket backpressure are all consumers of the same one‑line interface.

HTTP/1.1 request framing

The parser (http1_1/request_parser) is zero‑copy and allocation‑light:

• Framing (frame_request_length_lim) does a single pass over the header block, detecting Content-Length / Transfer-Encoding as it goes, and returns -1 until the full body (per Content-Length) is buffered — so a handler never sees a partial body, no matter how the request is split across TCP segments.
• Decoding (decode_into) fills an HttpRequest in place, with no !HttpRequest Result boxing on the hot path. Method/path/query are returned as borrowed slices (tos/views) into the read buffer, not copies.
• A skip‑decode fast path recognizes the fixed pipelined‑plaintext request by raw prefix and blits a precomputed response before any parsing — this is the ~39.8 M req/s path.

Request lifecycle (async DB endpoint)

recv ──► frame (full body?) ──► decode_into ──► route
                                                  │
                                       handler builds params,
                                       acquire_pipelined() a pooled pg conn,
                                       async_submit(query), async_flush()
                                                  │
                                       ac.watch_persistent(conn_fd, …) ; return .suspend   (PARK)
                                                  │
                          … worker serves other connections …
                                                  │
        conn_fd readable ──► drain replies ──► resume continuation ──► render into w.scratch
                                                  │
                                       emit response ; return .done   (UNPARK → send)

Everything on this path — params, the wire bytes, the reply buffers, the rendered body — is built into reused, per‑worker buffers, not fresh allocations. That is the whole point of Memory Management under gc none.