Skip to content

CWIST v3.4

Choose a tag to compare

View all tags
@gg582 gg582 released this 10 Sep 15:16
· 1078 commits to main since this release
v3.4
d760b84

CWIST v3.4 — gRPC client, proto codegen completeness, and WASM client-side support

Major changes compared to v3.3:

  1. gRPC Client: Load Balancing & gRFC A6 Retries (714b9b6, bb04dae)

    • HTTP/2 gRPC client (h2c cleartext or TLS with ALPN "h2"): unary and server-streaming calls, grpc-timeout deadlines, RST_STREAM cancellation.
    • cwist_grpc_channel: dns/ipv4/ipv6 target resolution (doc/naming.md) into per-address subchannels; failed dials back off per doc/connection-backoff.md (1s initial, x1.6, 120s cap, +/-0.2 jitter, reset on SETTINGS handshake).
    • Client-side load balancing (doc/load-balancing.md): pick_first (default, sticky with in-order failover) and round_robin (eager connect, READY-set rotation), plus aggregated channel connectivity state.
    • gRFC A6 retry engine between the channel and the LB pick: maxAttempts (client-capped at 5), jittered exponential backoff, retryable status codes, call deadline shared across attempts, per-attempt remaining grpc-timeout, and the grpc-previous-rpc-attempts header on every retry.
    • Transparent retries (A6): RPCs that never left the client retry until the deadline; RPCs refused before server application logic (RST_STREAM REFUSED_STREAM, GOAWAY last-stream-id below the stream) get one immediate retry — neither counts against maxAttempts nor the throttle.
    • Server pushback: grpc-retry-pushback-ms honored in both directions (retry after exactly N ms / do not retry); retryThrottling token bucket per channel.
    • JSON service config subset (doc/service_config.md): loadBalancingConfig/loadBalancingPolicy, methodConfig (name matching, retryPolicy, waitForReady, timeout), retryThrottling — with A6 validation rules.
    • Server spec compliance: error responses are now Trailers-Only end-to-end (unary: single HEADERS frame with END_STREAM; streaming: Response-Headers delayed until the first message), so conforming clients can actually retry. cwist_grpc_stream_set_retry_pushback() emits pushback from handlers.
    • tests/test_grpc_channel: LB stickiness/failover/rotation, retry matrix (retryable/non-retryable codes, maxAttempts exhaustion, pushback both ways, throttling, deadline across attempts), JSON config validation, GOAWAY transparent retry against a raw-socket fake, and resolver schemes.

  2. cwist proto Codegen Completeness (b9a4fc7, 6ebd78a)

    • oneof, map, fixed-width types (fixed32/64, sfixed32/64, double) for proto3 fields.
    • protoc --descriptor_set_out binary input (auto-detected or via --descriptor-set), alongside the existing text path.

  3. WASM Client-Side Support Wave (df01f68, f0ac2ef, 04907c9, 6e0ee35)

    • cwist_app_dispatch_memory(): run cwist_app routing and handlers on in-memory request/response buffers with no sockets — the same C handlers run inside a Service Worker or a JS fetch-interception layer.
    • make wasm: libcwist_wasm.a Emscripten target shipping the socket-independent core (app dispatch, mux/middleware, HTTP/1 parser/serializer, query map, cJSON, memory utilities).
    • cwist_db_open_memory() / cwist_db_serialize(): official in-memory/blob database API over sqlite3_deserialize-style buffers.
    • <cwist/wasm/typedarray.h>: zero-copy serialization helpers mapping C struct arrays onto HEAP TypedArrays instead of round-tripping through snprintf JSON.

  4. gRPC Server Leftovers (cc4cd15)

    • The builtin grpc.health.v1.Health Watch method now streams status changes live over the HTTP/2 transport path (snapshot fallback on the buffered path).

  5. v3.4 Performance Wave (45e6552, 668625d, 48088c4, a046efe, 95ee64f, 86ded77, c5bc5cc)

    • C1M reactor tail latency: cooperative request-batch yields (CWIST_HTTP_YIELD_BATCH), batch responses coalesced into one writev per turn (256 KiB stash), reactor wake eventfds registered with io_uring (IORING_REGISTER_EVENTFD), post bursts coalesced to a single wake.
    • BDR cache: lock-free learn/read paths (CAS-published entries, atomic blob swaps, EBR reclamation) and hit-time revalidation hooks (cwist_bdr_put_revalidatable).
    • Classic pool: CWIST_POOL_PREWARM / CWIST_POOL_IDLE_TIMEOUT_MS tunables; HTTPS handshake shepherd shard count tunable via CWIST_HTTPS_HS_SHARDS.
    • WAF: signatures scanned with one Aho-Corasick pass.

  6. Distribution (cef1524, 9a77fd8)

    • Homebrew tap published: brew tap c4punks/cwist && brew install c4punks/cwist/cwist (verified end-to-end on Linuxbrew). vcpkg stays an in-tree draft under packaging/vcpkg/.

Known limits (see ROADMAP.md): client-side hedging (hedgingPolicy) is not implemented — gRPC C-core does not implement it either; the channel resolver supports dns (default), ipv4, and ipv6 targets, while unix/vsock transports, grpclb/xDS policies, and perAttemptRecvTimeout remain unimplemented.

--- Patch Updates (tag refreshed) ---

  1. HTTP/2 Flow-Control Hardening — Vulnerability Defense (e121dac)

    • Integer-overflow defense: the pacing token-bucket refill now caps the idle elapsed time at the exact bucket fill time (target_window / rate) before multiplying by the rate. Previously, a connection left idle for an arbitrarily long period combined with a very large computed pacing rate could overflow the uint64 multiplication (elapsed x rate) — the same arithmetic-defect class as the classic WINDOW_UPDATE integer overflows seen in RFC 7540/9113 implementations.
    • Availability defense (self-inflicted DoS): pacing_rate_bytes_per_sec is floored at min_window x 10, so a single spiked RTT sample (retransmission burst, scheduler interference) can no longer collapse the pacing rate and throttle the connection to a crawl. The peer's flow-control windows still gate in-flight bytes; loopback/small-RTT paths stay effectively unpaced as before.
    • Oscillation defense: each BDP window retune now moves at most a 2x/0.5x step from the current target, so a micro-interval scheduling-jitter sample cannot slam the window between its floor and ceiling under bursty load; fast-link convergence stays exponential, and a zero-length interval keeps the current target instead of collapsing it to the minimum.

  2. HTTP/3 Concurrency Hardening — Vulnerability Defense (6b4b4a0, bd8ba21)

    • g_h3_dgram was a single process-wide struct shared by every HTTP/3 connection with no lock: cwist_http3_send_datagram() on one connection could free() and overwrite another connection's still-pending payload mid-flight (double-free/UAF). Replaced with a per-connection, mutex-guarded FIFO datagram queue (h3_conn_ctx_t, torn down in on_conn_closed), which also fixes a same-connection bug where a second send before the first flushed silently dropped the first payload instead of queuing it.
    • cwist_h3_free_session_ticket_key() read the SSL_CTX ex_data index without the same pthread_once gate cwist_h3_setup_session_tickets() uses to write it; a context destroyed from a different thread than the one that created it had no happens-before edge to the writer. Both functions now take the same gate.
    • The use_file_stream write loop looped pread() + lsquic_stream_write() until EAGAIN with no bound: a large file transfer on a wide-open congestion window could hold the single H3 event-loop thread in back-to-back synchronous disk reads, delaying packet I/O/ACK processing for every other connection that thread services. Capped at 4 chunks per on_write() callback, re-arming wantwrite between bursts.
    • Follow-up perf pass: removed the per-header malloc/free pair in h3_process_stream_headers() (QPACK decode — was two heap round trips per header, tens of them per request), and moved the recvmmsg() batch scratch buffers (2.1MB, previously living on the H3 event-loop thread's stack) onto the heap, removing an implicit large-stack-thread requirement.

  3. Benchmark Fairness: Spring Boot Tuned Run (93a5eb5)

    • The "tuned low-latency run" (wrk -t4 -c100 vs the main -t12 -c400 profile) was CWIST-only and published in the README as a standalone headline number, with no equivalent Spring Boot figure anywhere nearby — comparing CWIST's best case against a number Spring was never measured at, contradicting this repo's own "no framework gets a hand-tuned advantage the others do not get" claim. Spring Boot now gets an identical second boot under the identical -t4 -c100 -d10s profile, replaying the same trained AOT cache used for its main run (no extra cold-start penalty CWIST doesn't also avoid), and the README callout now shows both side by side. Axum/Gin are not yet included in this second pass (documented as a known remaining asymmetry in docs/webserver-benchmark.md).
Assets 3
Loading