v0.36.1 — request_data is a real IPC stream, and a faster request path

⚠️ request_data in the access log now decodes as specified

docs/access-log-spec.md §4.3 requires request_data to be base64 of a self-contained Arrow IPC stream — a schema message followed by a record batch message — and calls round-trip equivalence the conformance test for the field.

It wasn't. The reference emitted RecordBatch.serialize(), which writes a single encapsulated IPC message. A reader following the spec got:

OSError: Expected IPC message of type schema but got record batch

It also dropped the batch's custom_metadata, so an archived record lost vgi_rpc.method and vgi_rpc.request_version — the two fields a consumer replaying a request most needs.

Why it survived: nothing looked at the field. access_log_conformance.py never inspected its contents, and every test fixture used the placeholder "QQ==" (base64 of "A") while asserting zero violations. A port implementing to spec would have emitted a stream, disagreed with the reference, and had no test to say which was right.

Both halves are fixed:

• HTTP hands the access log the raw request body, which already is the self-contained IPC stream the spec describes — free, byte-faithful, metadata intact. Pipe and Unix have no discrete body, so they re-frame the batch properly instead, and only at DEBUG.
• access_log_conformance.py now round-trips the field, so ports are held to it. It checks decode-through-open_stream, strict base64 (RFC 4648 padding), and exactly one row of parameters — round-trip, not bytes, so a port may use whatever encoding its Arrow library produces.

If you consume this field: its content changes shape. That is the fix — it was undecodable by a spec-following reader and is now decodable. Turning the validator on immediately failed three of this repo's own tests whose fixtures asserted conformance against bytes no reader could open.

Request path is faster

From the profiling that turned up the bug:

change	effect
Request readers are pa.BufferReader, not BytesIO	_read_request 58.3 µs → 43.9 µs (server-side, 16-core)
Request batch no longer serialized on every call	removes work a default server never uses
Token zstd codecs per-thread, import at module scope	_pack_plaintext 1.04 µs → 0.79 µs

Arrow reads a BufferReader from C++; a BytesIO makes it cross back into Python on every read. Measured 26% off open_stream+read_batch at 256 B and 31% at 16 KiB, with the gap widening as bodies grow.

The per-request serialization existed for request_data, which is DEBUG-gated and discarded at INFO — so a default server, where access logging is off entirely, paid for it and threw it away. It now happens inside _emit_access_log, behind the guard that was already there.

Codecs are per-thread rather than one shared instance because python-zstandard does not promise two threads may call into one codec object at once, and this runs on a WSGI thread pool.

Benchmarks

TCP is now benchmarked. It was correctness-tested by the conformance fixtures but absent from the benchmark suite's seven transports — and it's the one whose numbers are least predictable from the others, having neither a shared page cache nor a local socket's short path. Added through a separate make_bench_conn fixture rather than by widening make_conn, so the extra parametrisation doesn't multiply across every test using that fixture.

State serialization is measured against the codec it actually takes. Flat stream state has used a compact codec since 0.32.0; the benchmarks still timed an Arrow-shaped dataclass. Both paths are now measured, because the ratio is the point:

flat state, compact codec :  1,425 ns
flat state, through Arrow : 12,556 ns   (8.8x)

---

Upgrading: no action required unless you parse request_data out of access logs — see above. No public API changes; no wire-protocol changes to the RPC surface; protocol_hash unchanged.