perf(hub): gzip the SSE stream without delaying delivery - #1231
Conversation
|
Self-review turned up three defects in the first version; pushed fixes in d65056b. Cancelling the source threw. The wrapper holds a reader for the whole life of the connection, and cancelling a locked stream is invalid — in Bun it throws Reads were not gated on downstream demand. Only zlib's own buffer was consulted, and SSE compresses well enough that a slow client can be megabytes behind while the compressed queue still looks nearly empty — a test with a non-reading consumer pulled 1752 chunks before stalling. Reading now waits for
Six new tests cover these (upstream cancel propagation, no rejection on client cancel, q=0 refusal, non-zero q, wildcard, and the backpressure stall). Re-verified the property this PR exists for is intact: real Chromium |
SSE payloads are plain JSON that repeat the same field names on every event, so they compress well - measured 72-77% on real captured traffic from a hub with 15 active sessions. Compression could not simply be turned on, though. Hono's compress() middleware bails out whenever Transfer-Encoding is set, which streamSSE always sets, so mounting it is a no-op. Wrapping the body in a CompressionStream does compress, but it buffers until the stream ends - measured on a 10-event stream, every event arrived at once when the stream closed. On a connection that stays open for hours that means events never arrive at all. So drive zlib directly and issue a Z_SYNC_FLUSH after each chunk. That costs about one percentage point of ratio and keeps delivery immediate: verified in a real Chromium EventSource, first event at 13ms and each subsequent event at its own 500ms tick, with no error events. Clients that do not send Accept-Encoding: gzip keep the uncompressed stream. No event payload or timing changes.
Three defects in the first version of the SSE gzip wrapper: Cancelling the source directly threw. The wrapper holds a reader for the whole life of the connection, and cancelling a locked stream is invalid - in Bun it throws TypeError: Invalid state: ReadableStream is locked synchronously out of the cancel callback. Since SSE clients disconnect mid-stream as a matter of course, this fired on essentially every disconnect, and the upstream cancel never ran. Cancel through the reader instead, which is allowed to. Reads were not gated on downstream demand. Only zlib's own buffer was consulted, and SSE compresses well enough that a slow client can be megabytes behind while the compressed queue still looks nearly empty: a test with a non-reading consumer pulled 1752 chunks before stalling. Reading now waits for desiredSize to go positive, resumed from pull(). Accept-Encoding was matched with a substring test, so "gzip;q=0" - which means the client refuses gzip - was read as acceptance. Parse the q-value. Re-verified that none of this costs the property the change exists for: in a real Chromium EventSource the first event still arrives at 13ms and each one after it on its own 500ms tick, with no error events.
d65056b to
94fa321
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Findings
- 2 Minor findings; inline comments attached.
Summary
Review mode: initial
- Encoding negotiation is order-dependent when
gzipand*both appear. - Compressed and identity variants are not marked with
Vary: Accept-Encoding.
Testing
- Not run (automation); PR code was not executed.
- Added tests inspected; missing coverage for wildcard/specific precedence and
Varyon both variants.
HAPI Bot
| if (q && Number(q.slice(2)) === 0) { | ||
| return false | ||
| } | ||
| return true |
There was a problem hiding this comment.
[MINOR] Resolve gzip before the wildcard regardless of list order
This returns on the first matching coding, but an explicit gzip entry is more specific than *; header order does not change that precedence. Accept-Encoding: *;q=0, gzip is incorrectly sent identity, while *;q=1, gzip;q=0 is incorrectly sent gzip despite the explicit refusal. Either case can make the SSE representation unusable for a valid client. Evidence: hub/src/web/sseCompression.ts:25.
Suggested fix:
function acceptsGzip(acceptEncoding: string | undefined): boolean {
if (!acceptEncoding) return false
let wildcardQuality: number | undefined
for (const part of acceptEncoding.split(",")) {
const [rawName, ...params] = part.split(";")
const name = rawName?.trim().toLowerCase()
const q = params.map((param) => param.trim().toLowerCase()).find((param) => param.startsWith("q="))
const quality = q ? Number(q.slice(2)) : 1
if (name === "gzip") return quality > 0
if (name === "*") wildcardQuality = quality
}
return (wildcardQuality ?? 0) > 0
}| manager.unsubscribe(subscriptionId) | ||
| }) | ||
|
|
||
| return compressSseResponse(response, c.req.header('Accept-Encoding')) |
There was a problem hiding this comment.
[MINOR] Mark both negotiated representations as varying by Accept-Encoding
This response now changes representation based on Accept-Encoding, but neither the gzip nor identity path adds Vary. Cache-Control: no-cache still permits storage with revalidation, so an intermediary can select a cached representation without knowing that this request header participated in selection. Set the header before compression so both branches carry it. Evidence: hub/src/web/routes/events.ts:122.
Suggested fix:
response.headers.append("Vary", "Accept-Encoding")
return compressSseResponse(response, c.req.header("Accept-Encoding"))
2 participants
Why
SSE payloads are plain JSON that repeat the same field names on every event, so they compress well. Measured on real captured traffic from a hub with 15 active sessions:
JSON field names alone account for 38% of the raw stream.
This mostly helps clients on mobile data, weak links, or long-haul relays — it saves bytes and battery rather than fixing latency.
Why it is not just
compress()Two obvious approaches both fail, and both fail silently — the stream still looks fine, events just never show up:
Hono's
compress()middleware is a no-op here. It bails out whenTransfer-Encodingis set (hono/dist/middleware/compress/index.js:11), andstreamSSEalways setschunked. NoContent-Encodingis ever emitted.CompressionStreambuffers until the stream ends. Measured on a 10-event stream: every event arrived at once, at the moment the stream closed (5.0s). On a connection that stays open for hours, that means events never arrive.So this drives
zlibdirectly and issues aZ_SYNC_FLUSHafter each chunk. That costs about one percentage point of ratio (see table) and keeps delivery immediate.Verification
Verified in a real Chromium
EventSource, not just unit tests:Also confirmed at the socket level that the server flushes each event as it is written (first byte at 2ms, then one per 500ms tick).
Clients that do not send
Accept-Encoding: gzipkeep the uncompressed stream unchanged.Notes
No event payload or timing changes — transport only.
Testing
tsc --noEmit -p hub/tsconfig.jsoncleanhubsuite: 648 pass, 0 fail