Skip to content

fix(streams): strip content-encoding on proxy hop — durable-stream gzip-mislabel (#219 pt.A / epic #238 slice 0) #239

Description

@rickylabs

Slice 0 of epic #238 — the standalone streams gzip fix (anchors #219)

Ships first and independent of every AI-stack architectural decision. ~15 LOC, no new packages.

Bug

The streams proxy decompresses but does not relabel. The durable-stream engine (@durable-streams/server) gzips response bodies correctly, but only for bodies ≥ 1024 B. Deno's inner fetch inside the proxy transparently decompresses the upstream body while retaining the content-encoding: gzip header, and the proxy returns that header verbatim. Compliant decoders (browser DecompressionStream, Deno auto-decompress fetch) then try to gunzip an already-plain [{… body → TypeError: Decoding failed / Invalid gzip header.

Because it is gated on the 1024 B threshold, it presents as intermittent — which is why #219 looked random.

Root cause (single site)

plugins/streams/services/src/main.ts — the proxyHandler returns the inner fetch Response with its headers intact (return await fetch(proxyReq)), and even CORS-exposes content-encoding.

Fix

After the inner fetch, rebuild the Response stripping content-encoding, content-length, and hop-by-hop headers (the same STRIP set the consumer already applies client-side). The engine and the client are both correct; the proxy is the wrong actor.

  • NOT disabling engine compression (throws away real wire compression + leaves the latent Deno relabel bug).
  • NOT a client change.

Verification

  • curl -sD - --compressed '<front>/v1/stream/netscript/<path>' against a > 1 KB snapshot: pre-fix the header says gzip while body magic is 5b 7b ([{) not 1f 8b; post-fix no content-encoding header + valid JSON.
  • Add a > 1 KB durable-stream read case to the scaffold.runtime E2E with a compliant decoder, so this can't silently regress.

Acceptance

Deleting the three Accept-Encoding: identity workaround sites in the consumer (eis-chat chat-stream.ts, streams/[...path].ts, stream-loaders.ts, all marked netscript#219 pt.3) and having chat + shape reads work over a > 1 KB body.


Part of epic #238. Closes the framework-bug half of #219 (the durable-CHAT seam is a separate slice).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:pluginsplugins/* and plugin-core packagesbugSomething isn't workingtype:fix

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions