Skip to content

fix(client): :http streaming stalls small chunks until 8KB buffer fills#102

Merged
tanmaykm merged 1 commit into
mainfrom
fix-streaming-small-chunk-stall
Jul 25, 2026
Merged

fix(client): :http streaming stalls small chunks until 8KB buffer fills#102
tanmaykm merged 1 commit into
mainfrom
fix-streaming-small-chunk-stall

Conversation

@tanmaykm

Copy link
Copy Markdown
Member

Problem

The :http backend's streaming read loop fills a fixed 8KB buffer with readbytes!(io, buf). On HTTP.jl 2.x, readbytes! on a response body blocks until the whole buffer is filled (or the body ends), so a streamed chunk smaller than 8KB is not forwarded to the consumer until enough later data accumulates. On a quiet long-lived stream — e.g. a Kubernetes watch, where a single event is a few KB and the next event may be minutes away — every event was delivered one flush behind the action that caused it, effectively an indefinite stall.

Found with a watch reaction-time probe (Kuber.jl) against a live k3s cluster: with the :http backend, MODIFIED/DELETED watch events took 30s+ — delivered only when later events flushed the buffer — while curl -N through the same kubectl proxy delivered every event in under 100ms, and the :downloads backend reacted in ~5-10ms.

Fix

Read with readavailable instead: eof(io) blocks until data is available, and readavailable then returns whatever has arrived without further blocking, on both HTTP.jl 1.x and 2.x (the same pattern HTTP.jl's own download uses). The readbytes! loop was itself a workaround for read(io, n) being unavailable on 2.x streams.

Test

Adds a regression test (test/streaming_latency_tests.jl): a raw-TCP chunked HTTP/1.1 server (so the test controls exactly when bytes hit the wire, independent of HTTP.jl server API differences between 1.x and 2.x) sends one small chunk, stalls 4s, then sends another; the first chunk must reach the consumer during the stall. Runs for both :http and :downloads backends, with a warmup request so the timing measures I/O behavior rather than JIT.

Verified:

  • unfixed code fails the test on HTTP.jl 2.5.5 (:http first chunk arrives at 4.0s, together with the second chunk); HTTP.jl 1.x and the :downloads backend already delivered promptly and keep passing
  • fixed code passes on both HTTP.jl 2.5.5 and 1.11
  • end-to-end with Kuber.jl against a live k3s cluster (20 iterations of pod create/patch/delete under a watch): :http watch reaction goes from every update/delete event missed (30s timeout) to ~6-9ms median, zero missed events

The :http backend's streaming read loop fills a fixed 8KB buffer with
`readbytes!(io, buf)`. On HTTP.jl 2.x, `readbytes!` on a response body
blocks until the whole buffer is filled (or the body ends), so a
streamed chunk smaller than 8KB is not forwarded to the consumer until
enough later data accumulates. On a quiet long-lived stream — e.g. a
Kubernetes watch, where a single event is a few KB and the next event
may be minutes away — every event was delivered one flush behind the
action that caused it, effectively an indefinite stall.

Read with `readavailable` instead: `eof(io)` blocks until data is
available, and `readavailable` then returns whatever has arrived
without further blocking, on both HTTP.jl 1.x and 2.x (the same pattern
HTTP.jl's own `download` uses). The `readbytes!` loop was itself a
workaround for `read(io, n)` being unavailable on 2.x streams.

Found with a watch reaction-time probe (Kuber.jl) against a live k3s
cluster: with the :http backend, MODIFIED/DELETED watch events took
30s+ — delivered only when later events flushed the buffer — while
`curl -N` through the same `kubectl proxy` delivered every event in
under 100ms, and the :downloads backend reacted in ~5-10ms. With the
fix, :http reacts in milliseconds as well.

Adds a regression test: a raw-TCP chunked HTTP/1.1 server sends one
small chunk, stalls, then sends another; the first chunk must reach the
consumer during the stall. Unfixed code fails this on HTTP.jl 2.x
(the first chunk arrives only together with the second); HTTP.jl 1.x
and the :downloads backend already delivered promptly and keep passing.
@tanmaykm

Copy link
Copy Markdown
Member Author

For the record, this fix was originally proposed in #101 and dropped there due to a confusion worth documenting.

In #101 I questioned the "blocks until the buffer is filled" premise by quoting the readbytes! docstring — "Read at most nb bytes from stream into b, returning the number of bytes read" — and an empirical check appeared to confirm that readbytes! on a real HTTP.Stream returns per-chunk. So the readavailable change was dropped and #101 was repurposed for the watcher busy-spin fix.

Both observations were misleading:

  • The "at most nb bytes" docstring (with its all::Bool keyword) describes Base's IOStream method only. The all keyword is not accepted by any other readbytes! method — not generic IO, not LibuvStream, and not HTTP.jl's stream types on either 1.x or 2.x — so readbytes!(io, buf; all=false) was never an available alternative here; it throws a MethodError.
  • The per-chunk behavior is real, but only on HTTP.jl 1.x (HTTP.Streams.Stream's readbytes! does a single http_unsafe_read). On HTTP.jl 2.x the client stream delegates to an internal _ConnReader whose readbytes! loops while total < target, refilling from the connection until the destination buffer is full or the body ends — which is exactly the stall this PR fixes. (Ironically, the refill itself calls readbytes!(conn, buf; all=false) on the raw TCP/TLS connection — the partial-read semantics exist at the transport layer, they're just not exposed through the stream API.)

So the empirical check in #101 validated 1.x behavior and the conclusion was wrongly extended to 2.x. The regression test added here fails on unfixed code only under HTTP.jl 2.x, matching that split.

@tanmaykm
tanmaykm requested a review from krynju July 25, 2026 11:07
@tanmaykm
tanmaykm merged commit 312e2b6 into main Jul 25, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant