net.http: stream response callbacks and stop limits over HTTP/2#27369
Merged
Conversation
vlang#27368) The HTTP/2 fetch path (vlang#27362) buffered the entire response body, so requests using on_progress / on_progress_body / stop_copying_limit / stop_receiving_limit were forced onto HTTP/1.1. This adds real streaming support so they work on the HTTP/2 path too. - New H2ClientRequest fields: on_data (per-DATA-frame callback) and stop_copying_limit / stop_receiving_limit, mirroring the HTTP/1.1 semantics. - H2Conn.read_response now tracks cumulative body bytes, reads Content-Length if present, fires on_data per DATA frame (including chunk, cumulative body_so_far, content-length, and status), respects stop_copying_limit (caps the response body while still firing callbacks and draining the stream), and respects stop_receiving_limit (breaks the read loop early). - The h2_do shim in backend.c.v adapts the Request's on_progress and on_progress_body into a single H2DataFn closure and threads the two stop limits through. The previous gate (uses_response_streaming) is removed, and the enable_http2 docs note that on_progress fires per DATA payload on h2 rather than per raw network read. Tests over the in-memory transport assert: on_data fires per DATA frame with cumulative body_so_far, Content-Length (when present), and status; stop_copying_limit caps the response body while callbacks keep firing across all chunks; stop_receiving_limit breaks the loop early. Verified end-to-end against https://www.google.com/ — http.fetch(enable_http2: true, on_progress_body: f) reports HTTP/2.0, status 200, and the on_progress_body callback fires once per 16 KiB DATA frame with cumulative bytes matching the final body length. Passes under -W -cstrict -cc clang. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e7ae53cd54
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…ation When stop_receiving_limit triggered, the response stream was left open without sending RST_STREAM. On a reused H2Conn the peer's in-flight DATA frames for the abandoned stream would still arrive, consuming the connection-level receive window and risking starvation of subsequent requests. Fix: when bailing early, send RST_STREAM with error code CANCEL on the request stream (RFC 7540 Section 8.1.4 / 5.4.2) so the peer stops sending more DATA, and set a new H2Conn.aborted flag so subsequent H2Conn.do() calls return a clear error rather than proceeding on a half-drained connection. Strengthens the stop_receiving_limit test to assert the client emitted RST_STREAM(CANCEL) on the request stream and that a second do() on the same connection errors out. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds real streaming-response support to the HTTP/2 fetch path. Previously, the
HTTP/2 shim (#27362) buffered the entire response body, so requests using
`on_progress` / `on_progress_body` / `stop_copying_limit` /
`stop_receiving_limit` were forced onto HTTP/1.1 to preserve their semantics.
This PR makes them work over HTTP/2 too.
Closes #27368.
Changes
h2_conn.vH2DataFntype;H2ClientRequestgainson_data,stop_copying_limit,stop_receiving_limit;read_responsehonors them while reading DATA framesbackend.c.vh2_doadapts the request'son_progress/on_progress_bodyinto a singleH2DataFnclosure and threads the stop limits through; removes the streaming gate soenable_http2is no longer overriddenh2_client.vuses_response_streaminghelperrequest.v,http.venable_http2doc — streaming callbacks now work on h2;on_progressfires per DATA payload (rather than per raw network read)h2_conn_test.vh2_client_test.vtest_uses_response_streamingDesign
H2ClientRequestcarries the streaming options soH2Connstaysdecoupled from
net.http.Request. The shim layer adaptsRequest's callbacksonto them.
on_datafires per DATA frame, with(chunk, body_so_far, body_expected, status).body_so_faris cumulative (including this chunk),body_expectedis
Content-Lengthif present (else 0), andstatusis the response status(known since headers arrive before DATA).
stop_copying_limit: caps the cumulative body bytes copied intoresp.body; further chunks are dropped from the body but the callback stillfires, matching the HTTP/1.1 behaviour.
stop_receiving_limit: breaks the read loop once that many body byteshave been received.
on_progresson HTTP/2 fires per DATA payload, not per raw network read —documented on the field. This is the closest faithful mapping since HTTP/2's
wire is framed and encrypted.
Tests
test_h2_on_data_fires_per_chunk— asserts per-DATA-frame callback deliverywith cumulative
body_so_far,Content-Length, andstatus.test_h2_stop_copying_limit_caps_body_but_keeps_callback— body capped atthe limit; callbacks fire for every chunk;
body_so_farreports true totals.test_h2_stop_receiving_limit_breaks_early— loop breaks after the limit;later chunks are not delivered.
End-to-end verified against `https://www.google.com/\`:
`./vnew -W -cstrict -cc clang test vlib/net/http/h2_conn_test.v
vlib/net/http/h2_client_test.v` passes; the full `vlib/net/http` suite is
green (1 Windows-only skip).
🤖 Generated with Claude Code