Skip to content

net.http: stream response callbacks and stop limits over HTTP/2#27369

Merged
JalonSolov merged 2 commits into
vlang:masterfrom
quaesitor-scientiam:net-http2-streaming
Jun 7, 2026
Merged

net.http: stream response callbacks and stop limits over HTTP/2#27369
JalonSolov merged 2 commits into
vlang:masterfrom
quaesitor-scientiam:net-http2-streaming

Conversation

@quaesitor-scientiam

Copy link
Copy Markdown
Contributor

What

Adds real streaming-response support to the HTTP/2 fetch path. Previously, the
HTTP/2 shim (#27362) buffered the entire response body, so requests using
`on_progress` / `on_progress_body` / `stop_copying_limit` /
`stop_receiving_limit` were forced onto HTTP/1.1 to preserve their semantics.
This PR makes them work over HTTP/2 too.

Closes #27368.

fn on_body(r &http.Request, chunk []u8, body_so_far u64, body_expected u64, status int) ! {
    eprintln('chunk ${chunk.len} bytes, total ${body_so_far}')
}

resp := http.fetch(
    url: 'https://example.com/large'
    enable_http2: true
    on_progress_body: on_body  // now fires per DATA frame on h2
)!

Changes

File Change
h2_conn.v new H2DataFn type; H2ClientRequest gains on_data, stop_copying_limit, stop_receiving_limit; read_response honors them while reading DATA frames
backend.c.v h2_do adapts the request's on_progress / on_progress_body into a single H2DataFn closure and threads the stop limits through; removes the streaming gate so enable_http2 is no longer overridden
h2_client.v drops the now-unused uses_response_streaming helper
request.v, http.v updated enable_http2 doc — streaming callbacks now work on h2; on_progress fires per DATA payload (rather than per raw network read)
h2_conn_test.v new streaming tests over the in-memory transport
h2_client_test.v removes the obsolete test_uses_response_streaming

Design

  • H2ClientRequest carries the streaming options so H2Conn stays
    decoupled from net.http.Request. The shim layer adapts Request's callbacks
    onto them.
  • on_data fires per DATA frame, with (chunk, body_so_far, body_expected, status). body_so_far is cumulative (including this chunk), body_expected
    is Content-Length if present (else 0), and status is the response status
    (known since headers arrive before DATA).
  • stop_copying_limit: caps the cumulative body bytes copied into
    resp.body; further chunks are dropped from the body but the callback still
    fires, matching the HTTP/1.1 behaviour.
  • stop_receiving_limit: breaks the read loop once that many body bytes
    have been received.
  • on_progress on HTTP/2 fires per DATA payload, not per raw network read —
    documented on the field. This is the closest faithful mapping since HTTP/2's
    wire is framed and encrypted.

Tests

  • test_h2_on_data_fires_per_chunk — asserts per-DATA-frame callback delivery
    with cumulative body_so_far, Content-Length, and status.
  • test_h2_stop_copying_limit_caps_body_but_keeps_callback — body capped at
    the limit; callbacks fire for every chunk; body_so_far reports true totals.
  • test_h2_stop_receiving_limit_breaks_early — loop breaks after the limit;
    later chunks are not delivered.

End-to-end verified against `https://www.google.com/\`:

chunk 1: 16384 bytes, body_so_far=16384, expected=0, status=200
chunk 2: 16384 bytes, body_so_far=32768, expected=0, status=200
chunk 3: 16384 bytes, body_so_far=49152, expected=0, status=200
version: HTTP/2.0  status: 200  body: 80371 bytes  callbacks fired: 5
last body_so_far=80371

`./vnew -W -cstrict -cc clang test vlib/net/http/h2_conn_test.v
vlib/net/http/h2_client_test.v` passes; the full `vlib/net/http` suite is
green (1 Windows-only skip).

🤖 Generated with Claude Code

vlang#27368)

The HTTP/2 fetch path (vlang#27362) buffered the entire response body, so requests
using on_progress / on_progress_body / stop_copying_limit / stop_receiving_limit
were forced onto HTTP/1.1. This adds real streaming support so they work on the
HTTP/2 path too.

- New H2ClientRequest fields: on_data (per-DATA-frame callback) and
  stop_copying_limit / stop_receiving_limit, mirroring the HTTP/1.1 semantics.
- H2Conn.read_response now tracks cumulative body bytes, reads Content-Length
  if present, fires on_data per DATA frame (including chunk, cumulative
  body_so_far, content-length, and status), respects stop_copying_limit
  (caps the response body while still firing callbacks and draining the
  stream), and respects stop_receiving_limit (breaks the read loop early).
- The h2_do shim in backend.c.v adapts the Request's on_progress and
  on_progress_body into a single H2DataFn closure and threads the two stop
  limits through. The previous gate (uses_response_streaming) is removed, and
  the enable_http2 docs note that on_progress fires per DATA payload on h2
  rather than per raw network read.

Tests over the in-memory transport assert: on_data fires per DATA frame with
cumulative body_so_far, Content-Length (when present), and status;
stop_copying_limit caps the response body while callbacks keep firing across
all chunks; stop_receiving_limit breaks the loop early. Verified end-to-end
against https://www.google.com/ — http.fetch(enable_http2: true,
on_progress_body: f) reports HTTP/2.0, status 200, and the on_progress_body
callback fires once per 16 KiB DATA frame with cumulative bytes matching the
final body length. Passes under -W -cstrict -cc clang.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e7ae53cd54

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread vlib/net/http/h2_conn.v
…ation

When stop_receiving_limit triggered, the response stream was left open without
sending RST_STREAM. On a reused H2Conn the peer's in-flight DATA frames for the
abandoned stream would still arrive, consuming the connection-level receive
window and risking starvation of subsequent requests.

Fix: when bailing early, send RST_STREAM with error code CANCEL on the request
stream (RFC 7540 Section 8.1.4 / 5.4.2) so the peer stops sending more DATA,
and set a new H2Conn.aborted flag so subsequent H2Conn.do() calls return a
clear error rather than proceeding on a half-drained connection.

Strengthens the stop_receiving_limit test to assert the client emitted
RST_STREAM(CANCEL) on the request stream and that a second do() on the same
connection errors out.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@JalonSolov JalonSolov merged commit 94a763e into vlang:master Jun 7, 2026
76 of 83 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

net.http: support streaming response callbacks and stop limits over HTTP/2

2 participants