fix(h2): ensure every request settles - #5603
Merged
Merged
Conversation
Three ways an HTTP/2 request could stop making progress without the caller
ever hearing back:
1. A stream that closed with no response and no error had its queue slot
freed by completeRequestStream() without anyone calling
request.onResponseError(), so the request simply vanished. This is
reachable after a GOAWAY: detachRequestStreamForClose() releases the
request-facing listeners and installs `once('error', noop)`, but leaves
the 'close' listener and kRequestStreamState in place, so the abandoned
stream later completes — and splices out — the request that has since
been requeued onto another session. severRequestStream() now unbinds
such a stream for good, and completeRequestStream() errors a request
whose stream closed before the response was complete.
2. A peer that refuses a request with GOAWAY(lastStreamID = 0) had it
requeued with no attempt budget, so a peer that keeps refusing turned
one request into an unbounded connect/refuse/reconnect loop that never
settled. Refusals now count against MAX_REFUSED_ATTEMPTS.
3. A peer may advertise SETTINGS_MAX_CONCURRENT_STREAMS = 0 to refuse new
streams (RFC 9113 6.5.2). busy() then reports the client as permanently
busy, and queued requests can neither open a stream — so no per-stream
timeout covers them — nor trigger a reconnect, so the SETTINGS frame
that would lift the limit can never arrive. headersTimeout now covers a
request that cannot be sent at all, and the unusable session is dropped
so the next request gets a fresh connection.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01A49JamgF2TkZHu5h58ChUM
Signed-off-by: Matteo Collina <hello@matteocollina.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5603 +/- ##
==========================================
+ Coverage 93.49% 93.50% +0.01%
==========================================
Files 110 110
Lines 38281 38424 +143
==========================================
+ Hits 35789 35929 +140
- Misses 2492 2495 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
RST_STREAM(CANCEL) received before the response is the one reset code Node reports as a bare 'close' on the client stream -- no 'end' (unlike NO_ERROR) and no 'error' (unlike PROTOCOL_ERROR / INTERNAL_ERROR / REFUSED_STREAM) -- and destroying the stream unenrolls its timeout, so no 'timeout' follows either. That is the deterministic path into the case completeRequestStream() did not handle, and it is reachable against ordinary infrastructure: a proxy sends this upstream whenever its own downstream client goes away. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A49JamgF2TkZHu5h58ChUM Signed-off-by: Matteo Collina <hello@matteocollina.com>
ronag
approved these changes
Jul 29, 2026
7 tasks
mcollina
added a commit
that referenced
this pull request
Jul 29, 2026
Two follow-ups to the rebase onto main. RFC 9113 section 8.7 says a client SHOULD NOT automatically retry a request more than once. The REFUSED_STREAM path already followed that; the GOAWAY replay budget introduced in #5603 allowed three. Lower it to one so both refusal signals behave the same, and rename the budget after the signal it belongs to now that REFUSED_STREAM retries live in this file too: MAX_REFUSED_ATTEMPTS -> MAX_GOAWAY_REPLAY_ATTEMPTS kRefusedAttempts -> kGoAwayReplayAttempts A peer that refuses every connection now opens two connections rather than four before the request fails, so tighten the regression bound accordingly. retryRefusedStream() also open-coded the stream detach that detachRequestStreamForClose() already performs. Reuse the helper: it drops the 'close' listener rather than relying only on the nulled state, and it guards the stream-count decrement, so the abandoned attempt cannot touch the retried request. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01A49JamgF2TkZHu5h58ChUM Signed-off-by: Matteo Collina <hello@matteocollina.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three ways an HTTP/2 request could stop making progress without the caller ever hearing back. All three are reachable with default options now that
allowH2defaults totrue.1. A request could be dropped silently
completeRequestStream()runs on the stream's'close':There was no branch for "the stream closed with no response and no error". In that case the request was spliced out of the queue and nothing ever called
request.onResponseError(), so the caller's promise never settled — while the client's own stats looked perfectly healthy.The window is reachable after a GOAWAY.
detachRequestStreamForClose()→releaseRequestStream()removes the request-facing listeners and installsonce('error', noop), swallowing any later error, but leaves the'close'listener andstream[kRequestStreamState]in place. The request is requeued and re-dispatched on a fresh session, and then the old stream closes and takes the new in-flight request's queue slot with it.severRequestStream()now unbinds an abandoned stream for good, andcompleteRequestStream()errors a request whose stream closed before the response was complete.2. A refused request could be retried without bound
A peer that refuses with
GOAWAY(lastStreamID = 0)is saying "I processed nothing, retry elsewhere" (RFC 9113 §6.8) — what a draining proxy or a load balancer shedding load sends.onHttp2SessionGoAway()requeued the request and reconnected immediately, with no attempt counter and no backoff, so a peer that keeps refusing turned one request into an unbounded connect → HEADERS → GOAWAY → requeue → connect loop. Measured against a peer that always refuses: ~360 reconnects/s indefinitely, the request never settling, and unrelated origins in the same process losing ~75% of their throughput.Refusals now count against
MAX_REFUSED_ATTEMPTS, after which the request fails.3.
SETTINGS_MAX_CONCURRENT_STREAMS: 0wedged the client permanentlyA peer may advertise
0to temporarily refuse new streams (RFC 9113 §6.5.2) and raise it again later — graceful drain, overload shedding, an LB draining a backend.busy()then degenerates torunning >= 0, i.e. permanently busy, and nothing covers a queued request:headersTimeout/bodyTimeoutare armed on a stream that is never createdkSize !== 0_resume()returns atbusy(request), so no new connection is made — which means the later SETTINGS frame that would lift the limit can never arrive eitherresumeH2()also unrefs the session in this state, so the event loop can drain while a request is outstanding: the process exits with status 0 and the line after theawaitnever runs.headersTimeoutnow covers a request that cannot be sent at all — a request that never gets a stream has missed the same deadline as one whose headers never arrive — and the unusable session is dropped so the next request gets a fresh connection.Tests
Three new files, 7 tests. Each was verified to fail on a pristine
maincheckout and pass with this change:test/http2-request-never-settles.jstest/http2-goaway-refusal-storm.jstest/http2-max-concurrent-streams-zero.jstest/http2-goaway-refusal-storm.jscarries a small frame-level h2 peer because Node's http2 server clampslastStreamIDto the last stream it actually received, so it cannot send this GOAWAY.Existing suites:
test/+(http2|h2)*.js97 pass,npm run test:unit1447 pass, retry + redirect interceptors 87 pass.Not covered here
v7.xignoresheadersTimeoutentirely in the h2 path (onlybodyTimeoutarms the stream timer) — followed up separately.🤖 Generated with Claude Code
https://claude.ai/code/session_01A49JamgF2TkZHu5h58ChUM