Skip to content

Match chunked as the final transfer coding, order-independently#2500

Merged
yhirose merged 1 commit into
masterfrom
chunked-final-coding-detection
Jul 18, 2026
Merged

Match chunked as the final transfer coding, order-independently#2500
yhirose merged 1 commit into
masterfrom
chunked-final-coding-detection

Conversation

@yhirose

@yhirose yhirose commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Based on #2487 by @metsw24-max — reworked implementation, so opening separately to make the diff reviewable.

Problem

is_chunked_transfer_encoding compared the whole Transfer-Encoding field value against "chunked", so a message whose final coding is chunked but which names another coding first ("gzip, chunked", valid under RFC 9112 §6.1 where chunked must be the final coding) was read as unframed. On a keep-alive server the body is then left in the socket and parsed as the next request (smuggling). open_stream repeated the check case-sensitively with a raw ==, desyncing the client stream the same way.

Change

  • Match the last coding token of the Transfer-Encoding value case-insensitively instead of comparing the whole field, and route open_stream through the same helper so server and streaming client agree.
  • Scan all Transfer-Encoding lines rather than only index 0. Headers is an unordered_multimap whose duplicate-key iteration order is not portable (this is what broke CI on the original PR: libstdc++ and libc++ disagree), so the final coding of a multi-line field cannot be determined reliably. In that ambiguous case we fail safe — any multi-line field that names chunked is treated as chunked. A mis-parse only closes the connection, whereas the opposite error enables smuggling.

Tests

ChunkedTransferEncodingTest covers single-coding (case-insensitive), chunked as the final coding of a list, non-final chunked, absent/empty, and order-independent multi-line cases. Builds and passes on both the header-only and split (test_split) builds.

Difference from #2487

#2487 read only the first Transfer-Encoding line via get_header_value(..., 0) and dropped its multi-line test because of the unordered_multimap ordering issue. This version handles the multiple-line case deterministically by failing safe, so no ordering assumption is needed.

🤖 Generated with Claude Code

is_chunked_transfer_encoding compared the whole Transfer-Encoding field
value against "chunked", so a message whose final coding is chunked but
which names another coding first ("gzip, chunked", valid under RFC 9112
6.1) was read as unframed. On a keep-alive server the body was then left
in the socket and parsed as a smuggled request; open_stream repeated the
check case-sensitively with a raw ==, desyncing the client stream the
same way.

Rework the helper to match the last coding token case-insensitively and
route open_stream through it so both paths agree. The codings may also be
split across multiple Transfer-Encoding lines (RFC 9110 5.3); since
Headers is an unordered_multimap whose duplicate-key iteration order is
not portable, the final coding of a multi-line field cannot be
determined reliably, so treat any such message that names chunked as
chunked (fail safe: a mis-parse only closes the connection, whereas the
opposite error enables smuggling). A unit test covers the helper,
including order-independent multi-line cases.

Based on #2487 by @metsw24-max.
@yhirose

yhirose commented Jul 18, 2026

Copy link
Copy Markdown
Owner Author

@metsw24-max thanks for sending #2487. This PR is the revised version that I made with Opus. I think this is a more comprehensive solution. Please let me know if you have any question.

@yhirose
yhirose merged commit 75938f0 into master Jul 18, 2026
43 checks passed
@yhirose
yhirose deleted the chunked-final-coding-detection branch July 21, 2026 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant