Skip to content

http1: make header-writing tolerant to partial writes#48365

Merged
jkarneges merged 1 commit into
mainfrom
jkarneges/protocol-send-headers-partial
Jun 15, 2026
Merged

http1: make header-writing tolerant to partial writes#48365
jkarneges merged 1 commit into
mainfrom
jkarneges/protocol-send-headers-partial

Conversation

@jkarneges

@jkarneges jkarneges commented Jun 13, 2026

Copy link
Copy Markdown
Member

Recently we added support for receiving large header sections, but we still can't send large header sections. This PR is the first step in addressing that. It updates the http1 module's low-level header writing methods, send_header and send_response, to be able to recover after partial writes.

Currently, these methods take a mutable reference to an implementation of std::io::Write and return Result<(), std::io::Error>. Internally, they use write_all and the write! macro to serialize the HTTP request or response. In theory, send_header and send_response can write to anything that implements Write. In practice, they can't reliably write to a non-blocking socket, since they don't handle partial writes (write_all and write! return unrecoverable errors for anything less than a full write). This is why today we always point these methods at memory buffers. Of course, this means the amount we can write is constrained by the size of the buffers.

This PR updates these methods to be able to recover after a partial write by 1) not failing if only partial data could be written, and 2) providing a way to resume after such partial writes. This allows the methods to write to non-blocking sockets and with data of potentially unlimited size.

The APIs are tweaked slightly for client and server modes, following their existing styles:

  • For client mode, send_header (which sends an entire request header section) is updated to return a typestate for partial progress, which internally maintains a byte offset. To resume writing, the caller simply calls the method again and it picks up where it left off.
  • For server mode, send_response is updated to take an offset argument and to return an Option<usize> to indicate a partial write. The caller initially passes 0 for the offset. A return value of Some(offset) means the write was incomplete. To resume writing, the method should be called again with the previously returned offset.

Under the hood, these methods are implemented using vectored writes. This turns out to not only be the most performant approach (a single syscall for all header section elements) but it also makes resumption easier. Every call sets up the full set of slices to be written, and then write_vectored_offset is used to skip to a specific byte position to write from.

For now, the methods are still only used to write into memory buffers, meaning the output size is still limited and there is no change in behavior. However, after this, we'll look at using these methods to write directly to sockets instead of buffers.

@jkarneges jkarneges force-pushed the jkarneges/protocol-send-headers-partial branch 4 times, most recently from 977a0cb to 28d5594 Compare June 15, 2026 21:58
@jkarneges jkarneges force-pushed the jkarneges/protocol-send-headers-partial branch from 28d5594 to 9f04ab4 Compare June 15, 2026 22:01
@jkarneges jkarneges requested a review from a team June 15, 2026 22:23
@jkarneges jkarneges merged commit dc9adf9 into main Jun 15, 2026
19 checks passed
@jkarneges jkarneges deleted the jkarneges/protocol-send-headers-partial branch June 15, 2026 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants