http1: make header-writing tolerant to partial writes#48365
Merged
Conversation
977a0cb to
28d5594
Compare
28d5594 to
9f04ab4
Compare
clintjedwards
approved these changes
Jun 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recently we added support for receiving large header sections, but we still can't send large header sections. This PR is the first step in addressing that. It updates the
http1module's low-level header writing methods,send_headerandsend_response, to be able to recover after partial writes.Currently, these methods take a mutable reference to an implementation of
std::io::Writeand returnResult<(), std::io::Error>. Internally, they usewrite_alland thewrite!macro to serialize the HTTP request or response. In theory,send_headerandsend_responsecan write to anything that implementsWrite. In practice, they can't reliably write to a non-blocking socket, since they don't handle partial writes (write_allandwrite!return unrecoverable errors for anything less than a full write). This is why today we always point these methods at memory buffers. Of course, this means the amount we can write is constrained by the size of the buffers.This PR updates these methods to be able to recover after a partial write by 1) not failing if only partial data could be written, and 2) providing a way to resume after such partial writes. This allows the methods to write to non-blocking sockets and with data of potentially unlimited size.
The APIs are tweaked slightly for client and server modes, following their existing styles:
send_header(which sends an entire request header section) is updated to return a typestate for partial progress, which internally maintains a byte offset. To resume writing, the caller simply calls the method again and it picks up where it left off.send_responseis updated to take an offset argument and to return anOption<usize>to indicate a partial write. The caller initially passes 0 for the offset. A return value ofSome(offset)means the write was incomplete. To resume writing, the method should be called again with the previously returned offset.Under the hood, these methods are implemented using vectored writes. This turns out to not only be the most performant approach (a single syscall for all header section elements) but it also makes resumption easier. Every call sets up the full set of slices to be written, and then
write_vectored_offsetis used to skip to a specific byte position to write from.For now, the methods are still only used to write into memory buffers, meaning the output size is still limited and there is no change in behavior. However, after this, we'll look at using these methods to write directly to sockets instead of buffers.