Skip to content

Conversation

seanmonstar
Copy link
Member

This includes a custom BufReader, since the one in libstd doesn't allow
reading additional data into the buffer without consuming it. This is
required because some connections may send shorter packets, and so we
need to perform multiple reads. After each read, the contents of the
buffer are passed to httparse to see if have a valid message. If so, the
proper amount of bytes are consumed. The additional bytes are left in
the buffer since they are the beginning of the body.

The buffer in this BufReader also grows in size, compared to the libstd
which is sized once. This is because we start with a smaller buffer,
since the majority of messages will be able to include their head in a
packet or 2. Therefore, it's a wasteful performance hit to allocate the
maximum size for every message. However, some headers can be quite big,
and to allow for many of them to be set, we include a maximum size. Once
we've hit the maximum buffer size, and still haven't determined the end
of the headers, a HttpTooLargeError will be returned.

Closes #389

cc @reem @r-darwish

This includes a custom BufReader, since the one in libstd doesn't allow
reading additional data into the buffer without consuming it. This is
required because some connections may send shorter packets, and so we
need to perform multiple reads. After each read, the contents of the
buffer are passed to httparse to see if have a valid message. If so, the
proper amount of bytes are consumed. The additional bytes are left in
the buffer since they are the beginning of the body.

The buffer in this BufReader also grows in size, compared to the libstd
which is sized once. This is because we start with a smaller buffer,
since the majority of messages will be able to include their head in a
packet or 2. Therefore, it's a wasteful performance hit to allocate the
maximum size for every message. However, some headers can be quite big,
and to allow for many of them to be set, we include a maximum size. Once
we've hit the maximum buffer size, and still haven't determined the end
of the headers, a HttpTooLargeError will be returned.

Closes #389
seanmonstar added a commit that referenced this pull request Mar 28, 2015
fix(http): read more before triggering TooLargeError
@seanmonstar seanmonstar merged commit 6c31ea8 into master Mar 28, 2015
@seanmonstar seanmonstar deleted the packets branch March 28, 2015 18:37
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turning this down should be a nice perf win for short connections.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hyper client fails if the header isn't fully received in the first read

2 participants