-
Notifications
You must be signed in to change notification settings - Fork 168
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Zero-length responses break connection reuse #555
Comments
njsmith
added a commit
to njsmith/ureq
that referenced
this issue
Nov 17, 2022
For empty response bodies, there's no guarantee that `PoolReturnRead::read` will ever be called, so we might never notice that the body is exhausted and can be returned to the pool. Solve this by adding an exhaustion check to `PoolReturnRead::new`. Annoyingly, this requires changing its API (adding a new trait bound + making `new` fallible), but fortunately it's an internal API and the callers are easy to update. Fixes algesten#555
This was referenced Nov 26, 2022
jsha
added a commit
that referenced
this issue
Dec 4, 2022
Introduce PoolReturner, a handle on an agent and a PoolKey that is capable of returning a Stream to a Pool. Make Streams keep track of their own PoolReturner, instead of having PoolReturnRead keep track of that information. For the LimitedRead code path, get rid of PoolReturnRead. Instead, LimitedRead is responsible for returning its Stream to the Pool after its second-to-last read. In other words, LimitedRead will return the stream if the next read is guaranteed to return Ok(0). Constructing a LimitedRead of size 0 is always wrong, because we could always just return the stream immediately. Change the size argument to NonZeroUsize to enforce that. Remove the Done trait, which was only used for LimitedRead. It was used to try and make sure we returned the stream to the pool on exact reads, but was not reliable. This does not yet move the ChunkDecoder code path away from PoolReturnRead. That requires a little more work. Part 1 of #559. Fixes #555.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Here's the code for handling responses with fixed-length bodies:
ureq/src/response.rs
Lines 325 to 343 in 28d667a
We set up the
PoolReturnRead
, and then if the body is short enough, we call.read_exact
on thePoolReturnRead
to both fetch the body and, as a side-effect, forcePoolReturnRead
to notice that the body has been consumed and that it's free to return the connection to the pool.However, if the response body is empty (e.g. on a 304 Not Modified), then
len
is 0, and we callread_exact
on a zero-length buffer. In this case, the default implementation ofstd::io::Read::read_exact
exits immediately. In particular, it does not callPoolReturnRead::read
at all, soPoolReturnRead
has no chance to notice that the stream has been exhausted. Then we drop thePoolReturnRead
, and since it believes the body hasn't been exhausted, it drops the underlying connection and on our next request we'll have to set up a new connection from scratch.(Ironically, this seems to make my program slower on a warm cache than a cold cache, because in many cases fetching the data is cheaper than restarting the connection.)
The text was updated successfully, but these errors were encountered: