No access to response after receiving response and then RST_STREAM #1625
Replies: 2 comments 1 reply
-
|
Interesting. Would you be able to share a full traceback that you get at the point the exception is raised? I'd assume that from a code-flow perspective it probably occurs while the request body is still being sent in async def send_body(self, stream: AsyncByteStream, timeout: TimeoutDict) -> None:
async for data in stream:
while data:
max_flow = await self.connection.wait_for_outgoing_flow(
self.stream_id, timeout
)
chunk_size = min(len(data), max_flow)
chunk, data = data[:chunk_size], data[chunk_size:]
await self.connection.send_data(self.stream_id, chunk, timeout)
await self.connection.end_stream(self.stream_id, timeout)At the point that it's waiting for more outgoing flow in order to complete the sending of the request, it'll be receiving incoming events... async def wait_for_outgoing_flow(self, stream_id: int, timeout: TimeoutDict) -> int:
"""
Returns the maximum allowable outgoing flow for a given stream.
If the allowable flow is zero, then waits on the network until
WindowUpdated frames have increased the flow rate.
https://tools.ietf.org/html/rfc7540#section-6.9
"""
local_flow = self.h2_state.local_flow_control_window(stream_id)
connection_flow = self.h2_state.max_outbound_frame_size
flow = min(local_flow, connection_flow)
while flow == 0:
await self.receive_events(timeout) # I'd assume this is the point at which we "see" the RST_STREAM
local_flow = self.h2_state.local_flow_control_window(stream_id)
connection_flow = self.h2_state.max_outbound_frame_size
flow = min(local_flow, connection_flow)
return flowAnyways that's interesting from a POV of figuring out where in the codebase we'd want to tweak things if we wanted to handle this differently. But let's put that aside and move onto what we want the behaviour to be here. Relevant part of the HTTP/2 spec here is this...
I think there's no reason for us to want to raise an exception in this case. The server didn't require the entire request body, and did send a response. Ideally we just want to return that response. It's immaterial that the entire request body wasn't ever fully consumed. Perhaps we'll want to do something like this... await self.send_headers(method, url, headers, has_body, timeout)
if has_body:
try:
await self.send_body(stream, timeout)
except httpcore.RemoteProtocolError as exc:
if exc.reset_stream and exc.error_code == ErrorCodes.NO_ERROR:
# If we get RST_STREAM with NO_ERROR then it indicates that
# the response has been sent without fully ingesting the request body.
# We can forget about sending the remainder of the request body
# here, and move on to parsing the response.
pass
else:
raise exc
# Receive the response.
status_code, headers = await self.receive_response(timeout)
response_stream = AsyncIteratorByteStream(
aiterator=self.body_iter(timeout), aclose_func=self._response_closed
) |
Beta Was this translation helpful? Give feedback.
-
|
It also seems to me that we could easily get into a state where we don't ever see this error and just hang when sending the request... The response data is also flow controlled, so if the response is large, then on the client side we might end up stuck waiting for body request flow control (which we never get, because the server has stopped ingesting the request body) and also hanging on response flow control (which we never get, because we on the client are not yet ingesting the response body), and so get to a point where we don't see the final DATA frame that includes the RST_STREAM/NO_ERROR. Only ways around to deal with that is way more complex - restucturing the codebase so that we're performing parallel reading and writing of the request/response. |
Beta Was this translation helpful? Give feedback.
-
|
This is the backtrace with httpx==0.17.1: |
Beta Was this translation helpful? Give feedback.
-
We're using httpx for some end-to-end testing of an HTTP/2 proxy server, and we ran into an issue where httpx throws an exception on a RST_STREAM after having received the response headers and body. The sequence is as follows:
RemoteProtocolErrorexception, but the response headers/data is unavailable, despite having been received by the library.I turned on trace logging for httpx and this happens just prior to the exception being thrown:
Given that the entire POST is not received, I think it's a reasonable approach to throw an exception when the stream is reset in this way; however, I would hope to be able to inspect the response that was received on this sort of error.
By way of comparison, curl will print the response but exit with a non-zero status under these conditions. Firefox will dutifully display the response as well.
Naively (not being overly familiar with the library), I would think that a good solution to this would be to pack the received response into the exception so that it could be inspected in the handling of the exception. Any thoughts?
Beta Was this translation helpful? Give feedback.
All reactions