Skip to content

Commit

Permalink
Fix test_preserve_chunked_on_retry_after flakiness (urllib3#1816)
Browse files Browse the repository at this point in the history
Without `Content-Length: 0` or `Connection: close`, the client can try
reading more data after the socket gets closed. If it does try, it's
going to fail with "Connection reset by peer".

This was happening on Windows despite the `preload_content=False`
setting. Setting those headers fixes the issue and allows omitting
`preload_content=False`.

Since I'm a good citizen and want to close the sockets, I collect them
and close them when the connection pool is closed.

(I'm also renaming the test since it's specifically about the
Retry-After header.)
  • Loading branch information
pquentin committed Mar 16, 2020
1 parent 262e41e commit 10d7d26
Showing 1 changed file with 8 additions and 5 deletions.
13 changes: 8 additions & 5 deletions test/with_dummyserver/test_chunked_transfer.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,12 +102,14 @@ def test_provides_default_host_header(self):
host_headers = [x for x in header_lines if x.startswith(b"host")]
assert len(host_headers) == 1

def test_preserve_chunked_on_retry(self):
def test_preserve_chunked_on_retry_after(self):
self.chunked_requests = 0
self.socks = []

def socket_handler(listener):
for _ in range(2):
sock = listener.accept()[0]
self.socks.append(sock)
request = consume_socket(sock)
if b"Transfer-Encoding: chunked" in request.split(b"\r\n"):
self.chunked_requests += 1
Expand All @@ -116,16 +118,17 @@ def socket_handler(listener):
b"HTTP/1.1 429 Too Many Requests\r\n"
b"Content-Type: text/plain\r\n"
b"Retry-After: 1\r\n"
b"Content-Length: 0\r\n"
b"Connection: close\r\n"
b"\r\n"
)
sock.close()

self._start_server(socket_handler)
with HTTPConnectionPool(self.host, self.port) as pool:
retries = Retry(total=1)
pool.urlopen(
"GET", "/", chunked=True, preload_content=False, retries=retries
)
pool.urlopen("GET", "/", chunked=True, retries=retries)
for sock in self.socks:
sock.close()
assert self.chunked_requests == 2

def test_preserve_chunked_on_redirect(self):
Expand Down

0 comments on commit 10d7d26

Please sign in to comment.