Receiving zero byte chunks under certain conditions #3525
Description
Long story short
When running under "Red Hat Enterprise Linux Server release 7.5" the client only gets a partial result from an https URL using chunked transfer encoding.
In another ticket I found the following testcode:
import asyncio
import aiohttp
import ssl
async def main():
async with aiohttp.ClientSession() as session:
async with session.get(url=INTERNALURL, timeout=None, ssl=False) as resp:
buffer = b""
async for raw_data, end_of_http_chunk in resp.content.iter_chunks():
print('chunk received', len(raw_data))
buffer += raw_data
if not end_of_http_chunk:
continue
print("len(buffer)", len(buffer))
print(ssl.OPENSSL_VERSION)
loop = asyncio.get_event_loop()
loop.run_until_complete(main())Expected behaviour
Output of Python 3.6.6 in an up-to-date alpine docker container:
LibreSSL 2.7.4
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 8184
chunk received 5491
len(buffer) 111883
Actual behaviour
Ouput under RHEL7.5 with Python 3.6.3:
OpenSSL 1.0.1e-fips 11 Feb 2013
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 0
chunk received 8184
chunk received 5491
chunk received 0
len(buffer) 111883
The amount and position of zero byte length chunks is sometimes varying from call to call.
Of course, awaiting response.text() I get only 8184 (or a multiple) bytes of content instead of the expected 111883.
Calling the same URL over http I get varying chunk sizes, but never a zero length one before the end, so everything is good.
Using the Requests library in the same environment using the same https URL I also get the correct result.
Steps to reproduce
Unfortunately the URL is not publicly reachable. It's a webservice with many clients on different platforms (curl, wget, Java, Firefox) and noone else has the problem.
Your environment
aiohttp 3.5.3 (client)
The only obvious difference between the working/nonworking environment is the ssl library.
Is there anything else I can contribute to find the error?