Line too long error when proxying a request

[arthurdarcet]

When sending a request with a body set to a StreamReader (ie when your aiohttp server proxy a request to another server), the body could contain very long lines.

Such line go over the high water mark, and trigger an exception.
The payload module should not write the StreamReader by (async)-iterating over the lines, but should instead iterate over whatever is available. This patch implements that.

Here is a small script reproducing the error:

import aiohttp
import asyncio


async def handle(request):
	return aiohttp.web.Response(body=b'0' * 2 ** 18)

async def main(loop):
	app = aiohttp.web.Application()
	app.add_routes([aiohttp.web.get('/', handle)])
	handler = app.make_handler()
	await loop.create_server(handler, '0.0.0.0', 1111)

	async with aiohttp.ClientSession() as session:
		async with session.get('http://0.0.0.0:1111') as resp:
			# this is just a stream of 2**18 0-bytes
			stream = resp.content

			async with session.post('http://httpbin.org/anything', data=stream):
				pass


loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))

[codecov-io]

Codecov Report

Merging #3054 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #3054      +/-   ##
==========================================
+ Coverage   98.11%   98.11%   +<.01%     
==========================================
  Files          42       42              
  Lines        7735     7739       +4     
  Branches     1348     1348              
==========================================
+ Hits         7589     7593       +4     
  Misses         51       51              
  Partials       95       95

Impacted Files	Coverage Δ
aiohttp/payload.py	98.97% <100%> (+0.02%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 466cb4a...5106d36. Read the comment docs.

[asvetlov]

Thanks!

[arthurdarcet]

Thank you for the release!

[asvetlov]

Welcome!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new