"Incorrect padding" error on request.post()

[hyzyla]

Long story short

Our clients often send multipart-form/data with field decoded in base64 format. For retrieving this data we use on server await request.post(), but sometimes it failed with error

Error handling request
Traceback (most recent call last):
  File "/home/y.hyzyla/.local/lib/python3.7/site-packages/aiohttp/web_protocol.py", line 418, in start
    resp = await task
  File "/home/y.hyzyla/.local/lib/python3.7/site-packages/aiohttp/web_app.py", line 458, in _handle
    resp = await handler(request)
  File "/home/y.hyzyla/.PyCharm2019.1/config/scratches/scratch_1.py", line 4, in hello
    await request.post()
  File "/home/y.hyzyla/.local/lib/python3.7/site-packages/aiohttp/web_request.py", line 621, in post
    chunk = field.decode(chunk)
  File "/home/y.hyzyla/.local/lib/python3.7/site-packages/aiohttp/multipart.py", line 431, in decode
    data = self._decode_content_transfer(data)
  File "/home/y.hyzyla/.local/lib/python3.7/site-packages/aiohttp/multipart.py", line 452, in _decode_content_transfer
    return base64.b64decode(data)
  File "/usr/lib/python3.7/base64.py", line 87, in b64decode
    return binascii.a2b_base64(s)
binascii.Error: Incorrect padding

I dived in Request.post method and found that aiohttp reads field content using chunks, and every chunk are decoded separately from another chunk. But I did not find that when aiohttp read streaming data there is a rule for a reading chunk with a size that is correct for base64 decoding.

Expected behaviour

Expected that request.post decodes multipart-froms with base64 content without errors

Actual behaviour

Throws error about "Incorrect padding"

Steps to reproduce

from aiohttp import web

async def hello(request):
    await request.post()
    return web.json_response({})

app = web.Application()
app.add_routes([web.post('/', hello)])
web.run_app(app)

test-malformed.txt

curl \
-X POST "http://0.0.0.0:8080/" \
-H "Content-Type: multipart/form-data; boundary=0e2e504df1877f240c61646bf13a0a1e" \
--data-binary "@test-malformed.txt"

Your environment

aiohttp.server 3.5.4
Ubuntu 18.04 python3.7

[socketpair]

I see double newlines in the file, but https://tools.ietf.org/html/rfc2045#page-7:

"All line breaks or other characters not found in Table 1 must be ignored by decoding software."

base64.b64decode(b''.join(binascii.b2a_base64(b'aaa') for i in range(4)).replace(b'\n', b'\x0d\x0d\x0a'))
b'aaaaaaaaaaaa'

So, these newlines are not a problem.

[hyzyla]

Yes, I also thought that this a problem with breaks \r\r\n, but for some requests this breaks are valid.

[hyzyla]

@asvetlov, fix for this problem was written and was merged into master before 3.6.0 release. But all 3.6.x versions don't contain this fix. Could it be added to at least 3.7 release or earlier?