ValueError when redirecting to URL with incorrect schema

[berislavlopac]

Long story short

When the Client handles redirection, if the redirected URL contains an incorrect schema, it throws a ValueError:

aiohttp/aiohttp/client.py

Line 453 in ced1058

if scheme not in ('http', 'https', ''):

While this is technically correct, it is more sensible to expect some kind of more specific error.

Expected behaviour

I would expect the Client to throw an aiohttp-specific error, preferably InvalidURL.

Actual behaviour

The Client throws a generic ValueError, which is difficult to specifically handle.

Steps to reproduce

Create a redirection URL, e.g. using an URL shortener like https://tinyurl.com/, and have it redirect to any URL with an incorrect schema, for example file://foo/bar. Then, call it with aiohttp

import aiohttp, asyncio

async def get_url(url):
    async with aiohttp.ClientSession() as s, s.get(url) as resp:
        print(await resp.text())

asyncio.get_event_loop().run_until_complete(get_url('https://tinyurl.com/y9rf58n8'))

Output:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/asyncio/base_events.py", line 468, in run_until_complete
    return future.result()
  File "<stdin>", line 2, in get_url
  File "/home/berislav/.virtualenvs/foobar/lib/python3.6/site-packages/aiohttp/client.py", line 855, in __aenter__
    self._resp = await self._coro
  File "/home/berislav/.virtualenvs/foobar/lib/python3.6/site-packages/aiohttp/client.py", line 456, in _request
    'Can redirect only to http or https')
ValueError: Can redirect only to http or https

Your environment

Python 3.6.6, and the installed packages:

Package       Version
------------- -------
aiohttp       3.4.4  
async-timeout 3.0.0  
attrs         18.2.0 
chardet       3.0.4  
idna          2.7    
idna-ssl      1.1.0  
multidict     4.4.2  
pip           18.0   
setuptools    40.4.3 
wheel         0.32.0 
yarl          1.2.6 

[aio-libs-bot]

GitMate.io thinks the contributor most likely able to help you is @asvetlov.

Possibly related issues are #3149 (Raise "TooManyRedirects" when redirect to new url with different host), #499 (Url GET parameters are lost when redirecting), #824 (URL with #), #1996 ([Client] Incorrect handling of '..' in url path.), and #56 (Redirect loop).

[webknjaz]

@berislavlopac I'd like to emphasize that it is not an invalid/incorrect schema, it's just non-HTTP.
Please refer to my earlier explanation: #2479 (comment).

I however agree with you that there must be a better exception retaining all context information needed.

Ref: #2507
Ref: #2335

[berislavlopac]

Yes, that was basically my comment that we need something more informative and easier to manage upstream than ValueError. Seeing that this is essentially a duplicate of #2507 I'll close it.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new