aiohttp VS requests proxy from environment variables difference #1791
Closed
Description
Long story short
I know the problem was already shortly discussed in the issues #529 and #102 but I would like to present it again as I believe the discussion was not satisfying for people who would assume aiohttp would work as similar as possible to the requests library
Expected behaviour
I expect aiohttp to work as similar as possible to requests. This is not the case for proxies from environment variables
Actual behaviour
proxy info from environment variables is not used.
Steps to reproduce
The following code will work differently on a machine with proxy and without proxy. The aiohttp code will just raise TimeoutError permanently...
import aiohttp
import asyncio
import requests
async def main():
params = {'language': 'DE', 'address': 'Yorckstr. 1, Berlin'}
url = 'https://maps.googleapis.com/maps/api/geocode/json'
timeout = 5.0
# The first synchronous request works the same with or without proxy
with requests.Session() as session:
location = session.get(url=url, params=params, timeout=timeout)
print(location)
# The asynchronous aiohttp request works properly without proxy
# but times out on a machine requiring a proxy and having it in an environment variable
async with aiohttp.ClientSession() as session:
location = await session.get(url=url, params=params, timeout=timeout)
print(location)
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Your environment
As well OSX as Linux
Suggestions
- Similarly to requests (https://github.com/kennethreitz/requests/blob/master/requests/compat.py#L57), we could import the
getproxies()(https://docs.python.org/3/library/urllib.request.html#urllib.request.getproxies) function from urllib.requests. Although people expressed concerns about it, it would be similar behaviour - If not, it should be stated explicitely in the docs that the behaviour towards environment variables is different from requests!
- We could create a
session_from_env()method. But I would then also tend to update the_request()method to be able to takeproxy_from_env=Falseas kwargs, in order to give enough flexibility to the user.
I'm willing to work on these ones, just tell me where the road goes!