Implement proxy_form_env() function

[Fuyukai]

Long story short

Custom keyword arguments are not provided to the request class constructor by the _request method on the client session. This prevents certain arguments from being provided (e.g. proxy_from_env) as there's no arguments on the request method for them.

Expected behaviour

They would be passed.

Actual behaviour

They aren't.

Steps to reproduce

async def something():
    async with aiohttp.ClientSession() as sess:
        async with sess.get("https://httpbin.org/ip", proxy_from_env=True) as r:
            return (await r.json())

TypeError: _request() got an unexpected keyword argument **'proxy_from_env'**

Your environment

aiohttp 2.2.0 on Python 3.5.3

[asvetlov]

Please don't do it.
If you need to get proxy settings from environment variables -- use explicit function:

def proxy_from_env():
    proxy, proxy_auth = _parse(os.environ)
    return {'proxy': proxy, 'proxy_auth': proxy_auth}

async def something():
    async with aiohttp.ClientSession() as sess:
        async with sess.get("https://httpbin.org/ip", **proxy_from_env()) as r:
            return (await r.json())

BTW if you would to make a PR for adding proxy_from_env() function to aiohttp -- you are welcome.

[fafhrd91]

we load proxy info from env already. https://github.com/aio-libs/aiohttp/blob/master/aiohttp/client_reqrep.py#L298

[asvetlov]

Ooops, I've missed this.
So what is the issue?

[Fuyukai]

we load proxy info from env already.

Only if proxy_from_env=True is passed to ClientRequest - which it can't be, because there's no way to pass extra keyword arguments through _request.

[asvetlov]

I see. The feature is not completed.
@fafhrd91 maybe it's time to remove it and replace with proxy_from_env() function?
ClientRequest is in gray zone anyway, we shouldn't support backward compatibility for it.

[fafhrd91]

+1

[GDICommander]

Since the feature is not complete, it would be useful for current users of aiohttp to remove the mention of proxy_from_env from the Proxy support section of this page: http://aiohttp.readthedocs.io/en/stable/client.html#proxy-support

[asvetlov]

Unfortunately my proposal doesn't work. There are two proxies: HTTP and HTTPS, proxy_from_env should either return a dict with both http/https keys or accept desired schema name.
First solution requires a new param to client api, second is very unconvenient.

Thus let's add bool proxy_from_env param to client.request() and others as mentioned in doc and intended by first incomplete implementation.

[asvetlov]

Actually it should be parameter for ClientSession() constructor, if user need pass different proxy settings for different request -- hi should do it explicitly.