New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proxy behavior is inconsistent #2981

Closed
voidfyoo opened this Issue May 7, 2018 · 2 comments

Comments

Projects
None yet
2 participants
@voidfyoo

voidfyoo commented May 7, 2018

Long story short

request proxy behavior is inconsistent within one session:

import asyncio

import aiohttp


class ProxyCrawler:
    def __init__(self):
        loop = asyncio.get_event_loop()
        self.session = aiohttp.ClientSession(loop=loop)

    async def req(self):
        async with self.session.get('http://httpbin.org/ip') as resp:
            text = await resp.text()
            print('First request:')
            print(text)

        async with self.session.get('http://httpbin.org/ip', proxy='http://127.0.0.1:1087') as resp:
            text = await resp.text()
            print('Second request:')
            print(text)

        await self.session.close()


if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    c = ProxyCrawler()
    loop.run_until_complete(c.req())

Expected behaviour

The first request goes directly, and the second request goes through the specified proxy.

Actual behaviour

Both requests go directly without proxy.

( But if you set the first request go through a proxy, the second request will go through the specified proxy correctly. )

Steps to reproduce

Run the code above ( replace the proxy with yours ).

Your environment

aiohttp: 3.2.0
Mac OS X

@asvetlov

This comment has been minimized.

Member

asvetlov commented May 7, 2018

Thanks for the report.
ClientRequest.connection_key should include proxy attribute.
I see the problem with custom SSL certificate also: we need to check SSL context as well.

asvetlov added a commit that referenced this issue May 8, 2018

asvetlov added a commit that referenced this issue May 10, 2018

Don't reuse a connection with the same URL but different proxy/TLS se…
…ttings (#2985)

* Fix #2981: Don't reuse a connection with the same URL but different proxy/TLS settings

* Add TLS to white list

* Fix broken TLS over proxy

asvetlov added a commit that referenced this issue May 10, 2018

Don't reuse a connection with the same URL but different proxy/TLS se…
…ttings (#2985)

* Fix #2981: Don't reuse a connection with the same URL but different proxy/TLS settings

* Add TLS to white list

* Fix broken TLS over proxy
@asvetlov

This comment has been minimized.

Member

asvetlov commented May 10, 2018

Fixed in aiohttp 3.2.1 release

kornicameister added a commit to kornicameister/korni-stats-collector that referenced this issue Jun 7, 2018

Update aiohttp to 3.3.1 (#78)
This PR updates [aiohttp](https://pypi.org/project/aiohttp) from **3.2.1** to **3.3.1**.



<details>
  <summary>Changelog</summary>
  
  
   ### 3.3.0
   ```
   ==================

Features
--------

- Raise ``ConnectionResetError`` instead of ``CancelledError`` on trying to
  write to a closed stream. (`2499 &lt;https://github.com/aio-libs/aiohttp/pull/2499&gt;`_)
- Implement ``ClientTimeout`` class and support socket read timeout. (`2768 &lt;https://github.com/aio-libs/aiohttp/pull/2768&gt;`_)
- Enable logging when ``aiohttp.web`` is used as a program (`2956 &lt;https://github.com/aio-libs/aiohttp/pull/2956&gt;`_)
- Add canonical property to resources (`2968 &lt;https://github.com/aio-libs/aiohttp/pull/2968&gt;`_)
- Forbid reading response BODY after release (`2983 &lt;https://github.com/aio-libs/aiohttp/pull/2983&gt;`_)
- Implement base protocol class to avoid a dependency from internal
  ``asyncio.streams.FlowControlMixin`` (`2986 &lt;https://github.com/aio-libs/aiohttp/pull/2986&gt;`_)
- Cythonize ``helpers.reify``, 5% boost on macro benchmark (`2995 &lt;https://github.com/aio-libs/aiohttp/pull/2995&gt;`_)
- Optimize HTTP parser (`3015 &lt;https://github.com/aio-libs/aiohttp/pull/3015&gt;`_)
- Implement ``runner.addresses`` property. (`3036 &lt;https://github.com/aio-libs/aiohttp/pull/3036&gt;`_)
- Use ``bytearray`` instead of a list of ``bytes`` in websocket reader. It
  improves websocket message reading a little. (`3039 &lt;https://github.com/aio-libs/aiohttp/pull/3039&gt;`_)
- Remove heartbeat on closing connection on keepalive timeout. The used hack
  violates HTTP protocol. (`3041 &lt;https://github.com/aio-libs/aiohttp/pull/3041&gt;`_)
- Limit websocket message size on reading to 4 MB by default. (`3045 &lt;https://github.com/aio-libs/aiohttp/pull/3045&gt;`_)


Bugfixes
--------

- Don&#39;t reuse a connection with the same URL but different proxy/TLS settings
  (`2981 &lt;https://github.com/aio-libs/aiohttp/pull/2981&gt;`_)
- When parsing the Forwarded header, the optional port number is now preserved.
  (`3009 &lt;https://github.com/aio-libs/aiohttp/pull/3009&gt;`_)


Improved Documentation
----------------------

- Make Change Log more visible in docs (`3029 &lt;https://github.com/aio-libs/aiohttp/pull/3029&gt;`_)
- Make style and grammar improvements on the FAQ page. (`3030 &lt;https://github.com/aio-libs/aiohttp/pull/3030&gt;`_)
- Document that signal handlers should be async functions since aiohttp 3.0
  (`3032 &lt;https://github.com/aio-libs/aiohttp/pull/3032&gt;`_)


Deprecations and Removals
-------------------------

- Deprecate custom application&#39;s router. (`3021 &lt;https://github.com/aio-libs/aiohttp/pull/3021&gt;`_)


Misc
----

- 3008, 3011
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/aiohttp
  - Changelog: https://pyup.io/changelogs/aiohttp/
  - Repo: https://github.com/aio-libs/aiohttp
</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment