Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception occurred while requesting https URL using proxy aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host #8350

Closed
1 task done
ohblue opened this issue Apr 19, 2024 · 3 comments
Labels

Comments

@ohblue
Copy link

ohblue commented Apr 19, 2024

Describe the bug

Error when requesting https site through proxy using ClientSession.get()

To Reproduce

run this demo:

from proxy import Proxy
import requests
import asyncio
import aiohttp

proxy = Proxy()

proxies = proxy.get_proxy()
print(proxies)

# use requests
res = requests.get("https://ifconfig.me/all", proxies=proxies)
print(res.text)

# use aiohttp
async def main():
    url = "https://ifconfig.me/all"
    async with aiohttp.ClientSession(connector=aiohttp.TCPConnector(ssl=False),trust_env=True) as session:
        async with session.get(url,proxy=proxies['http'],ssl=False) as resp:
            text = await resp.text()
            print(text)
asyncio.run(main())

Expected behavior

The normal return when requesting https://ifconfig.me/all is as follows:

ip_addr: *.*.*.*
remote_host: unavailable
user_agent: python-requests/2.31.0
port: 49880
language:
referer:
connection:
keep_alive:
method: GET
encoding: gzip, deflate
mime: */*
charset:
via: 1.1 google
forwarded: *.*.*.*,*.*.*.*

Logs/tracebacks

Traceback (most recent call last):
  File "/Users/xx/projects/pytool/beian/test.py", line 38, in <module>
    asyncio.run(main())
  File "/Users/xx/.pyenv/versions/3.10.13/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/xx/.pyenv/versions/3.10.13/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/Users/xx/projects/pytool/beian/test.py", line 34, in main
    async with session.get(url,proxy=_proxy['http'],ssl=False) as resp:
  File "/Users/xx/.pyenv/versions/3.10.13/lib/python3.10/site-packages/aiohttp/client.py", line 1194, in __aenter__
    self._resp = await self._coro
  File "/Users/xx/.pyenv/versions/3.10.13/lib/python3.10/site-packages/aiohttp/client.py", line 578, in _request
    conn = await self._connector.connect(
  File "/Users/xx/.pyenv/versions/3.10.13/lib/python3.10/site-packages/aiohttp/connector.py", line 544, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/Users/xx/.pyenv/versions/3.10.13/lib/python3.10/site-packages/aiohttp/connector.py", line 909, in _create_connection
    _, proto = await self._create_proxy_connection(req, traces, timeout)
  File "/Users/xx/.pyenv/versions/3.10.13/lib/python3.10/site-packages/aiohttp/connector.py", line 1357, in _create_proxy_connection
    return await self._start_tls_connection(
  File "/Users/xx/.pyenv/versions/3.10.13/lib/python3.10/site-packages/aiohttp/connector.py", line 1139, in _start_tls_connection
    raise client_error(req.connection_key, OSError(msg))
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host ifconfig.me:443 ssl:False [None]

Using the same URLs and proxies, it works fine when requesting with the requests library

ip_addr: *.*.*.*
remote_host: unavailable
user_agent: python-requests/2.31.0
port: 52302
language:
referer:
connection:
keep_alive:
method: GET
encoding: gzip, deflate
mime: */*
charset:
via: 1.1 google
forwarded: *.*.*.*,*.*.*.*


### Python Version

```console
$ python --version
Python 3.10.13

aiohttp Version

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.9.3
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author:
Author-email:
License: Apache 2
Location: /Users/chenm/.pyenv/versions/3.10.13/lib/python3.10/site-packages
Requires: aiosignal, async-timeout, attrs, frozenlist, multidict, yarl
Required-by:

multidict Version

$ python -m pip show multidict
Name: multidict
Version: 6.0.5
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /Users/chenm/.pyenv/versions/3.10.13/lib/python3.10/site-packages
Requires:
Required-by: aiohttp, yarl

yarl Version

$ python -m pip show yarl
Name: yarl
Version: 1.9.4
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /Users/chenm/.pyenv/versions/3.10.13/lib/python3.10/site-packages
Requires: idna, multidict
Required-by: aiohttp

OS

macOS and Debian

Related component

Server, Client

Additional context

No response

Code of Conduct

  • I agree to follow the aio-libs Code of Conduct
@ohblue ohblue added the bug label Apr 19, 2024
@Dreamsorcerer
Copy link
Member

Can you try in Python 3.12? It think HTTPS-in-HTTPS is only supported in Python 3.11+, so could be something to do with that.

@ohblue
Copy link
Author

ohblue commented Apr 22, 2024

Can you try in Python 3.12? It think HTTPS-in-HTTPS is only supported in Python 3.11+, so could be something to do with that.

I tried in python3.12.2, the problem remains the same

@ohblue
Copy link
Author

ohblue commented Apr 22, 2024

I tried using socks5 proxy via aiohttp_socks library and the problem solved.

@ohblue ohblue closed this as completed Apr 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants