New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL exception when using any HTTPS proxy #1182

Closed
Miarevo opened this Issue Feb 11, 2013 · 12 comments

Comments

Projects
None yet
4 participants
@Miarevo

Miarevo commented Feb 11, 2013

When I attempt to load a page using an HTTPS proxy, I get an odd SSL exception that I never got before (with the same code). This happens with any valid HTTPS proxy. Unfortunately, I don't know when this began to occur. If I were to guess, I'd say version 1.1 introduced the issue. Here's an example:

Python 3.3.0 (default, Dec 22 2012, 21:02:07) 
[GCC 4.7.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> sess = requests.Session()
>>> r = sess.get('https://google.com/', proxies={'https': proxy})

Traceback (most recent call last):
  File "/usr/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 422, in urlopen
    body=body, headers=headers)
  File "/usr/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 274, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python3.3/http/client.py", line 1049, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.3/http/client.py", line 1087, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.3/http/client.py", line 1045, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.3/http/client.py", line 890, in _send_output
    self.send(msg)
  File "/usr/lib/python3.3/http/client.py", line 828, in send
    self.connect()
  File "/usr/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 105, in connect
    ssl_version=self.ssl_version)
  File "/usr/lib/python3.3/site-packages/requests/packages/urllib3/util.py", line 289, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/lib/python3.3/ssl.py", line 210, in wrap_socket
    _context=self)
  File "/usr/lib/python3.3/ssl.py", line 310, in __init__
    raise x
  File "/usr/lib/python3.3/ssl.py", line 306, in __init__
    self.do_handshake()
  File "/usr/lib/python3.3/ssl.py", line 513, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:547)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.3/site-packages/requests/adapters.py", line 174, in send
    timeout=timeout
  File "/usr/lib/python3.3/site-packages/requests/packages/urllib3/poolmanager.py", line 158, in urlopen
    return self.proxy_pool.urlopen(method, url, **kw)
  File "/usr/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 453, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:547)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.3/site-packages/requests/sessions.py", line 310, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3.3/site-packages/requests/sessions.py", line 279, in request
    resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
  File "/usr/lib/python3.3/site-packages/requests/sessions.py", line 374, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python3.3/site-packages/requests/adapters.py", line 213, in send
    raise SSLError(e)
requests.exceptions.SSLError: [SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:547)
@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Feb 11, 2013

Member

Which version of requests are you using? Also, can you test out the current HEAD of the master branch in this repository to see if the problem persists?

Member

sigmavirus24 commented Feb 11, 2013

Which version of requests are you using? Also, can you test out the current HEAD of the master branch in this repository to see if the problem persists?

@t-8ch

This comment has been minimized.

Show comment
Hide comment
@t-8ch

t-8ch Feb 11, 2013

Contributor

Does your proxy variable have a https:// or http:// scheme?
It should be http:// if you want to connect by http, regardless if the the proxied connection itself is encrypted.

Contributor

t-8ch commented Feb 11, 2013

Does your proxy variable have a https:// or http:// scheme?
It should be http:// if you want to connect by http, regardless if the the proxied connection itself is encrypted.

@Miarevo

This comment has been minimized.

Show comment
Hide comment
@Miarevo

Miarevo Feb 12, 2013

My proxy variable was of the format '1.1.1.1:80', and after prepending 'http://', it seems to be working fine. Thanks t-8ch! 👍 How come I never had to do that before?

Miarevo commented Feb 12, 2013

My proxy variable was of the format '1.1.1.1:80', and after prepending 'http://', it seems to be working fine. Thanks t-8ch! 👍 How come I never had to do that before?

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Feb 12, 2013

Member

@Miarevo I'm guessing it was either a change in our proxy handling or a change in urllib3's. Nice and specific eh? :-P Frankly though, @Lukasa has been more engaged in the proxy work lately so he might have an idea.

Member

sigmavirus24 commented Feb 12, 2013

@Miarevo I'm guessing it was either a change in our proxy handling or a change in urllib3's. Nice and specific eh? :-P Frankly though, @Lukasa has been more engaged in the proxy work lately so he might have an idea.

@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Feb 12, 2013

Member

@Miarevo: You did, you just didn't notice. =) If you don't provide a scheme, Requests guesses at what it should be. For HTTP proxies, it guesses it should be 'http', and for HTTPS proxies it guesses it should be 'https'.

Member

Lukasa commented Feb 12, 2013

@Miarevo: You did, you just didn't notice. =) If you don't provide a scheme, Requests guesses at what it should be. For HTTP proxies, it guesses it should be 'http', and for HTTPS proxies it guesses it should be 'https'.

@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Feb 12, 2013

Member

It should be noted that this behaviour is forward-looking in a way that it might not want to be. There is an open PR on urllib3 (urllib3/urllib3#139) which will add support for the CONNECT verb to urllib3. Unfortunately, until that PR is accepted, HTTPS proxies don't work.

Member

Lukasa commented Feb 12, 2013

It should be noted that this behaviour is forward-looking in a way that it might not want to be. There is an open PR on urllib3 (urllib3/urllib3#139) which will add support for the CONNECT verb to urllib3. Unfortunately, until that PR is accepted, HTTPS proxies don't work.

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Feb 12, 2013

Member

@Lukasa thanks for reminding me about that issue. I would love that. I could get stuff working on PythonAnywhere then. :)

Member

sigmavirus24 commented Feb 12, 2013

@Lukasa thanks for reminding me about that issue. I would love that. I could get stuff working on PythonAnywhere then. :)

@t-8ch

This comment has been minimized.

Show comment
Hide comment
@t-8ch

t-8ch Feb 13, 2013

Contributor

Wouldn't it be better to force user to specify a scheme?
As one can use http over a https proxy and vice versa.
One does not imply the other.
To quote the zen of python:

Explicit is better than implicit.

This doesn't work after all:

requests.get("httpbin.org")

Sidenote:
Requests raises a MissingSchema-exception, while RFC2616 always uses about "scheme*"

Contributor

t-8ch commented Feb 13, 2013

Wouldn't it be better to force user to specify a scheme?
As one can use http over a https proxy and vice versa.
One does not imply the other.
To quote the zen of python:

Explicit is better than implicit.

This doesn't work after all:

requests.get("httpbin.org")

Sidenote:
Requests raises a MissingSchema-exception, while RFC2616 always uses about "scheme*"

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Feb 13, 2013

Member

@t-8ch we do very little work with proxies. If I remember our code correctly, we just pass it to urllib3 and they handle all of that for us. I guess we could validate it before passing it to urllib3 though. I'll defer to @kennethreitz and @Lukasa about whether we should be doing that or not though.

Member

sigmavirus24 commented Feb 13, 2013

@t-8ch we do very little work with proxies. If I remember our code correctly, we just pass it to urllib3 and they handle all of that for us. I guess we could validate it before passing it to urllib3 though. I'll defer to @kennethreitz and @Lukasa about whether we should be doing that or not though.

@t-8ch

This comment has been minimized.

Show comment
Hide comment
@t-8ch

t-8ch Feb 13, 2013

Contributor

adapters.py:HTTPAdapter.get_connection():

if proxy:
    proxy = prepend_scheme_if_needed(proxy, urlparse(url).scheme)
    conn = ProxyManager(self.poolmanager.connection_from_url(proxy))
Contributor

t-8ch commented Feb 13, 2013

adapters.py:HTTPAdapter.get_connection():

if proxy:
    proxy = prepend_scheme_if_needed(proxy, urlparse(url).scheme)
    conn = ProxyManager(self.poolmanager.connection_from_url(proxy))
@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Feb 13, 2013

Member

I think @t-8ch is probably right. It's a quick fix, I'll open a PR and see what Kenneth thinks.

Member

Lukasa commented Feb 13, 2013

I think @t-8ch is probably right. It's a quick fix, I'll open a PR and see what Kenneth thinks.

@ghost ghost assigned Lukasa Feb 13, 2013

@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Feb 13, 2013

Member

Ugh, need to wait until #1187 is accepted. =P

Member

Lukasa commented Feb 13, 2013

Ugh, need to wait until #1187 is accepted. =P

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment