Allow specifying the ssl protocol version to use #606

Closed
mvantellingen opened this Issue May 11, 2012 · 19 comments

Projects

None yet
@mvantellingen

Allow specifying the ssh protocol version (sslv2 or sslv3). This should fix problems connecting to various https sites (e.g. *.sharepoint.com)

See http://bugs.python.org/issue11220 for more information

@stantonk

Wouldn't it make more sense for the library to just handle whichever ssl protocol version is in use by the server? Having to manually specify is really klunky.

@kennethreitz
Owner

A web browser doesn't need you to specify what version of SSL a site's using, why should requests?

@mvantellingen

I agree with the whole automatic selection of the SSL version. But afaik recent openssl versions are broken in this regard and are not planning to fix it. See also http://trac.macports.org/ticket/33715

@davidfischer
Contributor

SSLv2 is not enabled by default by the major browser vendors because it is broken. Automatic selection of the SSL/TLS version is fine as long as that version is not SSLv2. See http://en.wikipedia.org/wiki/Transport_Layer_Security#Security

@kennethreitz
Owner

Perhaps we should attempt a fallback?

@davidfischer
Contributor

I don't think this issue is actually related to SSLv2 at all. Browsers don't support SSLv2 and as far as I can tell https://*.sharepoint.com is using TLSv1. I think we need more information from the bug reporter in order to continue such as the specific version of openssl in question and what protocol is being used.

% openssl version
% openssl s_client -connect subdomain.sharepoint.com:443

The output should contain something along the lines of Protocol : TLSv1.

A specific URL that others can try that triggers the problem would also be quite helpful.

@gregakespret

Is there a workaround for this? I am having the same issue.

import requests
requests.get('https://fed.princeton.edu')
Traceback (most recent call last):
File "<pyshell#6>", line 1, in
requests.get('https://fed.princeton.edu')
File "C:\Python32\lib\site-packages\requests-0.13.2-py3.2.egg\requests\api.py", line 54, in get
return request('get', url, *_kwargs)
File "C:\Python32\lib\site-packages\requests-0.13.2-py3.2.egg\requests\safe_mode.py", line 37, in wrapped
return function(method, url, *_kwargs)
File "C:\Python32\lib\site-packages\requests-0.13.2-py3.2.egg\requests\api.py", line 42, in request
return s.request(method=method, url=url, **kwargs)
File "C:\Python32\lib\site-packages\requests-0.13.2-py3.2.egg\requests\sessions.py", line 230, in request
r.send(prefetch=prefetch)
File "C:\Python32\lib\site-packages\requests-0.13.2-py3.2.egg\requests\models.py", line 605, in send
raise SSLError(e)
requests.exceptions.SSLError: [Errno 1] _ssl.c:392: error:140773F2:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert unexpected message

Apparently, the issue lies with the page enforcing only SSLv3 and urllib fails. Another verification with urrlib:

import urllib.request
urllib.request.urlopen('https://fed.princeton.edu')
Traceback (most recent call last):
File "<pyshell#4>", line 1, in
urllib.request.urlopen('https://fed.princeton.edu')
File "C:\Python32\lib\urllib\request.py", line 138, in urlopen
return opener.open(url, data, timeout)
File "C:\Python32\lib\urllib\request.py", line 369, in open
response = self._open(req, data)
File "C:\Python32\lib\urllib\request.py", line 387, in _open
'_open', req)
File "C:\Python32\lib\urllib\request.py", line 347, in _call_chain
result = func(*args)
File "C:\Python32\lib\urllib\request.py", line 1171, in https_open
context=self._context, check_hostname=self._check_hostname)
File "C:\Python32\lib\urllib\request.py", line 1138, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 1] _ssl.c:392: error:140773F2:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert unexpected message>

Some references:

http://stackoverflow.com/questions/9835506/urllib-urlopen-works-on-sslv3-urls-with-python-2-6-6-on-1-machine-but-not-wit
http://bugs.python.org/issue11220

By using a urllib.install_opener trick from http://bugs.python.org/issue11220 I am able to get the object without error with urllib.

import urllib.request
import ssl
https_sslv3_handler = urllib.request.HTTPSHandler(context=ssl.SSLContext(ssl.PROTOCOL_SSLv3))
opener = urllib.request.build_opener(https_sslv3_handler)
urllib.request.install_opener(opener)
urllib.request.urlopen('https://fed.princeton.edu')
<http.client.HTTPResponse object at 0x0000000003457390>

@ryanmark

I got around this issue by adjusting a line in my ssl.py file and adding it to my virtualenv:

$ cp /usr/lib/python2.7/ssl.py $VIRTUAL_ENV/lib/python2.7
$ vim $VIRTUAL_ENV/lib/python2.7/ssl.py

on line 372 change PROTOCOL_SSLv23 to PROTOCOL_SSLv3

Obviously this might have some crazy side effects, but it's contained to my virtual env for the bits of code that need it.

@kennethreitz kennethreitz reopened this Jul 27, 2012
@kennethreitz
Owner

Closing for now.

@von
von commented Aug 25, 2012

This issue is biting me trying to access https://www.torproject.org/projects/torbrowser.html.en

@davidfischer
Contributor

@von
What version of openssl are you using? Is your connection using TLSv1?

I do not get the same problem when accessing torproject.org:

import requests
requests.get('https://www.torproject.org/projects/torbrowser.html.en')
< Response [200] >

This issue must be related to your network setup or the version of openssl you are using.

@von
von commented Aug 26, 2012

@davidfischer > What version of openssl are you using? Is your connection using TLSv1?

$ openssl version
OpenSSL 0.9.8r 8 Feb 2011

EDITED to correct:

ssl.OPENSSL_VERSION
'OpenSSL 0.9.7l 28 Sep 2006'

$ python -V
Python 2.7.1

Looks like TLSv1 from what I can tell...
$ openssl s_client -connect www.torproject.org:443 | grep Protocol
depth=2 /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert High Assurance EV Root CA
verify error:num=20:unable to get local issuer certificate
verify return:0
Protocol : TLSv1
...

And to verify:

import requests
requests.get('https://www.torproject.org/projects/torbrowser.html.en')
Traceback (most recent call last):
File "", line 1, in
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-0.13.9-py2.7.egg/requests/api.py", line 65, in get
return request('get', url, *_kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-0.13.9-py2.7.egg/requests/safe_mode.py", line 39, in wrapped
return function(method, url, *_kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-0.13.9-py2.7.egg/requests/api.py", line 51, in request
return session.request(method=method, url=url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-0.13.9-py2.7.egg/requests/sessions.py", line 252, in request
r.send(prefetch=prefetch)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests-0.13.9-py2.7.egg/requests/models.py", line 632, in send
raise SSLError(e)
requests.exceptions.SSLError: [Errno 1] _ssl.c:499: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

@von
von commented Aug 26, 2012

@davidfischer Thanks for the pointer about the old version of openssl, updating to 2.7.3 with the latest Mac installer fixed it.

$ python
Python 2.7.3 (v2.7.3:70274d53c1dd, Apr  9 2012, 20:52:43)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 0.9.8r 8 Feb 2011'
>>> import requests
>>> requests.get('https://www.torproject.org/projects/torbrowser.html.en')
<Response [200]>
@davidfischer
Contributor

@von
Good point about the version of openssl Python is using, not just the openssl version on the system. Glad we got it resolved.

@idan
Contributor
idan commented Aug 28, 2012

For posterity, it sounds like this is solved in OpenSSL 1.0.1a

c.f. http://trac.macports.org/ticket/33715

@berdario

FWIW, this is not solved with OpenSSL 1.0.1a (1.0.1f still shows the problem)

>>> import requests
>>> requests.get('https://www.u-gov.sssup.it')
Traceback (most recent call last):
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 480, in urlopen
    body=body, headers=headers)
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 285, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/nix/store/xiqf99vr3b3fhzqfsdvx0aharx1h5s96-python3-3.3.3/lib/python3.3/http/client.py", line 1065, in request
    self._send_request(method, url, body, headers)
  File "/nix/store/xiqf99vr3b3fhzqfsdvx0aharx1h5s96-python3-3.3.3/lib/python3.3/http/client.py", line 1103, in _send_request
    self.endheaders(body)
  File "/nix/store/xiqf99vr3b3fhzqfsdvx0aharx1h5s96-python3-3.3.3/lib/python3.3/http/client.py", line 1061, in endheaders
    self._send_output(message_body)
  File "/nix/store/xiqf99vr3b3fhzqfsdvx0aharx1h5s96-python3-3.3.3/lib/python3.3/http/client.py", line 906, in _send_output
    self.send(msg)
  File "/nix/store/xiqf99vr3b3fhzqfsdvx0aharx1h5s96-python3-3.3.3/lib/python3.3/http/client.py", line 844, in send
    self.connect()
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/packages/urllib3/connection.py", line 164, in connect
    ssl_version=resolved_ssl_version)
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/packages/urllib3/util.py", line 639, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/nix/store/xiqf99vr3b3fhzqfsdvx0aharx1h5s96-python3-3.3.3/lib/python3.3/ssl.py", line 245, in wrap_socket
    _context=self)
  File "/nix/store/xiqf99vr3b3fhzqfsdvx0aharx1h5s96-python3-3.3.3/lib/python3.3/ssl.py", line 345, in __init__
    raise x
  File "/nix/store/xiqf99vr3b3fhzqfsdvx0aharx1h5s96-python3-3.3.3/lib/python3.3/ssl.py", line 341, in __init__
    self.do_handshake()
  File "/nix/store/xiqf99vr3b3fhzqfsdvx0aharx1h5s96-python3-3.3.3/lib/python3.3/ssl.py", line 548, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL] unknown error (_ssl.c:550)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/adapters.py", line 330, in send
    timeout=timeout
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/packages/urllib3/connectionpool.py", line 504, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL] unknown error (_ssl.c:550)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/sessions.py", line 383, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/sessions.py", line 486, in send
    r = adapter.send(request, **kwargs)
  File "/home/dario/.local/share/virtualenvs/6e75bebf401c6c9/lib/python3.3/site-packages/requests/adapters.py", line 385, in send
    raise SSLError(e)
requests.exceptions.SSLError: [SSL] unknown error (_ssl.c:550)
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.0.1f 6 Jan 2014'
@Lukasa
Collaborator
Lukasa commented Mar 24, 2014

If you are encountering problems I encourage you to use the SSLAdapter from the requests-toolbelt. This allows you to specify an SSL version to use for a specific site or set of sites.

@berdario

Thanks

I had already solved it by installing a custom opener with urllib, but I was looking into requests hoping that it offered a less cumbersome way to do it.

It's a pity that it's not shipped in requests itself, but the toolbelt is what I was looking for... thanks!

@chnrxn
chnrxn commented Jun 11, 2014

Here's a monkey patching variant of the solution courtesy of ryanmark's solution above, which should work for all Python SSL clients.

import ssl
from functools import wraps
def sslwrap(func):
    @wraps(func)
    def bar(*args, **kw):
        kw['ssl_version'] = ssl.PROTOCOL_TLSv1
        return func(*args, **kw)
    return bar

ssl.wrap_socket = sslwrap(ssl.wrap_socket)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment