Support Python Without SSL

[dstufft]

Right now it's impossible to use urllib3 from a Python that was compiled without SSL. Ideally urllib3 would still continue to function, but if you tried to do something that required the ssl module it would then bail. This would allow pip to support a limited set of functionality on non SSL Pythons.

Python 3.4.0b2+ (default, Jan 24 2014, 15:56:34)
[GCC 4.8.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
Traceback (most recent call last):
  File "/root/requests/requests/packages/urllib3/packages/ssl_match_hostname/__init__.py", line 3, in <module>
    from ssl import CertificateError, match_hostname
  File "/usr/local/lib/python3.4/ssl.py", line 97, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: No module named '_ssl'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/requests/requests/packages/urllib3/packages/ssl_match_hostname/__init__.py", line 7, in <module>
    from backports.ssl_match_hostname import CertificateError, match_hostname
ImportError: No module named 'backports'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/requests/requests/__init__.py", line 58, in <module>
    from . import utils
  File "/root/requests/requests/utils.py", line 25, in <module>
    from .compat import parse_http_list as _parse_list_header
  File "/root/requests/requests/compat.py", line 7, in <module>
    from .packages import chardet
  File "/root/requests/requests/packages/__init__.py", line 3, in <module>
    from . import urllib3
  File "/root/requests/requests/packages/urllib3/__init__.py", line 16, in <module>
    from .connectionpool import (
  File "/root/requests/requests/packages/urllib3/connectionpool.py", line 31, in <module>
    from .packages.ssl_match_hostname import CertificateError
  File "/root/requests/requests/packages/urllib3/packages/__init__.py", line 3, in <module>
    from . import ssl_match_hostname
  File "/root/requests/requests/packages/urllib3/packages/ssl_match_hostname/__init__.py", line 10, in <module>
    from _implementation import CertificateError, match_hostname
ImportError: No module named '_implementation'

[Lukasa]

So, does a relative import in that file fix this problem?

[shazow]

urllib3 should definitely work without ssl—there is much code written to allow this. A fix for this is very desirable. :)

[dstufft]

Yea, looks like changing that to from .implementation import CertificateError, match_hostname fixed it, at least for a really basic "can I import the module and make a HTTP request" test. So should be a simple fix :]

[shazow]

Anyone interested in doing this fix? Would be nice if we could get a test for import ssl raising ImportError and a scenario which imports these various submodules accordingly. (Paging @kevinburke, our resident Mockmaster.)

[kevinburke]

I am interested but also moving out of my apartment this weekend and
probably won't have much time.

Kevin Burke | Twilio
phone: 925.271.7005 | kev.inburke.com

On Fri, Jan 24, 2014 at 12:41 PM, Andrey Petrov notifications@github.comwrote:

Anyone interested in doing this fix? Would be nice if we could get a test
for import ssl raising ImportError and a scenario which imports these
various submodules accordingly. (Paging @kevinburkehttps://github.com/kevinburke,
our resident Mockmaster.)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/320#issuecomment-33258789
.

[sigmavirus24]

I think having a test that raises an ImportError like that will need to do less mocking and more playing with importlib anyway. I don't quite see a good way to mock that out.

[sigmavirus24]

I would like to write the test for this tonight. If anyone gets around to trying it before me, they should look into inserting a Finder object into sys.meta_path. I would take the approach of inserting a mocked object that has a method find_module which raises an ImportError.

[dstufft]

FWIW Python itself has some tests that do this I think... Not sure how they do it though.

[sigmavirus24]

I spent a few hours doing more research into this. To say the least, this is far less simple than I thought. Trying to patch 'builtins.__import__' on python 2 fails, so we'd have to run this test only on python 3. On python 3 though, the following works:

from unittest import mock

with mock.patch('builtins.__import__', side_effect=ImportError):
    import ssl
    print(ssl)

That of course will raise an ImportError for every call to import though. We would want to scope it to only do that on an attempt to import ssl. I don't think mock is sufficient enough for us to stub out the __import__ function. We might actually need pretend for it so we can make it call a different function instead of the normal import. Then again, I'm exhausted so I could be missing something right now.

[jschneier]

I think we should go ahead and push @Lukasa's fix even if we can't get the mocking correct. Like @shazow said a lot of code is written to ensure that urllib3 works without ssl.

This appears to only affect various versions of Python3 (fails for me on Python3.3 and @dstufft for Python3.4) but works for my Python2.7. Even if we can't get a test I was able to mock not having ssl by hacking my sys.path with an ssl.py file that consists solely of raise ImportError but it seems extremely fragile to do in a normal unittest.

I also probably went down the same path as @sigmavirus24 and tried to patch builtins.__import__ and then looked at this for help but between that and setUpClass I'd get interesting things ranging from Bus Error to RuntimeError for going over the recursion limit with cmp.

[shazow]

Happy to merge a fix. Is there a PR somewhere I'm missing?

I agree that while a test would definitely be nice to have, I don't want to block on it in this scenario.

[jschneier]

Just submitted one.

[shazow]

Woohoo merged.