Remove import from deprecated urllib3.contrib.pyopenssl

[intgr]

urllib3 is in the process of deprecating urllib3.contrib.pyopenssl, see urllib3/urllib3#2680

requests-toolbelt currently attempts to import from urllib3.contrib.pyopenssl import PyOpenSSLContext. Apparently this is used by X509Adapter.

What's the path forward here? Can X509Adapter be ported to newer APIs that are supported?

Otherwise it would be preferred if DeprecationWarning was only shown for code that actually uses X509Adapter, maybe make the import lazy and put it in a function?

[pquentin]

The urllib3 warning is only printed if a package named urllib3-secure-extra is installed in your virtualenv. The imports don't have any effect on this. See https://youtu.be/_jUXdX8e9Wg for a more complete explanation.

And indeed simply installing requests and request-toolbelt does not install this package because neither project tries to install urllib3[secure]. If you want to use X509Adapter you can always install pyopenssl manually.

In other words, there's no bug in request-toolbelt. You need to understand why you have urllib3-secure-extra installed. Can you please post the output of pip freeze?

[intgr]

In other words, there's no bug in request-toolbelt. You need to understand why you have urllib3-secure-extra installed.

urllib3-secure-extra is not installed, but I do have pyOpenSSL installed.

As urllib3/urllib3#2680 describes, it's not just the [secure] extra that was deprecated but also: "'urllib3.contrib.pyopenssl' module is deprecated and will be removed in a future release of urllib3 2.x."

I tried this with a fresh project/virtualenv:

poetry init -n
poetry shell
poetry add requests-toolbelt pyopenssl
python -W all

>>> from requests_toolbelt.adapters.ssl import SSLAdapter
.../pypoetry/virtualenvs/testpkg-erYTvXjZ-py3.10/lib/python3.10/site-packages/requests_toolbelt/_compat.py:56: DeprecationWarning: 'urllib3.contrib.pyopenssl' module is deprecated and will be removed in a future release of urllib3 2.x. Read more in this issue: https://github.com/urllib3/urllib3/issues/2680
  from requests.packages.urllib3.contrib.pyopenssl \

[pquentin]

Thanks, I was focused on the extra and forgot about pyopenssl itself, sorry! I am seeing two options:

• Import this lazily in create_ssl_context and check the requests version in _check_version instead of _compat.py. It won't work with urllib3 2.0 though.
• Inline PyOpenSSLContext inside requests-toolbelt. This is more code to maintain though.

[mgorny]

Ping. This is major pain the butt as it causes pytest to crash when trying to run e.g. hypothesis test suite when requests-toolbelt just happens to be imported indirectly:

ERROR: while parsing the following warning configuration:

  ignore::hypothesis.errors.NonInteractiveExampleWarning

This error occurred:

Traceback (most recent call last):
  File "/usr/lib/pypy3.9/site-packages/_pytest/config/__init__.py", line 1639, in parse_warning_filter
    category: Type[Warning] = _resolve_warning_category(category_)
  File "/usr/lib/pypy3.9/site-packages/_pytest/config/__init__.py", line 1677, in _resolve_warning_category
    m = __import__(module, None, None, [klass])
  File "/tmp/portage/dev-python/hypothesis-6.56.1/work/hypothesis-hypothesis-python-6.56.1/hypothesis-python-pypy3/install/usr/lib/pypy3.9/site-packages/hypothesis/__init__.py", line 56, in <module>
    run()
  File "/tmp/portage/dev-python/hypothesis-6.56.1/work/hypothesis-hypothesis-python-6.56.1/hypothesis-python-pypy3/install/usr/lib/pypy3.9/site-packages/hypothesis/entry_points.py", line 63, in run
    hook = entry.load()
  File "/usr/lib/pypy3.9/importlib/metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "/usr/lib/pypy3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<builtin>/frozen importlib._bootstrap_external", line 865, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/lib/pypy3.9/site-packages/pydantic/_hypothesis_plugin.py", line 53, in <module>
    import email_validator
  File "/usr/lib/pypy3.9/site-packages/email_validator/__init__.py", line 6, in <module>
    import dns.resolver
  File "/usr/lib/pypy3.9/site-packages/dns/resolver.py", line 38, in <module>
    import dns.query
  File "/usr/lib/pypy3.9/site-packages/dns/query.py", line 43, in <module>
    from requests_toolbelt.adapters.source import SourceAddressAdapter
  File "/usr/lib/pypy3.9/site-packages/requests_toolbelt/__init__.py", line 12, in <module>
    from .adapters import SSLAdapter, SourceAddressAdapter
  File "/usr/lib/pypy3.9/site-packages/requests_toolbelt/adapters/__init__.py", line 12, in <module>
    from .ssl import SSLAdapter
  File "/usr/lib/pypy3.9/site-packages/requests_toolbelt/adapters/ssl.py", line 16, in <module>
    from .._compat import poolmanager
  File "/usr/lib/pypy3.9/site-packages/requests_toolbelt/_compat.py", line 56, in <module>
    from requests.packages.urllib3.contrib.pyopenssl \
  File "/usr/lib/pypy3.9/site-packages/urllib3/contrib/pyopenssl.py", line 82, in <module>
    warnings.warn(
DeprecationWarning: 'urllib3.contrib.pyopenssl' module is deprecated and will be removed in a future release of urllib3 2.x. Read more 
in this issue: https://github.com/urllib3/urllib3/issues/2680

[pquentin]

In requests-toolbelt 0.10.1, the warning is only emitted for direct usage of X509Adapter, which should solve the problem for the vast majority of users.

[mgorny]

Thanks a lot! I can confirm that we can successfully run the hypothesis test suite again now.