Drop support for OpenSSL<1.1.1 #2168

pquentin · 2021-03-08T06:28:01Z

Context

The TLS situation in Python has considerably improved since the early years of urllib3, thanks to the hard work of persons like Christian Heimes and Cory Benfield. urllib3 took advantage of new features even when only a subset of users could use it, and still accepts OpenSSL versions that don't have SNI, for example.

Here's what OpenSSL currently supports:

Version 1.1.1 will be supported until 2023-09-11 (LTS).
Version 1.0.2 is no longer supported. Extended support for 1.0.2 to gain access to security fixes for that version is available.
Versions 1.1.0, 1.0.1, 1.0.0 and 0.9.8 are no longer supported.

RHEL 6 supports 1.0.1e+ and RHEL 7 only supports 1.0.2k and beyond.

We also know that Python 3.10+ will require OpenSSL 1.1.1+ thanks to PEP 644.

Given this the only operating systems that would be in a tough spot if we decide to drop support for OpenSSL <1.1.1 are OSes who:

Still support OpenSSL <1.1.1 as their default OpenSSL
Use Python >=3.7 but <3.10 for their default Python
Are likely to upgrade their system package for urllib3 to v2.0

The combination of the above three is very unlikely. We've identified a few OSes we'd like to evaluate to make sure before we release v2.0:

Amazon Linux 2
Gentoo

Minimum requirements

💰 You can get paid to complete this issue! Please read the docs for more information.

Evaluate the above OSes to see if they'd be impacted by dropping support for OpenSSL <1.1.1. Leave this in a comment in this issue.
Raise an ImportError if not OpenSSL or ssl.OPENSSL_VERSION < (1, 1, 1) with a message about urllib3 v2.0 requiring OpenSSL 1.1.1+
Remove work-arounds for conditional features around the ssl module that are due to OpenSSL <1.1.1 (minimum_version, HAS_SNI, _is_openssl_gt_v1_1_1, more examples below)
Add documentation for urllib3 requiring OpenSSL 1.1.1+
Add a newfragment

The text was updated successfully, but these errors were encountered:

sethmlarson · 2021-03-08T16:21:23Z

I'm in favor of dropping support for OpenSSL <1.0.2
- However I'm not sure if we can stop supporting ssl without SNI, might require a bit more research (can you disable SNI at compile time of OpenSSL? Might be something that microcontrollers do?)
If PEP 644 goes through we can discuss dropping OpenSSL <1.1.1, but I'm not sure beyond tighter support matrices what we gain from doing that? Does our code rely on detecting OpenSSL 1.1.1 features beyond TLS 1.3?
Definitely remove notOpenSSL098

pquentin · 2021-03-10T05:39:09Z

SNI can be disabled in 1.0.2, but no longer in 1.1.0: openssl/openssl@e481f9b. We'll keep the check for now.
We currently rely on detection of OpenSSL/Python versions to:
- Enable SNI (see SNIMissingWarning)
- Use OpenSSL ciphers for OpenSSL 1.1.1+
- Call context.set_alpn_protocols(ALPN_PROTOCOLS)
- Enable post-handshake authentication
- Use SSLKEYLOGFILE
- Call load_default_certs
~~Will remove notOpenSSL098, thanks~~ Done

pquentin · 2021-08-18T08:07:21Z

For reference, PEP 644 was accepted for Python 3.10.

graingert · 2022-06-13T11:23:38Z

how about requiring OpenSSL 1.1.0g or newer? This would fix #2636

pquentin · 2022-06-13T12:10:45Z

I think that requiring 1.1.1+ for v2 is reasonable, which is what RHEL 8 and Debian stable support. While RHEL 7 is still supporting 1.0.2k+, it is end of life, and for context they still ship urllib3 1.10! Also, while we have CI for 1.1.1 and 3.0.0, we don't have anything for 1.0.2 or 1.1.0.

graingert · 2022-06-13T20:56:00Z

requiring 1.1.1+ will also allow dropping of pyopenssl inject_into_urllib3 https://github.com/psf/requests/search?q=inject_into_urllib3&type=code and if urllib3 drops ssl 1.0.1 then SecureTransport can go too: https://github.com/pypa/pip/blob/bf91a079791f2daf4339115fb39ce7d7e33a9312/src/pip/_internal/utils/inject_securetransport.py#L24

sethmlarson · 2022-06-21T16:53:45Z

It'd be interesting to see which flavors of Linux are both using OpenSSL <1.1.1 and likely to update to urllib3 v2.0 as a part of their system libraries specifically. I suspect the list is extremely small. Given that information I'd like to consider only supporting OpenSSL 1.1.1+ for v2.0 unless we find an example that's still using OpenSSL <1.1.1.

Does anyone have such an example?

sethmlarson · 2022-06-21T17:04:35Z

From this link: https://repology.org/project/openssl/versions

Linux flavors of interest:

AlmaLinux 8 (OpenSSL 1.0.2 / 1.1.1)
AlmaLinux 9 (OpenSSL 3.0)
Alpine 3.8 (OpenSSL 1.0.2)
Alpine 3.9+ (OpenSSL 1.1.1+)
Amazon Linux 1 (OpenSSL 1.0.2)
Amazon Linux 2 (OpenSSL 1.0.2/1.1.1+) Drop support for OpenSSL<1.1.1 #2168 (comment)
Arch (OpenSSL 1.1.1)
CentOS 6 (OpenSSL 1.0.1)
CentOS 7 (OpenSSL 1.0.2)
CentOS 8+ (OpenSSL 1.1.1+)
Debian 10+ (OpenSSL 1.1.1/3.0)
Fedora 29+ (OpenSSL 1.1.1+)
Gentoo stable (OpenSSL 1.1.1o - Drop support for OpenSSL<1.1.1 #2168 (comment))
Raspbian stable (OpenSSL 1.1.1+)
Ubuntu 18.04+ (OpenSSL 1.1.1+)

~~Would be great to fill in the ⚠️ with details.~~ Done

sethmlarson · 2022-06-21T19:39:15Z

@pquentin Brought up the case where a downstream packaged pip unbundles urllib3, if the user were to upgrade their system's installation of urllib3 they'd essentially brick their system and not be able to downgrade urllib3 using pip alone.

I'm not sure there's a way we can both use static build system and disallow users from installing urllib3 v2.0 on a system with OpenSSL <1.1.1 so any guard rails here would be for cleaning up the mess afterwards?

Perhaps we can point to documentation on how to unmangle your system if you've upgraded urllib3 after running pip install with system Python?

graingert · 2022-06-21T19:51:56Z

a downstream packaged pip unbundles urllib3

The system unbundled system pip is usually on a very old python version

sethmlarson · 2022-06-21T19:56:42Z

True, so the system would also have to have Python 3.7-3.9 installed to have this issue since urllib3 v2.0 requires 3.7+ and 3.10 requires OpenSSL 1.1.1+.

graingert · 2022-06-21T20:02:18Z

Are there any distributions with an unbundled pip with openssl < 1.1.1 running on python 3.7+?

sethmlarson · 2022-06-28T00:24:59Z

Probably not. Outside of anyone else bringing a good reason for us to support OpenSSL 1.1.0 or earlier we're going to drop support of OpenSSL <1.1.1. Going to update this issue appropriately.

pquentin · 2022-06-28T01:13:03Z

@mgorny Is requiring OpenSSL 1.1.1+ for urllib3 2.0 going to be a problem for Gentoo?

mgorny · 2022-06-28T03:59:35Z

Not at all, Gentoo stable is at 1.1.1o already.

mgorny · 2022-06-28T03:59:46Z

(Thanks for asking)

pquentin · 2022-06-28T05:14:15Z

Amazon Linux 2 has an openssl11 package but comes with OpenSSL 1.0.2 preinstalled:

$ docker run -ti amazonlinux:2 yum list | grep openssl | grep installed
openssl-libs.x86_64                    1:1.0.2k-24.amzn2.0.3          installed

But then it's also preinstalled with Python 2.7:

$ docker run -ti amazonlinux:2 python
Python 2.7.18 (default, May 25 2022, 14:30:51) 
[GCC 7.3.1 20180712 (Red Hat 7.3.1-15)] on linux2

Updated the list above with that information. I don't think that changes our plans.

(I'm also not claiming that issue even if I completed the first item.)

IvanLauLinTiong · 2022-08-11T15:14:16Z

pyOpenSSL is deprecated and will be removed in future release version 2.x (#2691).

Version 2.0 of urllib3 was just released which fails if used against any version of python built against a version of the OpenSSL library older than 1.1.1 (see discussion [here](urllib3/urllib3#2168)) That seems to include all versions of Python older than 3.9. This change triggers the read the docs builds to fail when running sphinx with a message of the form: ``` Running Sphinx v6.2.1 Traceback (most recent call last): File "/home/docs/checkouts/readthedocs.org/user_builds/jupyter-server/envs/1267/lib/python3.8/site-packages/sphinx/registry.py", line 442, in load_extension mod = import_module(extname) File "/home/docs/checkouts/readthedocs.org/user_builds/jupyter-server/envs/1267/lib/python3.8/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 671, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 783, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/home/docs/checkouts/readthedocs.org/user_builds/jupyter-server/envs/1267/lib/python3.8/site-packages/sphinx/builders/linkcheck.py", line 20, in <module> from requests import Response File "/home/docs/checkouts/readthedocs.org/user_builds/jupyter-server/envs/1267/lib/python3.8/site-packages/requests/__init__.py", line 43, in <module> import urllib3 File "/home/docs/checkouts/readthedocs.org/user_builds/jupyter-server/envs/1267/lib/python3.8/site-packages/urllib3/__init__.py", line 38, in <module> raise ImportError( ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2n 7 Dec 2017. See: urllib3/urllib3#2168 ``` This change attempts to fix that by upgrading the Python version used for the read the docs build from 3.8 to 3.9

When building sphinx docs in the CI environment we get this error: ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2n 7 Dec 2017. See: urllib3/urllib3#2168 This commit pins urllib to < 2.0.0 so we can avoid this error until the environment the docs are built on supports uses a newer openssl version.

geopopos · 2023-05-07T01:13:50Z

Is there a way to force OpenSSL? I am getting the following error
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3. See

gopackgo90 · 2023-05-07T03:54:05Z

Is there a way to force OpenSSL? I am getting the following error ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3. See

LibreSSL isn't supported anymore as of 2.0, see https://urllib3.readthedocs.io/en/stable/v2-migration-guide.html#what-are-the-important-changes

Removed support for non-OpenSSL TLS libraries (like LibreSSL and wolfSSL).

and https://urllib3.readthedocs.io/en/stable/changelog.html

Removed support for Python with an ssl module compiled with LibreSSL, CiscoSSL, wolfSSL, and all other OpenSSL alternatives. Python is moving to require OpenSSL with PEP 644 (#2168).

vincent-paing · 2023-05-07T06:32:27Z

Seeing ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3. See: https://github.com/urllib3/urllib3/issues/2168 when I import tensorflow on macOS. it pointed me to this issue, can someone help me with what's the fix for this?

akrsh24 · 2023-05-07T08:47:07Z

I faced the same issue. I switched to a different Python kernel (3.11.2) and it worked.

A couple things happening here, but to summarize: - See readthedocs/readthedocs.org#10290 for more background - The bug exists because pyenv compiled Python was built against an old openssl package. This is outside our control - The intended fix is to rebuild the image with pyenv images that have hopefully been compiled against openssl 1.1.1+ - This PR doesn't yet resolve the bug, as it doesn't seem pyenv packages have indeed been updated recently. - We probably need to also update the version numbers for all of the pyenv installed binaries for this approach to work To test / repro (after rebuilding this image): docker run readthedocs/build:6.0.4 /bin/bash docs@778c7ce0877f:/$ eval "$(pyenv init -)" docs@778c7ce0877f:/$ pyenv shell 3.7.9 docs@778c7ce0877f:/$ pip install --quiet requests WARNING: You are using pip version 20.0.1; however, version 23.1.2 is available. You should consider upgrading via the '/home/docs/.pyenv/versions/3.7.9/bin/python3.7 -m pip install --upgrade pip' command. docs@778c7ce0877f:/$ python -m requests Traceback (most recent call last): File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/runpy.py", line 183, in _run_module_as_main mod_name, mod_spec, code = _get_module_details(mod_name, _Error) File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/runpy.py", line 142, in _get_module_details return _get_module_details(pkg_main_name, error) File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/runpy.py", line 109, in _get_module_details __import__(pkg_name) File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/site-packages/requests/__init__.py", line 43, in <module> import urllib3 File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/site-packages/urllib3/__init__.py", line 39, in <module> "urllib3 v2.0 only supports OpenSSL 1.1.1+, currently " ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2n 7 Dec 2017. See: urllib3/urllib3#2168

stefan11111 · 2023-05-07T22:26:40Z

This breaks libressl compatibility on gentoo.
https://forums.gentoo.org/viewtopic-p-8788598.html#8788598

gopackgo90 · 2023-05-07T22:32:23Z

This breaks libressl compatibility on gentoo.
https://forums.gentoo.org/viewtopic-p-8788598.html#8788598

Dropping LibreSSL support is intentional, see #2168 (comment).

ionenwks · 2023-05-08T00:04:29Z

This breaks libressl compatibility on gentoo.

Just for the record, Gentoo does not officially support LibreSSL anymore, so typical users are not affected.

orbea · 2023-05-08T01:01:26Z

Just for the record, Gentoo does not officially support LibreSSL anymore, so typical users are not affected.

LibreSSL is supported here. https://github.com/gentoo/libressl

orbea · 2023-05-08T01:29:08Z

I tested this patch locally against urllib3 2.0.2 and it seems to work so far with LibreSSL 3.7.2, is there a reason why it must be extra hard to use LibreSSL? If there are bugs or missing features inside of LibreSSL that are important then those should be reported upstream, but right now it seems like obstacles for the sake of having obstacles.

--- a/src/urllib3/__init__.py
+++ b/src/urllib3/__init__.py
@@ -32,7 +32,10 @@ except ImportError:
 else:
     # fmt: off
     if (
-        not ssl.OPENSSL_VERSION.startswith("OpenSSL ")
+        not (
+            ssl.OPENSSL_VERSION.startswith("OpenSSL ")
+            or ssl.OPENSSL_VERSION.startswith("LibreSSL ")
+        )
         or ssl.OPENSSL_VERSION_INFO < (1, 1, 1)
     ):  # Defensive:
         raise ImportError(

sigmavirus24 · 2023-05-08T01:55:23Z

As had been shared twice now in very recent comments above, update Python is doing libressl support for the standard library. #2168 (comment)

sethmlarson added the TLS label Aug 15, 2021

sethmlarson modified the milestones: v2.0, v2.x Aug 15, 2021

sethmlarson modified the milestones: v2.x, v2.0 Jun 28, 2022

sethmlarson changed the title ~~[v2] Drop support for OpenSSL<1.0.2~~ Drop support for OpenSSL<1.1.1 Jun 28, 2022

sethmlarson added the 💰 Bounty $300 If you complete this issue we'll pay you $300 on OpenCollective! label Jun 28, 2022

sethmlarson mentioned this issue Jun 28, 2022

urllib3 v2 (unreleased) negotiates the wrong version of TLS when used with OpenSSL 1.0.2 #2636

Closed

This comment was marked as spam.

Sign in to view

sethmlarson mentioned this issue Aug 9, 2022

Drop support for OpenSSL <1.1.1 #2705

Merged

sethmlarson self-assigned this Aug 9, 2022

ojarjur mentioned this issue May 5, 2023

Use Python 3.9 for the readthedocs builds jupyter-server/jupyter_server#1269

Merged

xingforce mentioned this issue May 6, 2023

SSL模块问题 manateelazycat/mind-wave#35

Closed

HAOCHENYE mentioned this issue May 6, 2023

[Fix] Limit urllib3 version for docs building open-mmlab/mmengine#1132

Merged

chrysle mentioned this issue May 6, 2023

Fix ReadTheDocs build pypa/twine#991

Merged

jacobtylerwalls mentioned this issue May 6, 2023

Use Python 3.11 for Read the Docs build pylint-dev/astroid#2165

Merged

shubham8garg mentioned this issue May 6, 2023

OpenSSL issue alon-albalak/TLiDB#4

Open

basnijholt mentioned this issue May 6, 2023

Add extra_headers option to get methods brettcannon/gidgethub#192

Open

br3ndonland mentioned this issue May 6, 2023

Fix urllib3.util.ssl_.DEFAULT_CIPHERS import httpie/httpie#1501

Open

jennank mentioned this issue May 6, 2023

Fixing missing ingredient in steps of carbonara recipe bernardopacini/ReadTheRecipe#33

Open

ExMikuPro mentioned this issue May 7, 2023

关于easyMirai中requests调用的urllib3 的错误解决方案 easyMirais/easyMirai#8

Open

pz-max mentioned this issue May 7, 2023

fix readthedocs built with python 3.11 pypsa-meets-earth/pypsa-earth#716

Merged

byncn mentioned this issue May 7, 2023

阿里云函数布置测试错误 Womsxd/MihoyoBBSTools#41

Open

keflavich mentioned this issue May 7, 2023

bump RTD to py39 astropy/astroquery#2730

Merged

simonw mentioned this issue May 7, 2023

sphinx.builders.linkcheck build error simonw/sqlite-utils#540

Closed

JeanSiegel mentioned this issue May 7, 2023

User Feedback on Tutorial from Write the Docs Writing Day readthedocs/readthedocs.org#10299

Open

testworksau mentioned this issue May 8, 2023

Please provide a Windows docker image GitGuardian/ggshield#522

Open

asukan007 mentioned this issue May 8, 2023

ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+ googleapis/python-bigquery#1565

Open

Drop support for OpenSSL<1.1.1 #2168

Drop support for OpenSSL<1.1.1 #2168

pquentin commented Mar 8, 2021 •

edited by sethmlarson

sethmlarson commented Mar 8, 2021

pquentin commented Mar 10, 2021 •

edited

pquentin commented Aug 18, 2021

graingert commented Jun 13, 2022

pquentin commented Jun 13, 2022

graingert commented Jun 13, 2022

sethmlarson commented Jun 21, 2022 •

edited

sethmlarson commented Jun 21, 2022 •

edited by pquentin

sethmlarson commented Jun 21, 2022

graingert commented Jun 21, 2022 •

edited

sethmlarson commented Jun 21, 2022 •

edited

graingert commented Jun 21, 2022

sethmlarson commented Jun 28, 2022

pquentin commented Jun 28, 2022

mgorny commented Jun 28, 2022 •

edited by pquentin

mgorny commented Jun 28, 2022

pquentin commented Jun 28, 2022 •

edited

This comment was marked as spam.

IvanLauLinTiong commented Aug 11, 2022

geopopos commented May 7, 2023

gopackgo90 commented May 7, 2023 •

edited

vincent-paing commented May 7, 2023

akrsh24 commented May 7, 2023

stefan11111 commented May 7, 2023

gopackgo90 commented May 7, 2023

ionenwks commented May 8, 2023

orbea commented May 8, 2023

orbea commented May 8, 2023

sigmavirus24 commented May 8, 2023

Drop support for OpenSSL<1.1.1 #2168

Drop support for OpenSSL<1.1.1 #2168

Comments

pquentin commented Mar 8, 2021 • edited by sethmlarson

Context

Minimum requirements

sethmlarson commented Mar 8, 2021

pquentin commented Mar 10, 2021 • edited

pquentin commented Aug 18, 2021

graingert commented Jun 13, 2022

pquentin commented Jun 13, 2022

graingert commented Jun 13, 2022

sethmlarson commented Jun 21, 2022 • edited

sethmlarson commented Jun 21, 2022 • edited by pquentin

sethmlarson commented Jun 21, 2022

graingert commented Jun 21, 2022 • edited

sethmlarson commented Jun 21, 2022 • edited

graingert commented Jun 21, 2022

sethmlarson commented Jun 28, 2022

pquentin commented Jun 28, 2022

mgorny commented Jun 28, 2022 • edited by pquentin

mgorny commented Jun 28, 2022

pquentin commented Jun 28, 2022 • edited

This comment was marked as spam.

IvanLauLinTiong commented Aug 11, 2022

geopopos commented May 7, 2023

gopackgo90 commented May 7, 2023 • edited

vincent-paing commented May 7, 2023

akrsh24 commented May 7, 2023

stefan11111 commented May 7, 2023

gopackgo90 commented May 7, 2023

ionenwks commented May 8, 2023

orbea commented May 8, 2023

orbea commented May 8, 2023

sigmavirus24 commented May 8, 2023

pquentin commented Mar 8, 2021 •

edited by sethmlarson

pquentin commented Mar 10, 2021 •

edited

sethmlarson commented Jun 21, 2022 •

edited

sethmlarson commented Jun 21, 2022 •

edited by pquentin

graingert commented Jun 21, 2022 •

edited

sethmlarson commented Jun 21, 2022 •

edited

mgorny commented Jun 28, 2022 •

edited by pquentin

pquentin commented Jun 28, 2022 •

edited

gopackgo90 commented May 7, 2023 •

edited