Cannot import name 'PROTOCOL_TLS' from 'urllib3.util.ssl_'

[mantoine96]

Describe the bug

This issue is a direct follow-up from #2562

When using older versions of urllib3 (often vendored), importing botocore.utils (or using botocore in any consideration) will fail.

Steps to reproduce

$ pip install --upgrade botocore==1.23.26
$ pip install --upgrade urllib3==1.22.0
$ python
>>> import botocore.utils
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/matthieu/.virtualenvs/reproavm/lib/python3.8/site-packages/botocore/utils.py", line 32, in <module>
    import botocore.httpsession
  File "/Users/matthieu/.virtualenvs/reproavm/lib/python3.8/site-packages/botocore/httpsession.py", line 10, in <module>
    from urllib3.util.ssl_ import (
ImportError: cannot import name 'PROTOCOL_TLS' from 'urllib3.util.ssl_' (/Users/matthieu/.virtualenvs/reproavm/lib/python3.8/site-packages/urllib3/util/ssl_.py)

Expected behavior

botocore should fallback to importing PROTOCOL_TLS from ssl.

[stobrien89]

Hi @thehunt33r,

Thanks for raising this! I don't see anything wrong with your change at surface-level, but I'll review with the team to get their thoughts.

[stobrien89]

Hi @thehunt33r,

I saw you mentioned using vendored older versions of urllib3, but is there a particular reason you need to use such an old version? You may already be aware, but our minimum version required at this time is 1.25.4.

[mantoine96]

Hey!

Sorry yeah I was using an old version of an AWS solution (LandingZone) I'm stuck with for the foreseeable future.

I did see the minimum requirement for the version of urllib3 but since an earlier similar PR was merged I thought it could be worth it to do this change? If you don't think that's worth it feel free to close the PR :-)

Thank you for taking the time to look through it.

Matthieu

[stobrien89]

Hi @thehunt33r,

That makes sense. I discussed this with a few team members yesterday and they were not inclined to merge the PR due to the problem occurring well below our urllib3 floor, but we were missing a few other team members due to the holidays— I'll bring this up again once we have the full group and see if we can get a better consensus.

[drigoli]

Hi @stobrien89,
I see this issue too.

awscli==1.22.26
botocore==1.23.26
urllib3==1.26.7

Statement on lines 10-12 in httpsession.py wants to import PROTOCOL_TLS from urllib3.util.ssl_, which does not exist there.

I can workaround by removing it from that statement and adding a separate statement, pulling from ssl. Though, I suspect the strategic direction is to level up on TLS version which would require more planning.

[nateprewitt]

Hi @drigoli,

PROTOCOL_TLS has been present in each release of urllib3 for the last ~3 years since 1.25.0. You can find the import workflow here. I'd double check the version of urllib3 you're using with botocore is actually 1.26.7. You can use urllib3.__version__ in your script to verify.

As @stobrien89 said earlier, I think we're disinclined to add support for these older versions. The primary reason is they contain multiple medium to high scoring CVEs which may compromise your security. We'd prefer to not make it easy for users to unknowingly use these versions on their systems.

For the time being, I think our recommendation is going to be investing in upgrading urllib3 to a secure version. As Sean, said we may revisit this after the New Year, but I believe it's unlikely to change.

[drigoli]

Good catch @nateprewitt,
Looks like pip isn't reporting the same version numbers as observed in the shell or in the site-packages directory.

pip reported version:

Package         Version
--------------- -------
awscli          1.22.26
botocore        1.23.26
...
pip             21.2.4
urllib3         1.26.7

interactive query versions:

In [2]: urllib3.__version__
Out[2]: '1.24.3'
...
In [5]: botocore.__version__
Out[5]: '1.23.19'

In [6]: boto3.__version__
Out[6]: '1.20.19'

hmm...

[stobrien89]

Hi @thehunt33r and @drigoli,

After further discussion, we've decided not to move forward with merging the PR related to this issue due to the security risks associated with the CVEs mentioned by @nateprewitt. Sorry for the inconvenience!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[mantoine96]

Thanks @stobrien89!

[dmitryivashkin]

Still reproducible on python 3.8.13:
botocore 1.27.70
urllib3 1.26.12