boto3-1.26.102 introduces import issue

[lmynsberge]

Describe the bug

We are trying to run a tool using boto3 that stopped working with the most recent patch version (1.26.101 worked fine). Here is the stack trace:

Traceback (most recent call last):
  File "/root/project/./cicd_scripts/ssm_associations.py", line 8, in <module>
    import boto3
  File "/usr/local/lib/python3.9/site-packages/boto3/__init__.py", line 17, in <module>
    from boto3.session import Session
  File "/usr/local/lib/python3.9/site-packages/boto3/session.py", line 25, in <module>
    from .resources.factory import ResourceFactory
  File "/usr/local/lib/python3.9/site-packages/boto3/resources/factory.py", line 17, in <module>
    from ..docs import docstring
  File "/usr/local/lib/python3.9/site-packages/boto3/docs/__init__.py", line 15, in <module>
    from botocore.docs import DEPRECATED_SERVICE_NAMES
ImportError: cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs' (/root/.local/lib/python3.9/site-packages/botocore/docs/__init__.py)

Exited with code exit status 1

It looks like this import was added in the patch version

Expected Behavior

No error to be thrown when importing the module.

Current Behavior

An error is thrown when importing the module

Reproduction Steps

Install the most recent patch version and reference boto3-1.26.102 on python 3.9

Possible Solution

No response

Additional Information/Context

For now we work around by pinning our boto3 version to 1.26.101

SDK version used

1.26.102

Environment details (OS name and version, etc.)

Kernel Version: 5.15.0-1030-aws Operating System: Ubuntu 20.04.5 LTS OSType: linux Architecture: x86_64

[lmynsberge]

This is not closed, we merged a PR pinning our version to the old patch version.

[tim-finnigan]

Hi @lmynsberge thanks for reaching out. Can you check which botocore version you're using? If you have botocore version 1.29.102 installed then you shouldn't get that error. The error message was recently added here to prevent deprecated service names from showing in the docs.

[georgemansoor]

With botocore 1.29.102 I am experiencing the same issue.

[tim-finnigan]

@georgemansoor I'm not able to reproduce the issue. Can you confirm the botocore version imported in your script by adding:

import botocore
print(botocore.__version__)

If you're still seeing the issue can you share your debug logs by adding boto3.set_stream_logger('') to your script? Please redact any sensitive info.

[vb-stephane]

I'm having a similar issue. I think it stems from conflicts between boto3 and s3fs.

If i install boto3 and s3fs serially with pip I get the same import error:

sudo python3 -m pip install boto3
sudo python3 -m pip install s3fs

But if I install them simultaneously it works:

sudo python3 -m pip install boto3 s3fs

[nateprewitt]

I took a look at s3fs, it relies on aiobotocore which pins to specific versions of botocore for stability because it modifies internals of botocore. Unfortunately, that's leading to this conflict when installed with recent versions of boto3.

Until aiobotocore updates to support a newer version, I'm afraid it's not compatible with boto3>=1.26.102. We could potentially remove the import but never adding new imports hampers our ability to actually develop the project. I think in the short term, the best option would be to pin boto3 while we discuss other potential options.

[lili-ud]

I'm having a similar issue. I think it stems from conflicts between boto3 and s3fs.

If i install boto3 and s3fs serially with pip I get the same import error:

sudo python3 -m pip install boto3
sudo python3 -m pip install s3fs

But if I install them simultaneously it works:

sudo python3 -m pip install boto3 s3fs

Yes, it works.

[lmynsberge]

Thanks for all the analysis and support y'all.

I'm guessing I'm also hitting some sort of dependency combination issue. We're using a fresh install through a CircleCI Job. I can't reproduce locally at all.

I don't see a dependency on s3fs or aiobotocore, so I'm not sure why. I leveraged pipdeptree to print out the full dependency tree ($ pipdeptree -fl) and that showed that the only dependency of botocore is boto3 (which we have pinned to 1.26.101 for now to avoid this bug).

Can anyone think of anything else that might be contributing to it?

ansible==7.4.0
  ansible-core==2.14.4
    cryptography==40.0.1
      cffi==1.15.1
        pycparser==2.21
    Jinja2==3.1.2
      MarkupSafe==2.1.2
    packaging==23.0
    PyYAML==5.4.1
    resolvelib==0.8.1
boto3==1.26.101
  botocore==1.29.104
    jmespath==1.0.1
    python-dateutil==2.8.2
      six==1.16.0
    urllib3==1.26.15
  jmespath==1.0.1
  s3transfer==0.6.0
    botocore==1.29.104
      jmespath==1.0.1
      python-dateutil==2.8.2
        six==1.16.0
      urllib3==1.26.15
DateTime==5.1
  pytz==2023.3
  zope.interface==6.0
    setuptools==50.3.2
deepdiff==6.3.0
  ordered-set==4.1.0
GitPython==3.1.31
  gitdb==4.0.10
    smmap==5.0.0
ipaddress==1.0.23
ipsecparse==0.3.0
  ply==3.11
pip==20.3.1
pipdeptree==2.7.0
pydash==6.0.2
PyJWT==2.6.0
python-iptables==1.0.1
requests==2.28.2
  certifi==2022.12.7
  charset-normalizer==3.1.0
  idna==3.4
  urllib3==1.26.15
uuid==1.30
wheel==0.36.0

[simonarcher99]

Can anyone think of anything else that might be contributing to it?

@lmynsberge it looks like you are using boto3==1.26.101 and botocore==1.29.104. The issue seems to come from using botocore >= 1.26.102 with boto3 < 1.26.102 or vice versa. Updating your boto3 version or downgrading your botocore version should fix the issue.

[Sainath-blackline]

I'm having a similar issue. I think it stems from conflicts between boto3 and s3fs.

If i install boto3 and s3fs serially with pip I get the same import error:

sudo python3 -m pip install boto3
sudo python3 -m pip install s3fs

But if I install them simultaneously it works:

sudo python3 -m pip install boto3 s3fs

I had similar import issue with latest version. It works . thanks!

[ksco92]

I am seeing this issue with boto3==1.26.106 and botocore==1.29.106 and python 3.8 in aws lambda.

[tim-finnigan]

@ksco92 are you also using a third-party software like s3fs? Are you using a Lambda layer and can you confirm that printing your botocore version as suggested here shows a version that is compatible with your version of boto3?

[Bhawika16]

any idea how can we fix this for python version 3.7 ? I tried all the above solutions but none worked for me

[Bhawika16]

I am seeing this issue with boto3==1.26.106 and botocore==1.29.106 and python 3.8 in aws lambda.

how did you fix the issue ?

[elachere]

any idea how can we fix this for python version 3.7 ? I tried all the above solutions but none worked for me

@Bhawika16 Happened to me today with python3.7, fixed with boto3==1.26.63 and botocore==1.29.63

[Parth3716]

Hi @lmynsberge thanks for reaching out. Can you check which botocore version you're using? If you have botocore version 1.29.102 installed then you shouldn't get that error. The error message was recently added here to prevent deprecated service names from showing in the docs.

This seems to work for me. Thanks! I was using botocore 1.29.103 before giving the same error. However downgrading botocore version to 1.29.102 seems to fix this issue.

[ag-TJNII]

I hit this with a odd stack of dependencies:

• Building off a gcr.io/google.com/cloudsdktool/google-cloud-cli:433.0.0-alpine image
• Installing boto3 using pip3 (No restrictions)
• Installing aws-cli via APK

I didn't dig into the exact versions but I did find that these 3 factors, in this order, triggered this issue. Installing aws-cli before boto3 in my Dockerfile resolved it.

So anyway heads up that package order may be a factor when installing from multiple sources, not just the packages themselves.

[jparkhill]

It's completely nuts this isn't fixed yet. AWS isn't cheap! You guys can't get an engineer to fix a version conflict? This is how you lose a market. Just sharing my pinned versions in case it helps anyone.

"aiobotocore==2.3", "boto3==1.21", "botocore==1.24.21",

[rachelwigell]

Does anyone have a working combination of versions where botocore is more recent, like >=1.31.4?

We need pyyaml 6.0.1, which requires awscli >= 1.29.4, which requires newer versions of botocore.

Right now I have boto3 == 1.28.4 and botocore == 1.31.4, which look like they ought to be compatible according to their setup.py, but I'm getting the error described in this issue. Very annoying!

[titowoche30]

Working config using Python 3.7

s3fs==2023.1.0
boto3==1.24.59
botocore==1.27.59

[jbsilva]

I just got this error using the version from the default runtime.

Adding my own version of boto3 (1.28.64) to the package solved the problem.
I was not packaging boto3 due to the size limits. My zip was 49MB.

[francesco-larghi-duferco]

I had the same issue, but in my case was for a completely different reason.
The stacktrace was raised by a package due to a mismatch of versions installed between pip packages and actual version python installed.
To solve this in my case just:

• Remove your virtual environment and create it from scratch with the correct version of python (for me it was Python 3.8)
• Upgrade pip and reinstall boto3 or the packages containing it etc. from scratch

Hope someone find this useful.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.

[devansh-purwar]

LAMBDA_WARNING: Unhandled exception. The most likely cause is an issue in the function code. However, in rare cases, a Lambda runtime update can cause unexpected function behavior. For functions using managed runtimes, runtime updates can be triggered by a function change, or can be applied automatically. To determine if the runtime has been updated, check the runtime version in the INIT_START log entry. If this error correlates with a change in the runtime version, you may be able to mitigate this error by temporarily rolling back to the previous runtime version. For more information, see https://docs.aws.amazon.com/lambda/latest/dg/runtimes-update.html
[ERROR] Runtime.ImportModuleError: Unable to import module 'adbrew_core.serverless.pending_reports_handler': cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs' (/var/task/botocore/docs/init.py)
Traceback (most recent call last):

boto3==1.19.9, botocore==1.22.9 facing same issue with python 3.8

@SamRemis @lmynsberge

[devansh-purwar]

solved by changing python version to 3.8.2024.03.04.10