Clash between awscli2 and aws-sam-cli's botocore

[JoeDupuis]

Describe the bug
There is a clash with the version of awscli2 and aws-sam-cli
Depending on the order they are loaded in the environment awscli2 won't run.

It looks like both will load a version of botocore on the path and one version is not compatible with the other.

botocore seems is not a direct dependency of aws-sam-cli, but a sub-dependency.

Still investigating for more details.

To Reproduce
nix-shell -p aws-sam-cli awscli2 --run "aws --version"
Will output :

Traceback (most recent call last):
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/bin/.aws-wrapped", line 28, in <module>
    sys.exit(main())
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/bin/.aws-wrapped", line 24, in main
    return awscli.clidriver.main()
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/clidriver.py", line 94, in main
    driver = create_clidriver()
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/clidriver.py", line 103, in create_clidriver
    load_plugins(session.full_config.get('plugins', {}),
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/plugin.py", line 48, in load_plugins
    _load_plugins(BUILTIN_PLUGINS, event_hooks)
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/plugin.py", line 62, in _load_plugins
    modules = _import_plugins(plugin_mapping)
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/plugin.py", line 76, in _import_plugins
    module = __import__(path, fromlist=[module])
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/handlers.py", line 43, in <module>
    from awscli.customizations.configure.configure import register_configure_cmd
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/customizations/configure/configure.py", line 27, in <module>
    from awscli.customizations.configure.sso import ConfigureSSOCommand
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/customizations/configure/sso.py", line 32, in <module>
    from awscli.customizations.sso.utils import do_sso_login
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/customizations/sso/__init__.py", line 17, in <module>
    from awscli.customizations.sso.login import LoginCommand
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/customizations/sso/login.py", line 14, in <module>
    from awscli.customizations.sso.utils import do_sso_login
  File "/nix/store/d0kb0gnkzzxmd307a90wdxvpdnqyv23m-awscli2-2.0.54/lib/python3.8/site-packages/awscli/customizations/sso/utils.py", line 17, in <module>
    from botocore.utils import SSOTokenFetcher
ImportError: cannot import name 'SSOTokenFetcher' from 'botocore.utils' (/nix/store/db2y4ykpmvakq0nlxyn1d8iy89k8zmqx-python3.8-botocore-1.18.9/lib/python3.8/site-packages/botocore/utils.py)

Expected behavior
Flipping the order:
nix-shell -p awscli2 aws-sam-cli --run "aws --version"
output the version.

SAM cli works in both case.

Additional context
I tested with various nixpkgs on unstable on osx. I can't test the latest unstable on osx yet.
I'll test it on linux later tonight if I don't find the fix.

Notify maintainers

aws-sam-cli:
@andreabedini
@lo1tuma

awscli2:
@bhipple
@davegallant

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

I ran with:

 - system: `"x86_64-darwin"`
 - host os: `Darwin 19.6.0, macOS 10.15.7`
 - multi-user?: `no`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.3.8`
 - channels(joe.dupuis): `"nixpkgs-21.03pre249162.1dc37370c48, darwin"`
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
 - nixpkgs: `/Users/joe.dupuis/.nix-defexpr/channels/nixpkgs

and this pin:
https://github.com/nixos/nixpkgs/archive/bdbcfb436060f46409a585c78335b1f75646695b.tar.gz

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
  - awscli2
  - aws-sam-cli 
# a list of nixos modules affected by the problem
module:

[davegallant]

It seems there is only a single $PYTHONPATH and ordering matters.

#10597 (comment)

$ nix-shell -p aws-sam-cli awscli2 --run 'echo $PYTHONPATH'

[davegallant]

Unsetting PYTHONPATH appears to solve this issue:

$ nix-shell aws-sam-cli awscli2 --run 'unset PYTHONPATH; aws --version'
aws-cli/2.1.17 Python/3.8.6 Linux/5.10.5 source/x86_64.nixos.21 prompt/off

[peterromfeldhk]

same issue :(

⋊> nix-shell -I nixpkgs=https://github.com/nixos/nixpkgs/archive/e14843556cb4ed2286c6752a32abf85beec0e19c.tar.gz -p python39 python39Packages.boto3 awscli2 --run "aws --version"
aws-cli/2.2.4 Python/3.8.9 Darwin/20.5.0 source/x86_64 prompt/off
⋊> nix-shell -I nixpkgs=https://github.com/nixos/nixpkgs/archive/e14843556cb4ed2286c6752a32abf85beec0e19c.tar.gz -p awscli2 python39 python39Packages.boto3 --run "aws --version"

cannot import name 'SSOTokenFetcher' from 'botocore.utils' (/nix/store/w2vpzhgiam1jh5r73z15m34gjhf0m1z7-python3.9-botocore-1.20.52/lib/python3.9/site-packages/botocore/utils.py)

nix-info:

⋊> nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-darwin"`
 - host os: `Darwin 20.5.0, macOS 10.16`
 - multi-user?: `no`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.4pre20210503_6d2553a`
 - channels(peter.romfeld): `"nixpkgs-21.11pre292293.84aa23742f6"`
 - nixpkgs: `/Users/peter.romfeld/.nix-defexpr/channels/nixpkgs`

[smarkin-repository]

Hi, I met the same issues, if somebody needs fast fix, try to remove boto3 and botocore from pip repository.

[bendlas]

I'm seeing a similar issue when combining aws-adfs with awscli2:

± nix-shell -p awscli2 -p python38Packages.aws-adfs
$ aws
...
ImportError: cannot import name 'resolve_imds_endpoint_mode' from 'botocore.utils'

Quickfix for me:

pkgs.writeScriptBin "aws" ''
  #! /usr/bin/env nix-shell
  #! nix-shell --pure -i bash -p awscli2
  exec aws "$@"
''

[voidus]

Quickfix for me:

pkgs.writeScriptBin "aws" ''
  #! /usr/bin/env nix-shell
  #! nix-shell --pure -i bash -p awscli2
  exec aws "$@"
''

I tried this while using aws-vault to assume roles, and got credentials errors, so I assume that aws-vault uses environment variables, which are blocked by nix-shell --pure. (Just as a warning to others because I wasted a few hours on this 😅)

[stale]

I marked this as stale due to inactivity. → More info

[vamega]

I'm not seeing this anymore.

Tested with nixpkgs at 3ad64d9e2d5, not on nixos.

[Silic0nS0ldier]

This sounds like an issue I've seen with using awscli2 and python39 in the same mkShell call (on commit c8bc2f2).

e.g.

# ...
with nixpkgs;
mkShell {
  buildInputs = [
    awscli2
    python39
  ];
}

The observed result is that Python 3.10 is resolved in which python3 because awscli2 comes first. The workaround used in my case was to force awscli2 to use the same Python version.

# ...
with nixpkgs;
mkShell {
  buildInputs = [
    (awscli2.override { python3 = python39; })
    python39
  ];
}

Ideally this "PATH leakage" wouldn't be occurring at all. As others have pointed out, PYTHONPATH is also involved with the specific issue reported.

[JTarasovic]

I just ran into what I believe is the same thing with ansible and awscli2. Having ansible listed first in mkShell causes aws to barf.

nix-shell -p ansible -p awscli2 --command aws
Traceback (most recent call last):
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/bin/.aws-wrapped", line 20, in <module>
    import awscli.clidriver
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/clidriver.py", line 21, in <module>
    import botocore.session
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/session.py", line 27, in <module>
    import botocore.client
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/client.py", line 16, in <module>
    from botocore import UNSIGNED, waiter, xform_name
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/waiter.py", line 17, in <module>
    from botocore.docs.docstring import WaiterDocstring
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/__init__.py", line 15, in <module>
    from botocore.docs.service import ServiceDocumenter
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/service.py", line 14, in <module>
    from botocore.docs.client import ClientDocumenter, ClientExceptionsDocumenter
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/client.py", line 16, in <module>
    from botocore.docs.example import ResponseExampleDocumenter
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/example.py", line 13, in <module>
    from botocore.docs.shape import ShapeDocumenter
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/docs/shape.py", line 19, in <module>
    from botocore.utils import is_json_value_header
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/utils.py", line 35, in <module>
    import botocore.httpsession
  File "/nix/store/y8qprb05wn6y1r6p29bygr48zd9hvm3j-awscli2-2.15.0/lib/python3.11/site-packages/awscli/botocore/httpsession.py", line 21, in <module>
    from urllib3.util.ssl_ import (
ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/nix/store/ram7ivmskyhmanjdkx92l408y8x7facd-python3.11-urllib3-2.0.7/lib/python3.11/site-packages/urllib3/util/ssl_.py)

Switching the order fixes aws but probably upsets ansible.

nix-shell -p awscli2 -p ansible --command aws

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help

aws: error: the following arguments are required: command

What's strange to me is that the urllib3 path from the error is for the wrong version but the path in the wrapper is correct. I don't have enough python / nix knowledge to speculate or postulate why that might be.