Unable to install aws-cli via pip - package dependency broke

[AMO-brandon]

Describe the bug

$ pip3 install awscli
Collecting awscli
Obtaining dependency information for awscli from https://files.pythonhosted.org/packages/43/9c/bf16d97f5de8aa4f9171c6c82bad0b4179921ddca5066ba980358da0e9a5/awscli-1.29.3-py3-none-any.whl.metadata
Downloading awscli-1.29.3-py3-none-any.whl.metadata (11 kB)
Collecting botocore==1.31.3 (from awscli)
Obtaining dependency information for botocore==1.31.3 from https://files.pythonhosted.org/packages/b0/f0/5755508b3305534cd4cf2a8a82bbbe42ee9d66fd2688be5ff3dfb85e9a99/botocore-1.31.3-py3-none-any.whl.metadata
Downloading botocore-1.31.3-py3-none-any.whl.metadata (5.9 kB)
Collecting docutils<0.17,>=0.10 (from awscli)
Downloading docutils-0.16-py2.py3-none-any.whl (548 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 548.2/548.2 kB 34.8 MB/s eta 0:00:00
Collecting s3transfer<0.7.0,>=0.6.0 (from awscli)
Downloading s3transfer-0.6.1-py3-none-any.whl (79 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 79.8/79.8 kB 12.2 MB/s eta 0:00:00
Collecting PyYAML<5.5,>=3.10 (from awscli)
Downloading PyYAML-5.4.1.tar.gz (175 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.1/175.1 kB 24.9 MB/s eta 0:00:00
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'error'
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [62 lines of output]
running egg_info
writing lib3/PyYAML.egg-info/PKG-INFO
writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
writing top-level names to lib3/PyYAML.egg-info/top_level.txt
/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in setup.cfg
!!

          ********************************************************************************
          The license_file parameter is deprecated, use license_files instead.
  
          By 2023-Oct-30, you need to update your project and remove deprecated calls
          or your builds will no longer be supported.
  
          See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
          ********************************************************************************
  
  !!
    parsed = self.parsers.get(option_name, lambda x: x)(value)
  Traceback (most recent call last):
    File "/usr/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/usr/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/usr/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 341, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 323, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 338, in run_setup
      exec(code, locals())
    File "<string>", line 271, in <module>
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 107, in setup
      return distutils.core.setup(**attrs)
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
      return run_commands(dist)
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
      dist.run_commands()
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
      self.run_command(cmd)
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1234, in run_command
      super().run_command(command)
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 314, in run
      self.find_sources()
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 322, in find_sources
      mm.run()
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 551, in run
      self.add_defaults()
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 5[89](https://gitlab.amopportunities.org/amopportunities/api/-/jobs/219794#L89), in add_defaults
      sdist.add_defaults(self)
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/command/sdist.py", line [104](https://gitlab.amopportunities.org/amopportunities/api/-/jobs/219794#L104), in add_defaults
      super().add_defaults()
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 251, in add_defaults
      self._add_defaults_ext()
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/sdist.py", line 336, in _add_defaults_ext
      self.filelist.extend(build_ext.get_source_files())
    File "<string>", line 201, in get_source_files
    File "/tmp/pip-build-env-vkrpwdkj/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line [107](https://gitlab.amopportunities.org/amopportunities/api/-/jobs/219794#L107), in __getattr__
      raise AttributeError(attr)
  AttributeError: cython_sources
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.

Expected Behavior

Install AWS cli

Current Behavior

errors out with dependency issues

Reproduction Steps

pip3 install awscli

Possible Solution

No response

Additional Information/Context

No response

CLI version used

1.29 (latest?)

Environment details (OS name and version, etc.)

linux gitlab runner

[saidsef]

I'm experiencing the same issue, suddenly it started failing with that error

[TobiasDahm]

I have the same error while deploying my app via GitLab pipeline to AWS!

[acamilleri]

Hi, same issue here, what version of setuptools you have ?

[pwo3]

Hi, same issue here with Python 3.11. Did you find any workaround?

[domhall]

yaml/pyyaml#601
Related thread/underlying cause

Also same issue here. Not sure what the best path forward is. Maybe can force an older version of cython when building pyyaml or wait for them to fix

[stianjensen]

I'm switching to installing awscli via apt-get instead of pip now.

#4947

[saidsef]

For GitLab runner, I've switched to apk add aws-cli

[armandofcom]

there is also the problem, and actually updating more than 800 gitlabci is not an option

[leadrTWilson]

My team is having this issue as well. following for updates.

[igarcia-nz]

Also moving to Alpine package, via apk add aws-cli

[fbarrerafalabella]

also moving to apk add worked

For GitLab runner, I've switched to apk add aws-cli

[2andrew]

I had the same problem when running pip install awscli and adding this line helped me:
pip3 install pyyaml==5.3.1

[hssyoo]

Hi everyone, thanks for reporting the issue and workarounds you've found. The AWS CLI team has been actively looking into the issue and discussing potential workarounds and long-term solutions.

Our recommended solution to unblock workflows is to upgrade to the AWS CLI v2. We understand that that's not straightforward for some users so if you need to stay on the AWS CLI v1, then using the --no-build-isolation flag should be successful. eg
pip install awscli --no-build-isolation

We do not recommend installing an older version of PyYAML as PyYAML version 5.3.1 is associated with CVE-2020-14343 that was fixed in version 5.4.

[lantrix]

Thanks - for the quick response @hssyoo

Given we don’t know the impact of replacing the AWS CLI v1 with V2 in bulk on these legacy system, I’ve tested the --no-build-isolation in a test 2.7 python environment and confirmed it now installs.

Thanks for the workaround.

[hssyoo]

Hi everyone, thanks for your patience.

We merged #8037 which extends our support for PyYAML 6.0. This is a partial solution as PyYAML 6.0 has wheels available for Python 3.10 and 3.11. This will not resolve issues for users on a platform with no PyYAML wheels (eg Alpine). You can see the list of available wheels for PyYAML 6.0 here: https://pypi.org/project/PyYAML/6.0/#files

Another immediate workaround for users that need to stay on AWS CLI v1 is to download and install our bundled installer.

[jacksonmelcher]

Fix for users using Alpine here

Our workaround is:

pip install wheel
pip install "Cython<3.0" "pyyaml<6" --no-build-isolation
pip3 install awscli

[IamAmitE]

Fix: pip3 install pyyaml==5.3.1 in the virtualenv, and remove lines 556-557 (def _install_ebcli()) in ebcli_installer.py:

            '--upgrade',
            '--upgrade-strategy', 'eager',

[joachimmartin]

Fix: pip3 install pyyaml==5.3.1 in the virtualenv

Hi @IamAmitE do you have any thoughts on how can I do this in a CI/CD env? I am installing ebcli for each build and I'm not sure how to specify pyyaml version in the virtualenv that ebcli_installer.py is setting up.

I'm just running:

git clone https://github.com/aws/aws-elastic-beanstalk-cli-setup.git
python ./aws-elastic-beanstalk-cli-setup/scripts/ebcli_installer.py

And getting the same AttributeError: cython_sources error as this ticket references.

[gdziuba]

@joachimmartin

I ran into the same issue. When you first run

python ./aws-elastic-beanstalk-cli-setup/scripts/ebcli_installer.py

I saw where it installed it. For me on Windows, it is installed in the following directory.

C:\Users\<<user>>\.ebcli-virtual-env

run: (May differ based off of OS)
\.ebcli-virtual-env\Scripts\Activate pip3 install pyyaml==5.3.1

Then run the install after removing the code mentioned by @IamAmitE .

[bakatz]

The above didn't work for me. The only thing that did was to edit the eb installer script and add --no-cache-dir to the pip command (needs to be placed right after install to work). The problem is that Ubuntu has its own version of pyyaml already installed (5.4.1) which cannot be uninstalled easily and will always be chosen unless you tell python to stop using the cache dir. This holds true even if you install 5.3.1 in the virtualenv (you still need to do that, just saying that installing 5.3.1 in the virtualenv on its own won't fix the issue on the latest version of Ubuntu). I hope AWS will fix this package soon to work properly instead of requiring this kind of workaround.

[IbrohimIsroilov]

pip install awscli --break-system-packages
added --break-system-packages flag and it is working fine