AZ CLI failing with Pip error on extension install.

[pkoushik]

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug

Az CLI version 2.18 fails with Pip error on extension install. (e.g. az extension add --source <path to .whl>)

cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.

To Reproduce

az extension add --source <path to .whl>
Expected behavior

No failure
Environment summary

Az CLI Version: 2.18
MacOS, Zsh
Additional context

[chjinche]

Also met the same error when adding extension. Workaround by upgrading pip version in az python path, but better to be solved by az cli.

See the below debug logs:
....
error: Can not find Rust compiler
Running setup.py install for cryptography: finished with status 'error'
Cleaning up...
Removing source in /tmp/pip-install-87ax_zl5/cryptography
Removing source in /tmp/pip-install-87ax_zl5/psutil
Removed build tracker '/tmp/pip-req-tracker-8c1b1o8u'
Command "/opt/az/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-install-87ax_zl5/cryptography/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file
__, 'exec'))" install --record /tmp/pip-record-slbdhf8x/install-record.txt --single-version-externally-managed --compile --home=/tmp/pip-target-1wo4hi5b" failed with error code 1 in /tmp/pip-install-87ax_zl5/cryptography/
Exception information:
Traceback (most recent call last):
File "/opt/az/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 143, in main
status = self.run(options, args)
File "/opt/az/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 366, in run
use_user_site=options.use_user_site,
File "/opt/az/lib/python3.6/site-packages/pip/_internal/req/init.py", line 49, in install_given_reqs
**kwargs
File "/opt/az/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 791, in install
spinner=spinner,
File "/opt/az/lib/python3.6/site-packages/pip/_internal/utils/misc.py", line 705, in call_subprocess
% (command_desc, proc.returncode, cwd))
pip._internal.exceptions.InstallationError: Command "/opt/az/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-install-87ax_zl5/cryptography/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n
', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-slbdhf8x/install-record.txt --single-version-externally-managed --compile --home=/tmp/pip-target-1wo4hi5b" failed with error code 1 in /tmp/pip-ins
tall-87ax_zl5/cryptography/

cli.azure.cli.core.extension.operations: Command '['/opt/az/bin/python3', '-m', 'pip', 'install', '--target', '/home/aml-custom-module/.azure/cliextensions/azure-cli-ml', '/tmp/tmpgl54fk79/azure_cli_ml-0.1.0.29211468-py3-none-any.whl', '-
-extra-index-url', 'https://azuremlsdktestpypi.azureedge.net/modulesdkpreview', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at /home/aml-custom-module/.azure/cliextensions/azure-cli-ml
cli.azure.cli.core.util: azure.cli.core.util.handle_exception is called with an exception:
cli.azure.cli.core.util: Traceback (most recent call last):
File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 664, in execute
raise ex
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 727, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 720, in _run_job
six.reraise(*sys.exc_info())
File "/opt/az/lib/python3.6/site-packages/six.py", line 703, in reraise
raise value
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 698, in _run_job
result = cmd_copy(params)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/init.py", line 331, in call
return self.handler(*args, **kwargs)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/init.py", line 816, in default_command_handler
return op(**command_args)
File "/opt/az/lib/python3.6/site-packages/azure/cli/command_modules/extension/custom.py", line 18, in add_extension_cmd
version=version, upgrade=upgrade)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/extension/operations.py", line 318, in add_extension
pip_extra_index_urls=pip_extra_index_urls, pip_proxy=pip_proxy, system=system)
File "/opt/az/lib/python3.6/site-packages/azure/cli/core/extension/operations.py", line 163, in _add_whl_ext
'Use --debug for more information.'.format(pip_status_code))
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

[yungezz]

HI @fengzhou-msft how's the cryptograpy upgrading status?

[mayurigupta13]

What's the workaround for this issue?

[chjinche]

What's the workaround for this issue?

@mayurigupta13 one way to workaround is upgrading pip version before az extension add, like this:

cd /opt/az/bin/
./python3 -m pip install --upgrade pip --user

[pkoushik]

@fengzhou-msft can we get an update regarding the issue?

[fengzhou-msft]

We've pinned the cryptography in azure-cli in #15687, but it won't fix the installing issue of extensions as the extension will install cryptography again in its own directory and ignoring the cryptography already installed by azure-cli.

The fix needs to be done by the extension to either pin cryptography in their setup.py or remove any dependencies that depends on cryptography if they are already installed by azure-cli.

The latest azure-cli-ml has pinned cryptography to fix installation.

[czofficial]

Is there any workaround for this issue?

Unfortunately, I get the same error message when trying out chjinche's solution:
cd: no such file or directory: /opt/az/bin/
zsh: no such file or directory: ./python3

[czofficial]

Is there any workaround for this issue?

Unfortunately, I get the same error message when trying out chjinche's solution:
cd: no such file or directory: /opt/az/bin/
zsh: no such file or directory: ./python3

Found the issue. I screened my debug log again and found the reference that my pip version was too old:
/usr/local/Cellar/azure-cli/2.20.0/libexec/bin/python -m pip install --upgrade pip
helped in my case.