Adding rdbms-connect fails in Azure CLI version 2.54.0

[krystian-w-fabres]

Describe the bug

Adding the extension rdbms-connect fails in an Azure CLI task in a release pipeline.

Related command

az extension add --name rdbms-connect --version 1.0.3

Errors

2023-11-21T09:19:22.5964957Z ERROR: cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
2023-11-21T09:19:22.5966400Z ERROR: az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.

Issue script & Debug output

2023-11-21T09:19:22.5900051Z Building wheels for collected packages: setproctitle
2023-11-21T09:19:22.5900776Z   Created temporary directory: C:\Users\VssAdministrator\AppData\Local\Temp\pip-wheel-_r4cueft
2023-11-21T09:19:22.5901647Z   Destination directory: C:\Users\VssAdministrator\AppData\Local\Temp\pip-wheel-_r4cueft
2023-11-21T09:19:22.5902353Z   Building wheel for setproctitle (pyproject.toml): started
2023-11-21T09:19:22.5903296Z   Running command Building wheel for setproctitle (pyproject.toml)
2023-11-21T09:19:22.5903768Z   running bdist_wheel
2023-11-21T09:19:22.5904006Z   running build
2023-11-21T09:19:22.5904223Z   running build_ext
2023-11-21T09:19:22.5904452Z   building 'setproctitle' extension
2023-11-21T09:19:22.5904764Z   creating build
2023-11-21T09:19:22.5905017Z   creating build\temp.win-amd64-cpython-311
2023-11-21T09:19:22.5905430Z   creating build\temp.win-amd64-cpython-311\Release
2023-11-21T09:19:22.5905892Z   creating build\temp.win-amd64-cpython-311\Release\src
2023-11-21T09:19:22.5911710Z   "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -DSPT_VERSION=1.2.3 "-IC:\Program Files\Microsoft SDKs\Azure\CLI2\include" "-IC:\Program Files\Microsoft SDKs\Azure\CLI2\Include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.29.30133\include" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\cppwinrt" /Tcsrc/setproctitle.c /Fobuild\temp.win-amd64-cpython-311\Release\src/setproctitle.obj
2023-11-21T09:19:22.5917701Z   setproctitle.c
2023-11-21T09:19:22.5918946Z   C:\Users\VssAdministrator\AppData\Local\Temp\pip-install-o4_pqwpi\setproctitle_cd0f4ebe1aef4cdebbbdcb122e6f9a65\src\spt_python.h(15): fatal error C1083: Cannot open include file: 'Python.h': No such file or directory
2023-11-21T09:19:22.5921023Z   error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\VC\\Tools\\MSVC\\14.29.30133\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
2023-11-21T09:19:22.5922050Z   error: subprocess-exited-with-error
2023-11-21T09:19:22.5922376Z   
2023-11-21T09:19:22.5922740Z   Building wheel for setproctitle (pyproject.toml) did not run successfully.
2023-11-21T09:19:22.5923279Z   exit code: 1
2023-11-21T09:19:22.5923610Z   
2023-11-21T09:19:22.5923789Z   See above for output.
2023-11-21T09:19:22.5924025Z   
2023-11-21T09:19:22.5924431Z   note: This error originates from a subprocess, and is likely not a problem with pip.
2023-11-21T09:19:22.5926170Z   full command: 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe' 'C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py' build_wheel 'C:\Users\VSSADM~1\AppData\Local\Temp\tmphjkyy0yp'
2023-11-21T09:19:22.5927973Z   cwd: C:\Users\VssAdministrator\AppData\Local\Temp\pip-install-o4_pqwpi\setproctitle_cd0f4ebe1aef4cdebbbdcb122e6f9a65
2023-11-21T09:19:22.5928900Z   Building wheel for setproctitle (pyproject.toml): finished with status 'error'
2023-11-21T09:19:22.5929476Z   ERROR: Failed building wheel for setproctitle
2023-11-21T09:19:22.5929840Z Failed to build setproctitle
2023-11-21T09:19:22.5930443Z ERROR: Could not build wheels for setproctitle, which is required to install pyproject.toml-based projects
2023-11-21T09:19:22.5931106Z Exception information:
2023-11-21T09:19:22.5931342Z Traceback (most recent call last):
2023-11-21T09:19:22.5932137Z   File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\base_command.py", line 180, in exc_logging_wrapper
2023-11-21T09:19:22.5932961Z     status = run_func(*args)
2023-11-21T09:19:22.5933234Z              ^^^^^^^^^^^^^^^
2023-11-21T09:19:22.5933910Z   File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\cli\req_command.py", line 245, in wrapper
2023-11-21T09:19:22.5934672Z     return func(self, options, args)
2023-11-21T09:19:22.5934971Z            ^^^^^^^^^^^^^^^^^^^^^^^^^
2023-11-21T09:19:22.5935651Z   File "C:\Program Files\Microsoft SDKs\Azure\CLI2\Lib\site-packages\pip\_internal\commands\install.py", line 429, in run
2023-11-21T09:19:22.5936387Z     raise InstallationError(
2023-11-21T09:19:22.5937270Z pip._internal.exceptions.InstallationError: Could not build wheels for setproctitle, which is required to install pyproject.toml-based projects
2023-11-21T09:19:22.5938365Z Removed build tracker: 'C:\\Users\\VssAdministrator\\AppData\\Local\\Temp\\pip-build-tracker-xumup4t6'
2023-11-21T09:19:22.5938838Z 
2023-11-21T09:19:22.5940795Z DEBUG: cli.azure.cli.core.extension.operations: Command '['C:\\Program Files\\Microsoft SDKs\\Azure\\CLI2\\python.exe', '-m', 'pip', 'install', '--target', 'C:\\Program Files\\Common Files\\AzureCliExtensionDirectory\\rdbms-connect', 'C:\\Users\\VSSADM~1\\AppData\\Local\\Temp\\tmpe66gnbk1\\rdbms_connect-1.0.3-py2.py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
2023-11-21T09:19:22.5943550Z DEBUG: cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at C:\Program Files\Common Files\AzureCliExtensionDirectory\rdbms-connect
2023-11-21T09:19:22.5944844Z DEBUG: cli.azure.cli.core.azclierror: Traceback (most recent call last):
2023-11-21T09:19:22.5945922Z   File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
2023-11-21T09:19:22.5947273Z   File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 663, in execute
2023-11-21T09:19:22.5950355Z   File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
2023-11-21T09:19:22.5954470Z   File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 697, in _run_job
2023-11-21T09:19:22.5956037Z   File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/__init__.py", line 333, in __call__
2023-11-21T09:19:22.5957467Z   File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
2023-11-21T09:19:22.5959088Z   File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
2023-11-21T09:19:22.5960673Z   File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 344, in add_extension
2023-11-21T09:19:22.5962440Z   File "D:\a\_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/extension/operations.py", line 173, in _add_whl_ext
2023-11-21T09:19:22.5963739Z knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.
2023-11-21T09:19:22.5964326Z 
2023-11-21T09:19:22.5964957Z ERROR: cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
2023-11-21T09:19:22.5966400Z ERROR: az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.

Expected behavior

Extension should add without fail.

Environment Summary

azure-cli                         2.54.0
core                              2.54.0
telemetry                          1.1.0

Extensions:
azure-devops                      0.26.0

Dependencies:
msal                            1.24.0b2
azure-mgmt-resource             23.1.0b2

Additional context

The command ran fine with Azure CLI version 2.53.1. The error seems the simillar to those described here:
#25869
#25067
However the workarounds provided either don't work (install setuptools) or were already present (specify version of rdbms-connect). Our hotfix was to install a previous version of Azure CLI before running any commands: python -m pip install azure-cli==2.50.0 (trying to install version 2.53.1 returned an error message that the newest available version was 2.50.0 for some reason?).
This is not something we would like to have in our pipeline obviously, also it increases the time it takes to run the pipeline by around 5 minutes.

[azure-client-tools-bot-prd]

Hi @krystian-w-fabres
Find similar issue #25188.

Issue title	rdbms-connect install fails with pip status code 2
Create time	2023-01-20
Comment number	5

Possible solution:
The issue is that the rdbms-connect extension fails to install with pip status code 2. The error message indicates that the module poetry was not found. Manually installing poetry with pip install poetry fixed the bug.

However, it seems that this issue is not reproducible on all environments. One user reported that they were able to install the extension without error on MacOS 13.1 ARM64, and they did not have poetry installed in Python.

Therefore, if you encounter this issue, you can try installing poetry with pip install poetry as a workaround.

---

Please confirm if this resolves your issue.

[yonzhan]

Thank you for opening this issue, we will look into it.

[bebound]

The bundled Python is bumped to 3.11 in 2.54.0.
rdbms-connect 1.0.3 requires setproctitle (~=1.2.2), which does not provide a wheel for python 3.11.
I think it should update the dependency to setproctitle~=1.3.3.

I can't find a workaround for this.

@alanenriqueo for awareness. rdbms-connect does not work on Windows.

[ambrahma]

The bundled Python is bumped to 3.11 in 2.54.0. rdbms-connect 1.0.3 requires setproctitle (~=1.2.2), which does not provide a wheel for python 3.11. I think it should update the dependency to setproctitle~=1.3.3.

I can't find a workaround for this.

@alanenriqueo for awareness. rdbms-connect does not work on Windows.

Fix for this Azure/azure-cli-extensions#7138, is currently blocked due to dbcli/mycli#1139.

[JimacoMS4]

@ambrahma Is this still blocked as noted above? I'm experiencing the same issue with Azure CLI 2.59.0 (on Windows). I'm not specifying a version in the command.

[ambrahma]

@ambrahma Is this still blocked as noted above? I'm experiencing the same issue with Azure CLI 2.59.0 (on Windows). I'm not specifying a version in the command.

My PR to fix this has been checked-in Azure/azure-cli-extensions#7138, and will be released as part of upcoming milestone

[julia-gb]

The issue is present in:
azure-cli 2.61.0

core 2.61.0
telemetry 1.1.0

Dependencies:
msal 1.28.0
azure-mgmt-resource 23.1.1

on macos

[Krumelur]

I guess I'm facing a related problem. Windows 11, az 2.62.0

az extension add --name azure-devops
An error occurred. Pip failed with status code 1. Use --debug for more information.