Homebrew: "Python location" not pointing to a virtual environment #4573

chrmarti · 2017-09-28T21:29:33Z

Description

"Python location" was added to the output of --version for the VSCode extension to pick up and use as the entry point for loading the command tree, documentation and completion proposals (#2986). That depends on "Python location" pointing to a virtual environment with all the CLI's modules in it. When installing on Mac through Homebrew that is not the case and the VSCode extension does not work.

Environment summary

Install Method: How did you install the CLI? (e.g. pip, interactive script, apt-get, Docker, MSI, nightly)
Answer here: Homebrew on Mac

CLI Version: What version of the CLI and modules are installed? (Use az --version)
Answer here: 2.0.18

OS Version: What OS and version are you using?
Answer here: Mac OS Sierra

Shell Type: What shell are you using? (e.g. bash, cmd.exe, Bash on Windows)
Answer here: zsh

/cc @derekbekoe @chrisdias

The text was updated successfully, but these errors were encountered:

chrmarti · 2017-09-28T21:31:25Z

The output:

az --version
azure-cli (2.0.18)

acr (2.0.12)
acs (2.0.16)
appservice (0.1.17)
backup (1.0.1)
batch (3.1.4)
billing (0.1.5)
cdn (0.0.9)
cloud (2.0.8)
cognitiveservices (0.1.8)
command-modules-nspkg (2.0.1)
component (2.0.7)
configure (2.0.11)
consumption (0.1.5)
container (0.1.11)
core (2.0.17)
cosmosdb (0.1.13)
dla (0.0.12)
dls (0.0.14)
eventgrid (0.1.4)
extension (0.0.3)
feedback (2.0.6)
find (0.2.7)
interactive (0.3.10)
iot (0.1.12)
keyvault (2.0.11)
lab (0.0.11)
monitor (0.0.10)
network (2.0.15)
nspkg (3.0.1)
profile (2.0.13)
rdbms (0.0.7)
redis (0.2.9)
resource (2.0.15)
role (2.0.12)
servicefabric (0.0.4)
sql (2.0.12)
storage (2.0.16)
vm (2.0.15)

Python location '/usr/local/bin/python3'
Extensions directory '/Users/chrmarti/.azure/cliextensions'

Python (Darwin) 3.5.2 (v3.5.2:4def2a2901a5, Jun 26 2016, 10:47:25)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]

Legal docs and information: aka.ms/AzureCliLegal

derekbekoe · 2017-09-28T21:47:57Z

Based on feedback in my Homebrew PR, the Homebrew install does not use a virtual environment.
Homebrew/homebrew-core#17344 (comment)

Instead, the PYTHONPATH is set to the correct site-packages directory manually (which is what the virtual env would have done).

$ cat $(which az)
#!/usr/bin/env bash
export PYTHONPATH="/usr/local/Cellar/azure-cli/2.0.17/libexec/lib/python3.6/site-packages"
python3.6 -m azure.cli "$@"

chrmarti · 2017-09-28T22:24:04Z

Not sure how to work around this from our side. Maybe you could add "Python path: '/usr/local/Cellar/.../site-packages'" to the output of --version?

derekbekoe · 2017-09-28T22:38:55Z

That would return a list of values rather than a single one.
e.g.:

Python path '['', '/usr/local/Cellar/azure-cli/2.0.17/libexec/lib/python3.6/site-packages', '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python36.zip', '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6', '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/site-packages']'

If you add the correct path to your PYTHONPATH, does the VSCode extension work as normal?

chrmarti · 2017-09-28T22:43:13Z

It complains ImportError: No module named '_cffi_backend':

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/chrmarti/Development/repos/vscode-azurecli/service/azservice/__main__.py", line 338, in <module>
    main()
  File "/Users/chrmarti/Development/repos/vscode-azurecli/service/azservice/__main__.py", line 304, in main
    command_table = load_command_table()
  File "/Users/chrmarti/Development/repos/vscode-azurecli/service/azservice/tooling.py", line 67, in load_command_table
    _install_modules(command_table)
  File "/Users/chrmarti/Development/repos/vscode-azurecli/service/azservice/tooling.py", line 73, in _install_modules
    command_table[cmd].load_arguments()
  File "/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 335, in load_arguments
    self.arguments.update(self.arguments_loader())
  File "/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 653, in arguments_loader
    return extract_args_from_signature(get_op_handler(operation), no_wait_param=no_wait_param)
  File "/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 537, in get_op_handler
    op = import_module(mod_to_import)
  File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 986, in _gcd_import
  File "<frozen importlib._bootstrap>", line 969, in _find_and_load
  File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 665, in exec_module
  File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
  File "/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/azure/cli/command_modules/acs/custom.py", line 35, in <module>
    from azure.cli.command_modules.acs import acs_client, proxy
  File "/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/azure/cli/command_modules/acs/acs_client.py", line 11, in <module>
    import paramiko
  File "/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/paramiko/__init__.py", line 31, in <module>
    from paramiko.transport import SecurityOptions, Transport
  File "/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/paramiko/transport.py", line 57, in <module>
    from paramiko.ed25519key import Ed25519Key
  File "/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/paramiko/ed25519key.py", line 17, in <module>
    import bcrypt
  File "/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/bcrypt/__init__.py", line 25, in <module>
    from bcrypt import _bcrypt
ImportError: No module named '_cffi_backend'

Still trying to figure out how to get around it.

chrmarti · 2017-09-28T23:44:55Z

@derekbekoe I see /usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages/_cffi_backend.cpython-36m-darwin.so and PYTHONPATH has that folder listed. Should that be enough to load this module or can I check anything else?

derekbekoe · 2017-09-29T21:16:34Z

@chrmarti Did the VS Code extension load and run properly after adding the Python path?

chrmarti · 2017-09-29T21:40:07Z

@derekbekoe No, it shows the above error about not finding _cffi_backend (#4573 (comment)).

derekbekoe · 2017-09-29T22:34:47Z

What version of Python are you using? I see 3.5/lib/python3.5 in the error message.
Can you try Python 3.6?

Really just running this should work as that's what az does:

export PYTHONPATH="/usr/local/Cellar/azure-cli/2.0.18/libexec/lib/python3.6/site-packages"
python3.6 -m azure.cli "$@"

chrmarti · 2017-09-29T23:04:28Z

You're right, the 'Python location' points to Python 3.5. It works with Python 3.6. python -m azure.cli works with both Python 3.5. and 3.6 when run with no parameters.

chrmarti · 2017-10-11T16:24:50Z

@derekbekoe Anything else I should try? Is the homebrew package installing 3.6 packages when using the installed python 3.5 at fault?

derekbekoe · 2017-10-13T17:14:12Z

Is the homebrew package installing 3.6 packages when using the installed python 3.5 at fault?

Yes it does.
It attempts to run python3.6 and if that doesn't exist on PATH, falls back to python3.

chrmarti · 2017-10-13T21:13:40Z

That is causing the problem then? I'm trying to understand what I can do.

JetPac33 · 2017-11-13T19:00:05Z

Had a similar problem with homebrew installed az. Python 3.6 was installed but not linked correctly by homebrew. The problem was fixed by doing brew link --overwrite python3

derekbekoe · 2018-03-06T19:46:46Z

I will submit a PR to https://github.com/Microsoft/vscode-azurecli with a fix shortly.

derekbekoe added Packaging Packaging/Homebrew labels Sep 28, 2017

chrmarti mentioned this issue Nov 15, 2017

Does not work with the Homebrew install of the Azure CLI microsoft/vscode-azurecli#25

Closed

derekbekoe closed this as completed Mar 6, 2018

haroldrandom added Packaging Packaging/Homebrew labels Oct 25, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Homebrew: "Python location" not pointing to a virtual environment #4573

Homebrew: "Python location" not pointing to a virtual environment #4573

chrmarti commented Sep 28, 2017

chrmarti commented Sep 28, 2017

derekbekoe commented Sep 28, 2017

chrmarti commented Sep 28, 2017

derekbekoe commented Sep 28, 2017

chrmarti commented Sep 28, 2017

chrmarti commented Sep 28, 2017

derekbekoe commented Sep 29, 2017 •

edited

chrmarti commented Sep 29, 2017

derekbekoe commented Sep 29, 2017 •

edited

chrmarti commented Sep 29, 2017

chrmarti commented Oct 11, 2017

derekbekoe commented Oct 13, 2017

chrmarti commented Oct 13, 2017

JetPac33 commented Nov 13, 2017

derekbekoe commented Mar 6, 2018

Homebrew: "Python location" not pointing to a virtual environment #4573

Homebrew: "Python location" not pointing to a virtual environment #4573

Comments

chrmarti commented Sep 28, 2017

Description

Environment summary

chrmarti commented Sep 28, 2017

derekbekoe commented Sep 28, 2017

chrmarti commented Sep 28, 2017

derekbekoe commented Sep 28, 2017

chrmarti commented Sep 28, 2017

chrmarti commented Sep 28, 2017

derekbekoe commented Sep 29, 2017 • edited

chrmarti commented Sep 29, 2017

derekbekoe commented Sep 29, 2017 • edited

chrmarti commented Sep 29, 2017

chrmarti commented Oct 11, 2017

derekbekoe commented Oct 13, 2017

chrmarti commented Oct 13, 2017

JetPac33 commented Nov 13, 2017

derekbekoe commented Mar 6, 2018

derekbekoe commented Sep 29, 2017 •

edited

derekbekoe commented Sep 29, 2017 •

edited