Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Terminals opened in Code adopt malformed $PATH (breaks virtualenv) #2333
Terminals opened with "Pthon: Create Terminal" or "Open New Terminal" command open with a $PATH that has the active virtualenv at the end, rather than the beginning:
Child terminals should have $PATH=
Steps to reproduce:
-In a terminal, I activate a virtualenv, my_env (
-Launch code into this environment
Notably, my virtualenv is in the path, but APPENDED rather than PREPENDED, as it is in both "process.env" and the $PATH of the original calling terminal window.
This leads to all sorts of downstream problems, like auto-opened terminals installing linters and such in the system environment rather than the virtualenv, etc.
referenced this issue
Aug 4, 2018
Thanks for supplying the extra information. We are planning on going over our conda environment logic this release (and you've already been part of that conversation so thanks x2!).
I've reproduced this information locally on Ubuntu 18.04 using a conda 4.4.0/python 3.6.1 version installed via pyenv. I've activated a conda environment in my terminal (called issue2333) and invoked
Here's the terminal's environment:
(issue2333) dekeeler@dek-laptop-lnx:~/dev/github/d3r3kk/test/issue2333$ printenv | grep -i py PYENV_ROOT=/opt/python/pyenv CONDA_PREFIX=/opt/python/pyenv/versions/anaconda3-4.4.0/envs/issue2333 CONDA_PATH_BACKUP=/opt/python/pyenv/shims:/opt/python/pyenv/bin:/home/dekeeler/perl5/bin:/home/dekeeler/.nvm/versions/node/v8.11.2/bin:/home/dekeeler/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/dekeeler/.dotnet/tools PYENV_SHELL=bash PATH=/opt/python/pyenv/versions/anaconda3-4.4.0/envs/issue2333/bin:/opt/python/pyenv/shims:/opt/python/pyenv/bin:/home/dekeeler/perl5/bin:/home/dekeeler/.nvm/versions/node/v8.11.2/bin:/home/dekeeler/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/dekeeler/.dotnet/tools
...and here's the environment reported within Code's interactive terminal:
dekeeler@dek-laptop-lnx:~/dev/github/d3r3kk/test/issue2333$ printenv | grep -i py PYENV_ROOT=/opt/python/pyenvCONDA_PREFIX=/opt/python/pyenv/versions/anaconda3-4.4.0/envs/issue2333 CONDA_PATH_BACKUP=/opt/python/pyenv/shims:/opt/python/pyenv/bin:/home/dekeeler/perl5/bin:/home/dekeeler/.nvm/versions/node/v8.11.2/bin:/home/dekeeler/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/dekeeler/.dotnet/toolsSNAP_COOKIE=6W8xGmk1jyDQ0RF798gtMVr41tm30Y5PylCyhBM3WJfyPYENV_SHELL=bash SNAP_CONTEXT=6W8xGmk1jyDQ0RF798gtMVr41tm30Y5PylCyhBM3WJfy PATH=/opt/python/pyenv/shims:/opt/python/pyenv/bin:/home/dekeeler/perl5/bin:/opt/python/pyenv/versions/anaconda3-4.4.0/envs/issue2333/bin:/opt/python/pyenv/shims:/opt/python/pyenv/bin:/home/dekeeler/perl5/bin:/home/dekeeler/.nvm/versions/node/v8.11.2/bin:/home/dekeeler/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/dekeeler/.dotnet/tools
The path does seem to come from elsewhere (I am confident this is a known issue), and is not aware of the current active conda environment yet.
If I look at the status bar in VSCode, I see this too:
...which tells me we actually do have the right information
When I click on the status bar interpreter, I see the command bar open up with this information:
and as expected, you can see that the current interpreter is just set to
If I select the actual environment I want though (issue2333) and then attempt to open a terminal, everything works as expected.
Oh interesting- you’re saying, if I go to the effort of actually selecting the interpreter that’s already reflected, terminals start working properly with the environment directory out in the front of the PATH? I’ll have to check that tomorrow.
Oh, I didn’t mention - for the sake of clarity, this actually works exactly as expected under Windows, using virtualenv-powershell (and, obviously, powershell)
EDIT: Yes ok, that makes sense - everything gets set up almost-correctly, and if I "Python: Select Interpreter" then terminals get created with the right PATH (or at least, my virtualenv directory prepended AND appended, which seems wrong, but works).
Interesting aside: I'm using a goofy hack for MacOS to fix framework-build-of-python vs the not-framework-build that virtualenv always uses no matter what. This enables matplotlib and other GUI libraries without doing any library-specific workaround. It's the virtualenv-pythonw-osx bit from the middle of pypa/virtualenv#54 (comment)
If I select the Python.app version, the GUI libs work, and if I select the .../bin/python3 version, they don't since it's the not-framework-build. But anyway, with the hack in place, the app version is the one default on my PATH, and code picks that up without intervention as I'd hope (except, still, the terminals don't, per this report)
Actually, this is super odd - now that I revisit this, my entire previous post seems no longer to be true.
That is, in the default case, the default-selected interpreter isn't the correct path. But I think that's probably a trick of symlinks, since the python shell that comes up ultimately is correct.
In a virtualenv:
So at this point, that previous workaround is totally nonfunctional and I can't get code to use the correct python interpreter no matter what I set or try.
@alexwhittemore this whole issue has gotten rather wordy and hard to follow.
Also, not strictly related, but in response to your earlier post (Sorry, looks like we happened to post coincidently!):
What I don't get is, even if this is the case, how is it possible that the $PATH 1:2:3:4:5 becomes 2:3:4:5:1 when VSCode is launched from the terminal? I don't even know how that's possible without deliberate modification, and it seems to me like everything would "just work" the way I intuitively expect it to if that path simply didn't get mangled.
@alexwhittemore yeah, it sounds like your shell setup is a bit too magical for us.
And the terminals are working as expected. I think what you want is #1387 which will activate automatically for any normal terminal versus having to use
(First, thanks for putting up with my walls-of-text, I wish I knew how to make them more concise than they are without leaving out critical info)
I'm actually not so sure about that, and what you mention with issue 1387 actually makes me think there's even MORE going on here out of the ordinary than what we've talked about.
Let me start by clearing up some of the water I muddied in my last couple posts, as I've just done some exhaustive differential diagnosis in a clean macOS virtual machine where there's no way there are any confounding variables.
There are four variables at play here - python 2 vs 3, virtualenvwrapper and thus virtualenv, this fix-osx-virtualenv wrapper, and vscode itself.
Starting with fix-osx-virtualenv: I've verified that it doesn't interfere at all in any of my test cases EXCEPT with a python3 virtualenv, and for a reason that's easily worked-around. It doesn't change the PATH at all - I think it compiles to a Python.app that contains the binary you started with, which is all macOS needs to make all of its GUI APIs work correctly.
What was getting in the way there: you need to call fix-osx-virtualenv to set up the magic Python.app in your virtualenv. The more-robust way to do that is in $WORKON_HOME/postactivate, which gets called every time you activate a virtualenv. Therefore, the check for the magic .app happens EVERY time the environment is activated, in case somehow it hadn't yet. But whatever that command does, it's like 5% broken in python 3.x, and it completes successfully, but it messes up vscode somehow in the case of vscode launched from an activated python3 virtualenv.
The solution to that is to put the magic check in $WORKON_HOME/postmkvirtualenv - only check for the magic Python.app when you make the virtualenv, and that code never runs after that. This doesn't piss off VSCode, since it happens 100% outside of the scope of VSCode.
With that out of the way:
Even though at the terminal,
That part I totally don't understand, but I've established that it's immaterial - pick the Python.app path and all works perfectly.
Okay, so why do I think there is still some actual bug here?
As a test, I did the following at a clean system terminal:
Then, in the integrated terminal:
So without doing ANYTHING else, Code upends my PATH. And I'm convinced, pending evidence, that this mangling is the root cause for why I can't simply
BUT I've got a test of that theory, and simultaneously a workaround!
Basically, I noticed that all the vscode contexts that I think SHOULD still have the virtualenv environment, and which have a mangled PATH, still have VIRTUAL_ENV set from the parent shell. So I figured, any time I open a shell, if that's set but the PATH is wrong, I should make sure the virtualenv in question is re-activated. This could break things in esoteric edge cases where I explicitly want something higher on my PATH than the venv, but otherwise, it achieves my exact desired result.
To test, I create a new project dir (with no .vscode/, no settings.json, no launch.json).
I then add a launch config (python: current file). my_env is already activated on all embedded terminals in Code, and when I click run, everything works as expected, launching the virtualenv python in the virtualenv context. In other words, this little bit of hackery achieves exactly the behavior I see under Windows by simply un-mangling the PATH.
Ok ok ok, so what about the "even more going on"? I'll hop on over to 1387 to chime in, might as well keep things in straight lines!
EDIT: OH, @d3r3kk - I think I actually take exception to your LAST line! At that point in your workflow, I bet the "run" command works great and launches with the correct python, BUT your TERMINAL is actually still out of whack. Namely, if you were to try to "pip install" something, it wouldn't install into the SELECTED environment, but rather, into the system environment. Even though other child processes (run, linting, etc) seem to behave correctly after you select the proper interpreter.
So we don't touch your
As for why
Maybe YOU don't, but SOMEONE sure is, and I'd love to know who.
I just spun up a totally new, fresh, virgin, not-even-updated High Sierra virtual machine. Haven't installed homebrew, aftermarket Python, nothing.
I downloaded and installed code, opened it once to run the "install code command" command, installed the Python extension, then closed it.
In a terminal window,
In fact, this environment is SO virgin the guest additions aren't installed yet and copy/paste doesn't work, so pardon any possible typos.
Back over in Code: Python: Create Terminal
That is, on an utterly virgin system, vscode is somehow taking the PATH and doing bad things to it.
Of course, maybe by "we" you mean "the vscode-python team." But it's certainly vscode, and not something external to it, unless its some kind of OS-interaction-level bug.
EDIT: for completeness, I uninstalled the Python extension and checked the integrated terminal - PATH is still getting mangled. So it seems like a Code or perhaps electron bug, but anyway, seems like kind of a problematic one?