Select a Python Interpreter without modifying the workspace's settings.json

[Tiedye]

Be default, a local pipenv environemnt is searched for and if found, is marked as the python interpreter. But it also then adds a line to settings.json indicating the path for the virtual environment which can cause issues for configs shared among a team. The python.pythonPath variable will not be constant for each user working on the repo (eg "python.pythonPath": "/home/daniel/.local/share/virtualenvs/ontariofresh_backend-NQNAEc6t/bin/python", note my home directory in the path) and thus prevents the settings.json file from being committed to the remote repository.

Ideally, it would behave just as it does now, but just implicitly sets the python interpreter as the one found without explicitly writing it out to settings.json.

Environment data

• VS Code version: 1.25.0
• Extension version (available under the Extensions sidebar): 2018.6.0
• OS and version: ubuntu 18.04
• Python version (& distribution if applicable, e.g. Anaconda): 2.7.15
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): pipenv

Actual behavior

The installed pipenv virtual environment is found and selected as the python interpreter and the path to that environment is added to settings.json.

Expected behavior

The installed pipenv virtual environment is found and selected as the python interpreter and the path to that environment is not added to settings.json.

[brettcannon]

[I edited your ✉️ ⬆️ to come off as more 😃 and less 😠 ]

Yes, VS Code's lack of "workspace-personal" on top of a "workspace-global" config can be a problem in this situation.

[Tiedye]

Ha sorry, didn't realize I was coming off as 😠

[brettcannon]

@Tiedye No problem! It was basically phrasing as "this is bad!" can be off-putting because it comes off as a demand instead of a request.

[brettcannon]

@Tiedye a work-around for this is to set PIPENV_VENV_IN_PROJECT and the extension will automatically pick up the .venv directory that gets created. And you can set python.pythonPath to ${workspaceFolder}/.venv to be consistent within your project without any hard-coded, absolute paths.

[Tiedye]

Awesome! That works. Should the issue be left open? Or is this more of a vs-code issue at this point.

[brettcannon]

We can leave this issue open to consider coming up with a solution that negates the need to force you to set an environment variable (at worst we could document this for users).

As for getting personal workspace settings, I believe there is a VS Code issue on this topic so if you can find it and upvote it then that might help get it implemented.

[Tiedye]

microsoft/vscode#15909 for reference

[mungojam]

set PIPENV_VENV_IN_PROJECT

I'm thinking of using this workaround too as I can easily roll out environment variables to all our devs. The unfortunate thing about it is that the .venv isn't in the default .gitignore in our version of GitHub Enterprise so we'll need to remember to add it to every repo. It is in the public github.com version

[thernstig]

@brettcannon Is that the only workaround or could you opt not to update .vscode/settings.json in some way and still use the Pipenv virtualenv's pythonPath?

It's great that VS code is working on a general fix in microsoft/vscode#15909 though. I am wondering however if it is customary for extensions to automatically update a workspace's settings? As settings.json is aimed to be shared amongst team members of a project it becomes problematic if it is updated with individual settings, such as in this issue.

[brettcannon]

@thernstig I don't know if there's any customary practice that has been established by the VS Code extension community. For us we do it because there's a visible startup cost in searching for a person's environment, so writing it out short circuits that search.

[thernstig]

@brettcannon Part of the reason I am making that comment is because we had this same discussion for issue #265. There I referred to the issue microsoft/vscode#851 on VS Code, where bpasero (his team started VS Code when it was still called Monaco) made this comment about .vscode/settings.json

These things are workspace related settings so they are stored in the workspace folder with the intent to share them to other people (e.g. a launch configuration is typically OS and path independent).

In relation to that, I was thinking that polluting .vscode/settings.json with user-specific settings must be some kind of antipattern?

Adding specific settings to .vscode/settings.json breaks the convention of sharing workspace specific settings for projects. I think this is a huge issue for teams utilizing this feature.

Every user in our project now has to do a git checkout .vscode/settings.json everytime before commit in order to not commit their personal venv path.

[DonJayamanne]

Here's @egamma response to this issue:
microsoft/vscode#1435 (comment)

We have gaps when it comes to managing many projects and sharing among them and the answer to your question is currently not supported, sorry, but we have it on the backlog to improve this.

Similar issues:
I am certain there are others
microsoft/vscode#15909

Its not an antipattern, there's just no alternative.

Until VSC come up with a solution, we have no choice but use settings.json
I'm reluctant to architect a completely new configuration settings model that would be not be supported by VSC.

@brettcannon
Not adding the python path into settings.json will be very very challenging.
E.g. When debugging, the launch.json inherits the python Path from settings.json. Trying to change this would be a unnecessarily complicated.
Storing in a separate file is equally messy.
I'd suggest we just wait for VSC to come up with a solution. We're not the only ones with this problem. E.g. other extension require similar configuration settings (go, java, etc).

@brettcannon If you disagree, please feel free to change the label

Temporary solutions:

• Solution 1 Easiest: Ensure everyone in the organization has an environment variable named XYZ created that points to the python interpreter.
  Then modify settings.json as follows:
  "python.pythonPath": "${env.XYZ}"

That's it. Everyone now shares the same settings.json.

• Solution 2: Select a Python Interpreter without modifying the workspace's settings.json #2125 (comment)

• Solution 3: Open the folder as a multi-root workspace.
  This way you can have multi-root settings and the standard workspace settings.
  The multi-root settings can be user specific, with the python path in that file.

More info on multi-root workspaces can be found here: https://code.visualstudio.com/docs/editor/multi-root-workspaces
See here for sample:

[brettcannon]

@DonJayamanne I think we should come up with a general solution and document the suggestion as this keeps cropping up.

[thernstig]

@DonJayamanne thank you for the detailed answer, much appreciated. Especially that you supplied multiple workarounds we can use.

[DonJayamanne]

Solution:

• Use the persistent store (API provided by VSC to store information against the workspace)
• I.e. instead of storing the information in settings.json we store it in here.
• This feature will be enabled via some flag such as enableXYZ

Work flow (if enableXYZ = true):

• When VS Code is opened, priority order for determining the python path for a given workspace:

1. we check if a python path has been specified in local workspace folder settings,
2. Check the persistent store for current workspace folder
3. we check if a python path has been specified in local workspace (multi-root scenarios) settings,
4. we check if a python path has been specified in global settings
5. Finally use default value from settings.json (that is always python)

• If use wishes to change the python path, they use the UI to change the path
• Selected path is written into the persistent store (any value in settings.json)

This could be extended for all other settings in python extension, however we'd need a UI for this. Today the UI for editing settings.json is either provided by VS Code or you edit the raw settings.json file.

Which is why I'm proposing this solution be scoped ONLY for the pythonPath setting and nothing else. All other values will have to be managed as done today in one of the 3 possible setting.json files (global|user, workspace and workspace folder).

@brettcannon @qubitron /cc

Summary:

• pythonPath setting is stored in some sandboxed location, scoped to a specific workspace folder.
• These values are not editable manually, has to be done via the UI
• This will be an opt-in feature
• This applies only to the pythonPath setting.

[brettcannon]

How will this affect e.g. tasks.json that refer to {env:python.pythonPath}?

[brettcannon]

@zor-el unfortunately not as they occurred in team meetings so they were never written down.

Regardless, we have opened #12313 on your behalf for you fill us in on how this change is specifically affecting you.

[kynan]

With deprecation of python.pythonPath, how will other extensions be able to get the currently selected interpreter path?

[brettcannon]

@kynan we are exposing an API which will give extensions actually more details than they currently get from the path setting along with the ability to be notified when the environment is changed by the user.

[zor-el]

@kynan we are exposing an API which will give extensions actually more details than they currently get from the path setting along with the ability to be notified when the environment is changed by the user.

@brettcannon, unfortunately this path is a path to closed ecosystem, as it doesn't do much for external tools, but only extensions will benefit.

[qt1]

IMHO The correct solution is as follows:

1. Add search option e.g. { autosearch. absolute, relative etc}
2. Allow the path to be
   2.1 automatically searched (I would not want this!, too many ways to go wrong) or
   2.2 relative to some anchor - the .vs folder, user's home,
   2.3 absolute path

This will give better control and will not break current projets

[tercerapersona]

I might have multiple projects (multi-root folders) in the same workspace.

I would expect the extension to read the first parent's .vscode folder of any file I select and use that python.pythonPath, but doesn't work. Am I missing something?

[drozzy]

This is very annoying and seems like a very easy fix. Python path changes from machine to machine.
Why not have settings.local.json that we can add to .gitignore and write python path there?

[brettcannon]

@drozzy if VS Code were to ever support such a config file we would, but trying to support a configuration outside of VS Code's mechanisms is an extra support burden and potentially confusing to users.

[drozzy]

But aren't you guys part of microsoft?

[SteelPhase]

Doesn't this have a workaround using the following setting, or am I mistaken?

"python.experiments.optInto": [
    "DeprecatePythonPath - experiment"
]

[drozzy]

@SteelPhase I didn't know about that, I guess it's still experimental. Seems like it's meant exactly for this purpose. Thanks. I'll wait until it's officially released.

[brettcannon]

@drozzy being part of Microsoft does not mean we get to tell the VS Code team what to do 😉 . You can go to github.com/microsoft/vscode and open a feature request if you want (although I suspect one already exists so I would check first).

@SteelPhase yep, that will get you what you want for now.

[kariharju]

Hi,
A long thread so sorry if I missed an answer to this.
We have a system that is building isolated python environments (trying our hardest to specifically avoid polluting target machines with multiple python installations 😉)
Our extension, related to this just stopped working on some people as a part of the A/B testing and we are looking for a way forward.

We would just need an API (or any way) to set the interpreter path of this extension from another extension.
As of now, we have not found a way.. are we missing something?

[brettcannon]

@kariharju you have a couple of options:

• Upvote https://github.com/microsoft/vscode-python/discussions/16370
• Universally opt out of the experiment and stick with python.pythonPath

We are actually hoping to have this whole situation resolved in a release or two in a way that should make everyone happy (famous last words, I know 😉).

[kariharju]

Hi, yeah I know these things are not easy to get done.

I forgot to upvote #16370 (I came from the "same source" 😉).

Basically, we don't care about the internals as much as the fact that we could just somehow get isolated python environments up-n-running with this extension (as pretty much every Python developer using VsCode has this one installed as it is kind of a default).

In enterprise-scale automation scenarios having the end-user or even the developer's machine "polluted" by multiple Python installations is a no-go. So not all Python environments are installed on the OS level and having a way to point a Python environment location to this extension would be a crucial thing to have.

[linette-zyy]

This issue is also reproduced on this build.

After change Python Interpreter, the file settings.json is not generated in .vscode in current folder/workspace.

[brettcannon]

Please see #12313 (comment) for an explanation of the change we are slowly rolling out to users to address this.

[brettcannon]

https://devblogs.microsoft.com/python/python-in-visual-studio-code-july-2021-release/#selecting-a-python-interpreter-no-longer-modifies-workspace-settings announced this feature.