New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to logic for selection of best (default) version of Python interpreter #3369

Closed
DonJayamanne opened this Issue Nov 15, 2018 · 5 comments

Comments

@DonJayamanne
Copy link

DonJayamanne commented Nov 15, 2018

Related to #3326

Today we use python as the default version of Python for the extension.

Logic for selection of best (default) version of Python interpreter:

  • Pick interpreter with highest version and bitness
@DonJayamanne

This comment has been minimized.

Copy link
Author

DonJayamanne commented Nov 28, 2018

Requirement

  • The extension needs to chose the best available Python Interpereter

Problems

  • How to quickly identify best available Python Interpereter
  • We store selected python in python.pythonPath in settings.json
  • Scenario 1:
    • User has Python 3.6 installed (in path)
    • Open workspace, and extension uses Python 3.6.
    • User installs pylint, flake8, etc in the above global environment.
    • Tomorrow user installs Python 3.7 and opens the above workspace
    • They'll need to install all the python dependencies again
    • Yes virtual envs are a solution, but not necessarily the right one.
  • Scenario 2:
    • User install VSC for first time on Windows
    • User opens a new workspace
    • Extension is running in the background attempting to determine the best available Python version
    • This can take > 30 seconds on Windows with Conda.
    • Extension is not usable a the best available version of Python hasn't been identified

Terminology

  • Store in workspace cache
    • Rather than storing information in settings.json file, we can store in a cache thats specific to a workspace.
    • This way, we do not write to settings.json, and the information stored is specific to a workspace.

Solutions:

  • How to quickly identify best available Python Interpereter

    • On Mac and Linux look for python3.7, python3, etc in current path.
    • On Windows look for py -3.7, etc in current path
    • If all of the above fail, then we use windows registry, conda, etc...
    • Which is available, pick that.
    • What ever algorithm we use, this MUST be simple and very quick
    • Once extension loads, we can iterate through all known interpreters an pick best. I.e. worst case scenario, first time we might not pick the best interpreter.
    • Again, if by the time the best available is determined, and we have a list of ALL interpreters, then we just go and pick best from ALL interpreters.
    • Lets discuss this item seprately
  • A. First time extension is installed and loaded:

    • A1: Look for local virtual envs (or pipenv), if found use that & store in workspace cache.
    • A2: If previous step was unsuccessful, then pick best available python interpreter (algorithm outline separately - see above) and store that in workspace cache as well as global cache.
    • A3: In the mean time, look for all interpreters (in the background), if we find a version of Python that's better than that was identified and stored earlier (A1/A2), then store this in global cache for next time..
  • B. Open new workspace (not first time loading extension)

    • B1: Look for local virtual envs (or pipenv), if found use that & store in workspace cache usual place as done today, i.e. settings.json.
    • B2: Else, get python interpreter stored in global cache (identified in previous A2/A3.
    • B3: In the mean time, look for all interpreters (in the background), if we find a version of Python that's better than that was identified and stored earlier, then store this in global cache for next time..
  • C. Open existing workspace (not first time loading extension)

    • C1: Check if we python defined in python.pythonPath in settings.json, if yes, then use that.
    • C2: Check if we have stored details of python interpreter in stored in workspace global cache. If yes, then use that.
    • C3: Look for local virtual envs (or pipenv), if found use that & store in workspace cache do what we do today, i.e. store in usual place - settings.json.
    • C4: In the mean time, look for all interpreters (in the background), if we find a version of Python that's better than that was identified and stored earlier, then store this in global cache for next time..
@brettcannon

This comment has been minimized.

Copy link
Member

brettcannon commented Dec 7, 2018

One thing we will need to help with debugging reported issues is a way for people to get to the path to the selected Python interpreter. Maybe a command to write it out to settings.json?

@DonJayamanne

This comment has been minimized.

Copy link
Author

DonJayamanne commented Dec 8, 2018

Sounds like we need to create a command that'll spit out debug logs

  • Python path
  • Versions
  • Settings
  • Launch config

Get all info and dump into clipboard or open an editor with all of that in there for user to copy and paste into the GitHub issue.

DonJayamanne added a commit that referenced this issue Dec 22, 2018

Add support for discovering python interpreters on windows using py.e…
…xe (#3783)

For #3369

* Depends on 3780
* Add support for discovering python interpreters on windows using py.exe

DonJayamanne added a commit that referenced this issue Dec 27, 2018

DonJayamanne added a commit that referenced this issue Dec 27, 2018

@DonJayamanne DonJayamanne self-assigned this Dec 29, 2018

@ericsnowcurrently ericsnowcurrently self-assigned this Jan 7, 2019

@ericsnowcurrently

This comment has been minimized.

Copy link
Member

ericsnowcurrently commented Jan 7, 2019

This mostly worked for me on master. Instead of picking "python" as it used to, it picked Python 3.7.0 64-bit ('.env': virtualenv). The following were available:

  • Python 2.7.15 64-bit
  • Python 3.5.2 64-bit
  • Python 3.6.4 64-bit ('.miniconda3': conda)
  • Python 3.6.7 64-bit
  • Python 3.7.0 64-bit ('.env': virtualenv)
  • Python 3.7.0 64-bit ('test env': conda)
  • Python 3.7.0 64-bit ('3.7.0': pyenv)
  • Python 3.7.1 64-bit ('base': conda)
  • Python 3.7.1 64-bit

So it's not clear why it did not pick 3.7.1, nor why it preferred the virtualenv over conda or pyenv (for 3.7.0). I would have expected it to pick 3.7.1. In the face of a tie (as with 3.7.0) I would have expected it to ask me.

All that said, picking 3.7.0 instead of 2.7 is an improvement. :)

@brettcannon

This comment has been minimized.

Copy link
Member

brettcannon commented Jan 7, 2019

@ericsnowcurrently the order preference is venv, virtualenv, pipenv, global, conda; IOW this worked the way it's designed. The thinking is the first three options were on purpose and are directly tied to the workspace folder, and conda is last because (a) it isn't directly tied to a directory and (b) it's slow to detect 😁 .

@lock lock bot locked as resolved and limited conversation to collaborators Feb 5, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.