Extension causes high cpu load due to interpreter discovery #11924

connerf · 2020-05-20T20:51:12Z

Issue Type: Performance
Extension Name: python
Extension Version: 2020.5.80290
OS Version: Windows_NT x64 10.0.18362
VSCode version: 1.45.1

⚠️ Make sure to attach this file from your home-directory:
⚠️C:\Users\cferguson\ms-python.python-unresponsive.cpuprofile.txt

Find more details here: https://github.com/microsoft/vscode/wiki/Explain-extension-causes-high-cpu-load

The text was updated successfully, but these errors were encountered:

shyamyadati · 2020-05-22T11:14:30Z

I can also confirm the same issue on OS X:

microsoft/vscode#98286 (comment)

karthiknadig · 2020-05-26T17:24:36Z

@connerf Can you please attach the cpu-profile.txt from here C:\Users\cferguson\ms-python.python-unresponsive.cpuprofile.txt

connerf · 2020-05-26T17:49:28Z

@karthiknadig is this what you are looking for? Or would you prefer the text in this box?

ms-python.python-unresponsive.cpuprofile.txt

ParthiBa · 2020-07-12T17:46:09Z

ms-python.python-unresponsive.cpuprofile.txt
it very slow while changing python interpreter . if i am using non -venv or non-pyenv then is not an issue

coreysharris · 2020-07-15T22:09:07Z

ms-python.python-unresponsive.cpuprofile.txt

karthiknadig · 2020-07-15T22:43:24Z

All these cases are due to how the extension tries to find and figure out the details about the interpreter. We are in the process of refactoring the code that does interpreter discovery. One of the thing we are making sure is to reduce the number of python processes we create while figuring out interpreter info.

Jigg97 · 2020-08-10T08:56:01Z

Same issue over here - here is mine

ms-python.python-unresponsive.cpuprofile.txt

byehack · 2020-09-11T10:11:54Z

same here on version 2020.8.106424

samoclay · 2020-10-14T11:59:18Z

Experiencing the same
ms-python.python-unresponsive.cpuprofile.txt

jmrodriguezg · 2020-10-30T04:08:34Z

Experiencing same issue...
ms-python.python-unresponsive.cpuprofile.txt

R4356th · 2020-11-01T20:33:53Z

Same on Windows 10. My version (result of running code --version):

1.50.1
d2e414d9e4239a252d1ab117bd7067f125afd80a
x64

Node.JS: 12.14.1, Electron: 9.2.1, Chrome: 83.0.4103.122

And the cpuprofile.txt file was not created, too.

zhyiyu · 2020-12-10T02:48:42Z

I think I should be experiencing the same issue, but I did not have the chance the produce the text file you wanted, I have some screenshots instead.

This is the high CPU demanding activity named "Python"

and this is where it comes from

eksortso · 2021-01-22T19:50:37Z

Told to attach this CPU profile file to this issue.
ms-python.python-unresponsive.cpuprofile.txt

LucianoBastian · 2021-02-12T22:29:17Z

I am experiencing the same issue since version 2020.2.62710.
I've installed a previous version 2020.1.58038 and it is working fine.

OneRain233 · 2021-02-18T02:51:29Z

I am experiencing the same issue.
ms-python.python-unresponsive.cpuprofile.txt

LukeHackett12 · 2021-02-18T13:32:11Z

I also have the same issue:
ms-python.python-unresponsive.cpuprofile.txt

Thomasalex2 · 2021-02-18T16:07:26Z

My issue is when I open Live Share. This extension just starts loading and everything goes unresponsive. It wasn't there two days back and I don't think anything changed between then and now
ms-python.python-unresponsive.cpuprofile.txt

mauza · 2021-02-18T17:35:36Z

I had to disable the python extension so I could connect to a liveshare. It hung on loading the extension. Looking at the htop there was a vscode process that looked like it was related to finding the terminal that was taking 100% of a cpu.

mblakew · 2021-02-19T01:41:29Z

I can confirm, just had this issue as well. Disabling the Python extension reduced my cpu load from 100% and unfroze live share.

avm5998 · 2021-02-19T11:31:12Z

Having the same issue with the Python extension on Macbook Pro.
ms-python.python-unresponsive.cpuprofile.txt

mpratscher · 2021-10-15T04:21:49Z

@karthiknadig Hope I got this correct. Please let me know if I missed something.

Like sn0wfa11 above, this is happening when using the root drive path (E: in this case) of a device (Adafruit CircuitPlayground Express) mounted as an external drive.

Python output using "python.logging.level": "debug":

Info 2021-10-14 23:05:58: Cached data exists KeyPrefix=Cache_Method_Output_h.addKeyToStorage-Args="workspace"-Arg-Separator-"autoSelectedWorkspacePythonInterpreter-e:\\"-Arg-Separator-undefined
Info 2021-10-14 23:05:58: Cached data exists KeyPrefix=Cache_Method_Output_h.addKeyToStorage-Args="workspace"-Arg-Separator-"autoSelectedWorkspacePythonInterpreter-e:\\"-Arg-Separator-undefined
Info 2021-10-14 23:05:58: Testing: Setting up watcher for e:\
Info 2021-10-14 23:05:58: Cached data exists KeyPrefix=Cache_Method_Output_h.addKeyToStorage-Args="workspace"-Arg-Separator-"autoSelectedWorkspacePythonInterpreter-e:\\"-Arg-Separator-undefined
Info 2021-10-14 23:05:58: Cached data exists KeyPrefix=Cache_Method_Output_h.addKeyToStorage-Args="workspace"-Arg-Separator-"autoSelectedWorkspacePythonInterpreter-e:\\"-Arg-Separator-undefined
Info 2021-10-14 23:05:58: Cached data exists KeyPrefix=Cache_Method_Output_h.addKeyToStorage-Args="workspace"-Arg-Separator-"autoSelectedWorkspacePythonInterpreter-e:\\"-Arg-Separator-undefined
Info 2021-10-14 23:05:58: Refresh Keys {} 1
Info 2021-10-14 23:05:58: Deferred promise created for progress with title: Discovering Python Interpreters
Info 2021-10-14 23:05:58: Create locator refreshing progress, Class name = g, completed in 0ms, has a falsy return value, , Return Value: undefined
Info 2021-10-14 23:05:58: Display locator refreshing progress, Class name = g, completed in 0ms, has a falsy return value, , Return Value: undefined
Info 2021-10-14 23:05:58: Refresh promise exists, wait on it
Info 2021-10-14 23:05:58: Searching for conda.
Info 2021-10-14 23:05:58: Probing conda binary: conda
Info 2021-10-14 23:05:58: Cached data exists KeyPrefix=Cache_Method_Output_h.addKeyToStorage-Args="workspace"-Arg-Separator-"autoSelectedWorkspacePythonInterpreter-e:\\"-Arg-Separator-undefined
Info 2021-10-14 23:05:58: > conda info --json
> conda info --json
Info 2021-10-14 23:05:58: [Error: spawn conda ENOENT
	at Process.ChildProcess._handle.onexit (internal/child_process.js:269:19)
	at onErrorNT (internal/child_process.js:465:16)
	at processTicksAndRejections (internal/process/task_queues.js:80:21)] {
  errno: -4058,
  code: 'ENOENT',
  syscall: 'spawn conda',
  path: 'conda',
  spawnargs: [ 'info', '--json' ]
}
Info 2021-10-14 23:05:58: Watcher disabled
Info 2021-10-14 23:05:58: Watcher disabled
Info 2021-10-14 23:05:58: Searching for workspace virtual envs in: e:\
Info 2021-10-14 23:05:58: Getting roots
Info 2021-10-14 23:05:58: Found roots
Info 2021-10-14 23:05:58: Watcher disabled
Info 2021-10-14 23:05:58: Getting roots
Info 2021-10-14 23:05:58: Found roots
Info 2021-10-14 23:05:58: Start watching root e:\ for globs ["python.exe","*/python.exe","*/Scripts/python.exe"]
Info 2021-10-14 23:05:58: Start watching: e:\ with pattern python.exe using VSCode API
Info 2021-10-14 23:05:58: Start watching: e:\ with pattern */python.exe using VSCode API
Info 2021-10-14 23:05:58: Start watching: e:\ with pattern */Scripts/python.exe using VSCode API
Info 2021-10-14 23:05:58: Cached data exists getEnvironmentVariables, <No Resource>
Info 2021-10-14 23:05:58: > "C:\Program Files\Python37\python.exe" "c:\Program Files\Visual Studio Code 1.61.0\data\extensions\ms-python.python-2021.11.1338943404-dev\pythonFiles\interpreterInfo.py"
> "C:\Program Files\Python37\python.exe" "c:\Program Files\Visual Studio Code 1.61.0\data\extensions\ms-python.python-2021.11.1338943404-dev\pythonFiles\interpreterInfo.py"
Info 2021-10-14 23:05:58: Cached data exists getEnvironmentVariables, <No Resource>
Info 2021-10-14 23:05:58: > "C:\Program Files\Python38\python.exe" "c:\Program Files\Visual Studio Code 1.61.0\data\extensions\ms-python.python-2021.11.1338943404-dev\pythonFiles\interpreterInfo.py"
> "C:\Program Files\Python38\python.exe" "c:\Program Files\Visual Studio Code 1.61.0\data\extensions\ms-python.python-2021.11.1338943404-dev\pythonFiles\interpreterInfo.py"
Info 2021-10-14 23:05:58: Couldn't locate the conda binary.
Info 2021-10-14 23:05:58: Found interpreter for C:\Program Files\Python37\python.exe,c:\Program Files\Visual Studio Code 1.61.0\data\extensions\ms-python.python-2021.11.1338943404-dev\pythonFiles\interpreterInfo.py
Info 2021-10-14 23:05:58: Found interpreter for C:\Program Files\Python38\python.exe,c:\Program Files\Visual Studio Code 1.61.0\data\extensions\ms-python.python-2021.11.1338943404-dev\pythonFiles\interpreterInfo.py
Info 2021-10-14 23:05:58: Environments added to cache [{"name":"venv","location":"d:\\Mike\\dev\\poster_pic_resize\\venv","kind":"virt-virtualenv","executable":{"filename":"d:\\Mike\\dev\\poster_pic_resize\\venv\\Scripts\\python.exe","sysPrefix":"d:\\Mike\\dev\\poster_pic_resize\\venv","ctime":-1,"mtime":-1},"searchLocation":{"$mid":1,"path":"/d:/Mike/dev/poster_pic_resize","scheme":"file"},"display":"Python 3.7.0 64-bit ('venv': virtualenv)","version":{"major":3,"minor":7,"micro":0,"release":{"level":"final","serial":0},"sysVersion":"3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)]"},"arch":3,"distro":{"org":""},"source":[],"hasCompleteInfo":true},{"name":"","location":"","kind":"global-other","executable":{"filename":"C:\\Program Files\\Python37\\python.exe","sysPrefix":"C:\\Program Files\\Python37","ctime":-1,"mtime":-1},"display":"Python 3.7.6 64-bit","version":{"major":3,"minor":7,"micro":6,"release":{"level":"final","serial":0},"sysVersion":"3.7.6 (tags/v3.7.6:43364a7ae0, Dec 19 2019, 00:42:30) [MSC v.1916 64 bit (AMD64)]"},"arch":3,"distro":{"org":"PythonCore","defaultDisplayName":"Python 3.7 (64-bit)"},"source":["windows registry"]},{"name":"","location":"","kind":"global-other","executable":{"filename":"C:\\Program Files\\Python38\\python.exe","sysPrefix":"C:\\Program Files\\Python38","ctime":-1,"mtime":-1},"display":"Python 3.8.2 64-bit","version":{"major":3,"minor":8,"micro":2,"release":{"level":"final","serial":0},"sysVersion":"3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AMD64)]"},"arch":3,"distro":{"org":"PythonCore","defaultDisplayName":"Python 3.8 (64-bit)"},"source":["windows registry"]}]
Info 2021-10-14 23:05:58: Will resolve deferred env collection refresh promise and delete locator query
Info 2021-10-14 23:05:58: Refresh Keys {} 1
Info 2021-10-14 23:05:58: Deferred env collection refresh promise resolved and locator query deleted, number of envs in cache: 3
Info 2021-10-14 23:05:58: Refresh Keys {} 0

ms-python.python-unresponsive.cpuprofile.txt

karrtikr · 2021-10-15T19:16:45Z

Can you try https://github.com/microsoft/vscode-python/suites/4067965844/artifacts/103392925 using the instructions here #17722 (comment)? Thank you both for testing this.

Also, try disabling VSCode filewatchers just in case, add the following in User settings:

"files.watcherExclude": {
  "**": true
}

Please also send User settings if possible.

mpratscher · 2021-10-15T19:48:11Z

@karrtikr Here you go. Happy to help, so please let me know if there's anything else you need or want me to try.

Edit: also still getting stuck at "Discovering Python Interpreters".

ms-python.python-unresponsive.cpuprofile.txt
python_output.txt
settings_json.txt

sn0wfa11 · 2021-10-15T19:56:10Z

@karrtikr See attached. Still getting stuck at "Discovering Python Interpreters". Let me know if you need anything else.
Debug Output.txt
ms-python.python-unresponsive.cpuprofile.txt
setting_json.txt

karrtikr · 2021-10-15T23:39:52Z

Thanks for the prompt response.

Also, as an experiment, I placed a python code file on a blank USB drive. When I opened the root directory in VS Code and then opened the python file, it locks up the same way on "Discovering Python Interpreters".

I tried opening the root directory (D: in my USB drive) as a folder in VSCode, and then tried opening a python file. I indeed get stuck at Discovering Python Interpreters! However if I open a folder inside D: say D:/hello, extension seems to work fine.

Bad news is that it seems extension never used to work when a drive (say D:/) was directly opened, atleast for me. Can you provide me an extension version which did work?

karrtikr · 2021-10-15T23:42:57Z

You can follow the prompts from "Extensions: Install Specific Version of Extension..." in the command palette:

Or you can open the extensions side bar, click on the settings for the Python extension and pick "Install Another Version...":

sn0wfa11 · 2021-10-15T23:44:24Z

@karrtikr, I am currently using v2021.9.1246542782 in my work environment and it is working correctly. I have the following opt out settings:

"python.experiments.optOutFrom": [
    
    "pythonDeprecatePythonPath",
        "pythonDiscoveryModuleWithoutWatcher",
        "pythonDiscoveryModule"
    ],

karrtikr · 2021-10-15T23:53:15Z

Very interesting, thank you. Just having

"python.experiments.optOutFrom": [
    "pythonDiscoveryModuleWithoutWatcher",
],

works for me.

Extension removed support for opting out of pythonDiscoveryModuleWithoutWatcher with 2021.10.1317843341 (see #17563), which is why it started happening now for you guys. But it seems the new discovery code (meaning experiment pythonDiscoveryModuleWithoutWatcher) was likely buggy going way back.

I'll let you know if I need anything else.

mpratscher · 2021-10-16T00:44:12Z

@karrtikr Version v2021.9.1246542782 also works for me. I tried it based on sn0wfa11's earlier post: #11924 (comment)
I do not have any opt outs in my settings (see my settings_json.txt file above). Also, note that python.logging.level and files.watcherExclude were only added for this testing; those are not typically there for my settings.

karrtikr · 2021-10-16T01:00:44Z

@mpratscher Noted, thanks.

I do not have any opt outs in my settings

You actually have telemetry disabled, which effectively means you have opted out of all experiments.

Touyu-Pan · 2021-10-16T12:54:55Z

I'm getting stuck at "Discovering Python Interpreters".
I really could use some helps
ms-python.python-unresponsive.cpuprofile.txt
Python output.txt
settings_json.txt
.

iScottMark · 2021-10-18T02:03:45Z

Just updated to 1.61.0 and this crap started again... The opt-outs are already in place. It gets stuck on "Discovering Python Interpreters". Trying to use the circuitpython extension. ms-python.python-unresponsive.cpuprofile.txt

It looks like the extension cannot find an interpreter when selecting "open folder" and opening the microcontroller drive. If I copy the files off to a non-external drive it works fine...

Create a new folder in D:\ drive not in D:\ and it worked well !

karrtikr · 2021-10-18T21:35:33Z

Should be fixed with #17781. Use Extension: Install from VSIX command to install the following VSIX: https://github.com/microsoft/vscode-python/suites/4087722570/artifacts/104116878 which has the fix!

Touyu-Pan · 2021-10-19T05:15:33Z

Should be fixed with #17781. Use Extension: Install from VSIX command to install the following VSIX: https://github.com/microsoft/vscode-python/suites/4087722570/artifacts/104116878 which has the fix!

Lots of thanks!!!

jing-bi · 2021-11-05T16:45:00Z

Seems the VSIX is expired, can you please repost it again?

karthiknadig · 2021-11-05T17:32:46Z

@jing-bi This is now released, you can find the VSIX here https://github.com/microsoft/vscode-python/releases/download/2021.11.1422169775/ms-python-release.vsix

ghost added the triage-needed Needs assignment to the proper sub-team label May 20, 2020

karthiknadig self-assigned this May 26, 2020

karthiknadig added triage bug Issue identified by VS Code Team member as probable bug labels May 26, 2020

ghost removed the triage-needed Needs assignment to the proper sub-team label May 26, 2020

karthiknadig added area-environments Features relating to handling interpreter environments needs PR and removed triage labels Jul 15, 2020

karthiknadig removed their assignment Jul 15, 2020

brettcannon added the reason-preexisting label Sep 9, 2020

karrtikr self-assigned this Oct 15, 2021

karrtikr added the investigating We are looking into the cause of the issue label Oct 15, 2021

karrtikr mentioned this issue Oct 17, 2021

Extension is unresponsive #17480

Closed

This was referenced Oct 18, 2021

Extension issue #17779

Closed

Application unresponsive after opening a file system provider based python file #17048

Closed

Fix unresponsive extension issues caused by discovery component #17781

Merged

karrtikr added this to the September 2021 Recovery 2 milestone Oct 18, 2021

karrtikr changed the title ~~Extension causes high cpu load~~ Extension causes high cpu load due to interpreter discovery Oct 18, 2021

karrtikr mentioned this issue Oct 18, 2021

Extension issue #17722

Closed

karrtikr added P0 and removed investigating We are looking into the cause of the issue labels Oct 18, 2021

karrtikr mentioned this issue Oct 18, 2021

Extension issue #17682

Closed

karrtikr closed this as completed in #17781 Oct 18, 2021

github-actions bot locked as resolved and limited conversation to collaborators Dec 6, 2021

Extension causes high cpu load due to interpreter discovery #11924

Extension causes high cpu load due to interpreter discovery #11924

Comments

connerf commented May 20, 2020

shyamyadati commented May 22, 2020

karthiknadig commented May 26, 2020

connerf commented May 26, 2020 • edited

ParthiBa commented Jul 12, 2020

coreysharris commented Jul 15, 2020

karthiknadig commented Jul 15, 2020

Jigg97 commented Aug 10, 2020

byehack commented Sep 11, 2020

samoclay commented Oct 14, 2020

jmrodriguezg commented Oct 30, 2020

R4356th commented Nov 1, 2020 • edited

zhyiyu commented Dec 10, 2020 • edited

eksortso commented Jan 22, 2021

LucianoBastian commented Feb 12, 2021

OneRain233 commented Feb 18, 2021

LukeHackett12 commented Feb 18, 2021

Thomasalex2 commented Feb 18, 2021

mauza commented Feb 18, 2021

mblakew commented Feb 19, 2021

avm5998 commented Feb 19, 2021

mpratscher commented Oct 15, 2021

karrtikr commented Oct 15, 2021 • edited

mpratscher commented Oct 15, 2021 • edited

sn0wfa11 commented Oct 15, 2021

karrtikr commented Oct 15, 2021 • edited

karrtikr commented Oct 15, 2021

sn0wfa11 commented Oct 15, 2021

karrtikr commented Oct 15, 2021 • edited

mpratscher commented Oct 16, 2021

karrtikr commented Oct 16, 2021

Touyu-Pan commented Oct 16, 2021 • edited

iScottMark commented Oct 18, 2021

karrtikr commented Oct 18, 2021

Touyu-Pan commented Oct 19, 2021

jing-bi commented Nov 5, 2021

karthiknadig commented Nov 5, 2021

connerf commented May 26, 2020 •

edited

R4356th commented Nov 1, 2020 •

edited

zhyiyu commented Dec 10, 2020 •

edited

karrtikr commented Oct 15, 2021 •

edited

mpratscher commented Oct 15, 2021 •

edited

karrtikr commented Oct 15, 2021 •

edited

karrtikr commented Oct 15, 2021 •

edited

Touyu-Pan commented Oct 16, 2021 •

edited