High CPU on starting pylance from get_pytest_options.py

[ethanh20009]

Environment data

• Language Server version: 2023.4.11
• OS and version: win32 x64
• Python version (and distribution if applicable, e.g. Anaconda): Python 3.10.9 Anaconda
• python.analysis.indexing: true
• python.analysis.typeCheckingMode: off

Code Snippet

Repro Steps

1. Install anaconda (and thus python)
2. install vscode, python extension pack.
3. create folder, open with vscode
4. create ipynb file
5. open window.
6. observe python thread in task manager using high cpu resources, won't stop until force kill or close vscode.

Expected behavior

Having vscode does not cause high cpu usage.

Actual behavior

Vscode open with python extension causes constant 25% CPU usage from ghost python threads.

Temporary found solution (not a proper fix)

If I comment out from C:\Users\ethan.vscode\extensions\ms-python.vscode-pylance-2023.4.11\dist\bundled\files get_pytest_options.py

#c = get_config()
#c.parse(sys.argv)
#print(f"Python classes: {c.getini('python_classes')}")
#print(f"Python files: {c.getini('python_files')}")
#print(f"Python functions: {c.getini('python_functions')}")

Pylance still works and the issue no longer remains for me.

I am very new to this sort of issue fixing and I know this is not a proper fix
Please advise on a proper fix for this.

Logs

[Info  - 8:41:39 PM] (3676) Pylance language server 2023.4.11 (pyright 64a33975) starting
[Info  - 8:41:39 PM] (3676) Server root directory: c:\Users\ethan\.vscode\extensions\ms-python.vscode-pylance-2023.4.11\dist
[Info  - 8:41:39 PM] (3676) Starting service instance "LemonadeStand"
[Info  - 8:41:39 PM] (3676) Auto-indent enabled
[Info  - 8:41:39 PM] (3676) Setting pythonPath for service "LemonadeStand": "C:\ProgramData\Anaconda3\python.exe"
[Info  - 8:41:40 PM] (3676) Assuming Python version 3.10
[Info  - 8:41:40 PM] (3676) Found 3 source files
[Info  - 8:41:40 PM] (3676) Background analysis(1) root directory: c:\Users\ethan\.vscode\extensions\ms-python.vscode-pylance-2023.4.11\dist
[Info  - 8:41:40 PM] (3676) Background analysis(1) started
[Info  - 8:42:03 PM] (3676) Indexer background runner(2) root directory: c:\Users\ethan\.vscode\extensions\ms-python.vscode-pylance-2023.4.11\dist (index)
[Info  - 8:42:03 PM] (3676) Indexing(2) started
[Info  - 8:42:04 PM] (3676) scanned(2) 466 files over 1 exec env
[Info  - 8:42:10 PM] (3676) [IDX(2)] Long operation: index execution environment c:\Users\ethan\Documents\Python\Jupyter\AI\LemonadeStand (5756ms)
[Info  - 8:42:10 PM] (3676) [IDX(2)] Long operation: index packages c:\Users\ethan\Documents\Python\Jupyter\AI\LemonadeStand (5771ms)
[Info  - 8:42:10 PM] (3676) indexed(2) 292 files over 1 exec env
[Info  - 8:42:10 PM] (3676) Indexing finished(2).

Full logs attached as .txt
pylanceLog.txt

[ankit-gautam23]

I can see it is assigned to someone. Is it open for contributions?

[ethanh20009]

I can see it is assigned to someone. Is it open for contributions?

I think it is via pyright

[ksergazy]

I was having the same issue with Pylance extension at version 2023.6.0, rolling back to 2023.4.1 fixed the issue. VSCode update auto updated it to new version, extension bisect didn't help because reload window wasnt killing spawned processes. Probably related: microsoft/vscode-python#20994.

[debonte]

So far I'm unable to repro this.

If I comment out from ...\ms-python.vscode-pylance-2023.4.11\dist\bundled\files\get_pytest_options.py...Pylance still works and the issue no longer remains for me.

@rchiodo, any ideas what might be going on here?

[rchiodo]

That code is attempting to import pytest and ask it for its configuration. If that hangs for some reason, it might cause the python process to use the CPU for a while.

It should be called just once though? Is it being called multiple times? What are the arguments to the python process?

[rchiodo]

Additionally, if you could try running this from the same environment:

python <pylance install folder>\ms-python.vscode-pylance-2023.4.11\dist\bundled\files\get_pytest_options.py

And report on the output (and how long it takes).

This would be what I get on my own machine:

C:\>python c:\users\aku91\.vscode\extensions\ms-python.vscode-pylance-9999.0.0-dev\dist\bundled\files\get_pytest_options.py
Python classes: ['Test']
Python files: ['test_*.py', '*_test.py']
Python functions: ['test']

It takes about 2 seconds to run for me in the root of my C drive, so I don't think the workspace has anything to do with it being slow. It might be the version of pytest though.

[rchiodo]

My guess is something is misbehaving in the environment and maybe we need to have a timeout for this script running.

[ethanh20009]

python <pylance install folder>\ms-python.vscode-pylance-2023.4.11\dist\bundled\files\get_pytest_options.py

And report on the output (and how long it takes).

So as I don't have python itself installed (my install is only through anaconda), I ran using the anaconda prompt to emulate what is by default happening in vscode.

The code did exactly the same as what I observed, thread hangs at constant cpu usage.
Escaping code yielded this log.

(base) C:\Users\ethan>python C:\Users\ethan\.vscode\extensions\ms-python.vscode-pylance-2023.4.11\dist\bundled\files\get_pytest_options.py
Traceback (most recent call last):
  File "C:\ProgramData\anaconda3\lib\tempfile.py", line 256, in _mkstemp_inner
    fd = _os.open(file, flags, 0o600)
PermissionError: [Errno 13] Permission denied: 'C:\\ProgramData\\anaconda3\\lib\\site-packages\\anyio\\__pycache__\\tmp5x9z9eb2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\ethan\.vscode\extensions\ms-python.vscode-pylance-2023.4.11\dist\bundled\files\get_pytest_options.py", line 5, in <module>
    c = get_config()
  File "C:\ProgramData\anaconda3\lib\site-packages\_pytest\config\__init__.py", line 1304, in parse
    self._preparse(args, addopts=addopts)
  File "C:\ProgramData\anaconda3\lib\site-packages\_pytest\config\__init__.py", line 1187, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "C:\ProgramData\anaconda3\lib\site-packages\pluggy\_manager.py", line 287, in load_setuptools_entrypoints
    plugin = ep.load()
  File "C:\ProgramData\anaconda3\lib\importlib\metadata\__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "C:\ProgramData\anaconda3\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "C:\ProgramData\anaconda3\lib\site-packages\_pytest\assertion\rewrite.py", line 163, in exec_module
    _write_pyc(state, co, source_stat, pyc)
  File "C:\ProgramData\anaconda3\lib\site-packages\_pytest\assertion\rewrite.py", line 315, in _write_pyc
    with atomic_write(os.fspath(pyc), mode="wb", overwrite=True) as fp:
  File "C:\ProgramData\anaconda3\lib\contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "C:\ProgramData\anaconda3\lib\site-packages\atomicwrites\__init__.py", line 166, in _open
    with get_fileobject(**self._open_kwargs) as f:
  File "C:\ProgramData\anaconda3\lib\site-packages\atomicwrites\__init__.py", line 183, in get_fileobject
    descriptor, name = tempfile.mkstemp(suffix=suffix, prefix=prefix,
  File "C:\ProgramData\anaconda3\lib\tempfile.py", line 341, in mkstemp
    return _mkstemp_inner(dir, prefix, suffix, flags, output_type)
  File "C:\ProgramData\anaconda3\lib\tempfile.py", line 256, in _mkstemp_inner
    fd = _os.open(file, flags, 0o600)
KeyboardInterrupt
^C

Debugged, located issue area

Adding debug logs:

print("1")
c = get_config()
print("2")
c.parse(sys.argv)
print("3")
print(f"Python classes: {c.getini('python_classes')}")
print(f"Python files: {c.getini('python_files')}")
print(f"Python functions: {c.getini('python_functions')}")

Yielded

(base) C:\Users\ethan>python C:\Users\ethan\.vscode\extensions\ms-python.vscode-pylance-2023.4.11\dist\bundled\files\get_pytest_options.py
1
2

Meaning there is an issue with c.parse(sys.argv) somehow.

Hope this helps.

[rchiodo]

Thanks. It sounds like a bug in pytest that we'll just have to work around.

[rchiodo]

@ethanh20009 do you know what version of pytest you have in that environment?

[ethanh20009]

Pytest 7.1.2

even launching pytest through anaconda cli is causing same issue. I believe root cause is in pytest init.

[rchiodo]

Hmm, that version works for me too. I wonder if anaconda is adding extra args to sys.argv somehow.

[ethanh20009]

I've just resolved the error on my end, seems anaconda messed up pytest install.
After calling 'conda update pytest', many packages where downloaded and installed, few where updated, many removed. However version of pytest did not change.
Running previous mentioned get pytest options now works the same for me as it did for you @rchiodo.

After fixing this though, it seemed to have uninstalled ipykernel which I was using for jupyter notebooks for pytorch. Maybe there's an incompatibility?

I'm going to stick with commenting out the code for now because that had no affect on the ability for my code to run.

[ethanh20009]

after more debugging, I messed up my conda environment - I could no longer get my code to run at all. creating and running in a new fresh environment seemed to have solved it for me, and solved the hanging thread issue as well. Still seems odd, now running pytest --version will say I do not have the package installed. All works normally now. Sorry for the confusion.

[rchiodo]

Thanks that's good to know. It's potentially only broken for people with environments that are broken.

.

[gramster]

Looks like it could be this: pytest-dev/pytest#5844

[bschnurr]

This issue has been fixed in prerelease version 2023.4.21, which we've just released. You can find the changelog here: CHANGELOG.md