One CPU at 100% because of pytest -p vscode_pytest --collect-only

[pawamoy]

VSCode keeps spawning a pytest process to collect tests (apparently).

    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--config-file=config/pytest.ini"
    ],

It runs it in my current .venv: /absolute/path/to/.venv/bin/python -m pytest -p vscode_pytest --collect-only --config-file=config/pytest.ini.

The process never stops and consumes one CPU entirely.

My pytest config:

[pytest]
python_files =
  test_*.py
  *_test.py
  tests.py
addopts =
  --cov
  --cov-config config/coverage.ini
testpaths =
  tests

# action:message_regex:warning_class:module_regex:line
filterwarnings =
  error
  # TODO: remove once pytest-xdist 4 is released
  ignore:.*rsyncdir:DeprecationWarning:xdist
  ignore:.*slated for removal in Python:DeprecationWarning:.*

If I run the command myself:

% .venv/bin/python -m pytest -p vscode_pytest --collect-only --config-file=config/pytest.ini
Traceback (most recent call last):
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 865, in import_plugin
    __import__(importspec)
ModuleNotFoundError: No module named 'vscode_pytest'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pytest/__main__.py", line 7, in <module>
    raise SystemExit(pytest.console_main())
                     ^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 206, in console_main
    code = main()
           ^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 159, in main
    config = _prepareconfig(args, plugins)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 346, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/helpconfig.py", line 106, in pytest_cmdline_parse
    config = yield
             ^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1152, in pytest_cmdline_parse
    self.parse(args)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1501, in parse
    self._preparse(args, addopts=addopts)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1384, in _preparse
    self.pluginmanager.consider_preparse(args, exclude_only=False)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 794, in consider_preparse
    self.consider_pluginarg(parg)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 817, in consider_pluginarg
    self.import_plugin(arg, consider_entry_points=True)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 867, in import_plugin
    raise ImportError(
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 865, in import_plugin
    __import__(importspec)
ImportError: Error importing plugin "vscode_pytest": No module named 'vscode_pytest'

Without the vscode_pytest plugin:

% .venv/bin/python -m pytest --collect-only --config-file=config/pytest.ini 
========================================================================================= test session starts =========================================================================================
platform linux -- Python 3.12.1, pytest-8.2.0, pluggy-1.5.0
Using --randomly-seed=153736517
rootdir: /home/pawamoy/data/dev/griffe/config
configfile: pytest.ini
plugins: cov-5.0.0, randomly-3.15.0, xdist-3.6.1
collected 777 items
... (truncated)
===== 777 tests collected in 2.97s =====

Any idea why the process hangs and consumes an entire CPU?

Could be related to microsoft/pylance-release#4218 🤷

[pawamoy]

Thanks for transferring the discussion to the right repo 🙇

Here are the last lines in the output tab for "Python":

2024-05-08 23:36:31.809 [info] Discover tests for workspace name: griffe - uri: /home/pawamoy/data/dev/griffe/src/griffe/loader.py
2024-05-08 23:36:31.809 [info] Running discovery for pytest using the new test adapter.
2024-05-08 23:36:31.809 [error] Test discovery already in progress, not starting a new one.

[pawamoy]

The output for "Python Test Log" ends with:

      <Function test_annotations_from_classes>
        Assert custom classes are correctly transformed to annotations.

No ======== like when pytest finishes reporting collected tests: is it stuck?

[pawamoy]

OK it seems to have trouble with my config file. If I remove it from python.testing.pytestArgs, the tests are quickly discovered.

[pawamoy]

And it's really just the config file. If I pass the options from the config file directly into the VSCode settings, it works fine:

    "python.testing.pytestArgs": [
        "--cov", "--cov-config=config/coverage.ini", "tests"
    ]

[pawamoy]

I isolated the issue: it only happens when my pytest config file is in a subfolder. If I put it in the root, no issue. Will create a new issue unless a maintainer wants to convert this discussion.

[eleanorjboyd]

resolved here: #23411. Thanks!

[chrisrodrigue]

I noticed an issue as well. It seems that vscode automatically executes pytest if you use the option

  "python.testing.pytestArgs": [
    "--config-file=${workspaceFolder}/pyproject.toml"
  ],

This is evidenced by the fact that code coverage and pytest caches are being generated in my project directory despite never explicitly running python tests from either the vscode GUI or the command line.

[eleanorjboyd]

Do you have "python.testing.pytestEnabled": True? It should only run if you set one of the testing frameworks to be enabled.

[chrisrodrigue]

Yes, as shown

    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--config-file=${workspaceFolder}/pyproject.toml"
    ],

It seems that the tests automatically run despite not executing them from the Testing tab or the command line.

After some digging I suspect the implicitly enabled setting python.testing.autoTestDiscoverOnSaveEnabled could be the culprit. The description for it says Enable auto run test discovery when saving a test file.

I don't think that automatically running test code should be enabled by default. This could be a security vulnerability if any of the tests happen to contain malicious code.

[brettcannon]

I don't think that automatically running test code should be enabled by default. This could be a security vulnerability if any of the tests happen to contain malicious code.

Because you must say you trust the code in the repository in order to activate the extension, this concern is outside our security boundary.