Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python.testing.pytestArgs: passing config file path when it's in a subfolder consumes 100% of a CPU #23411

Closed
pawamoy opened this issue May 11, 2024 Discussed in #23386 · 8 comments · Fixed by #23420
Closed
Assignees
Labels
area-testing bug Issue identified by VS Code Team member as probable bug needs PR Ready to be worked on verified Verification succeeded
Milestone

Comments

@pawamoy
Copy link

pawamoy commented May 11, 2024

Discussed in #23386

Originally posted by pawamoy May 8, 2024
VSCode keeps spawning a pytest process to collect tests (apparently).

    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--config-file=config/pytest.ini"
    ],

It runs it in my current .venv: /absolute/path/to/.venv/bin/python -m pytest -p vscode_pytest --collect-only --config-file=config/pytest.ini.

The process never stops and consumes one CPU entirely.

My pytest config:

[pytest]
python_files =
  test_*.py
  *_test.py
  tests.py
addopts =
  --cov
  --cov-config config/coverage.ini
testpaths =
  tests

# action:message_regex:warning_class:module_regex:line
filterwarnings =
  error
  # TODO: remove once pytest-xdist 4 is released
  ignore:.*rsyncdir:DeprecationWarning:xdist
  ignore:.*slated for removal in Python:DeprecationWarning:.*

If I run the command myself:

% .venv/bin/python -m pytest -p vscode_pytest --collect-only --config-file=config/pytest.ini
Traceback (most recent call last):
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 865, in import_plugin
    __import__(importspec)
ModuleNotFoundError: No module named 'vscode_pytest'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pytest/__main__.py", line 7, in <module>
    raise SystemExit(pytest.console_main())
                     ^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 206, in console_main
    code = main()
           ^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 159, in main
    config = _prepareconfig(args, plugins)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 346, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 122, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/helpconfig.py", line 106, in pytest_cmdline_parse
    config = yield
             ^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1152, in pytest_cmdline_parse
    self.parse(args)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1501, in parse
    self._preparse(args, addopts=addopts)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 1384, in _preparse
    self.pluginmanager.consider_preparse(args, exclude_only=False)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 794, in consider_preparse
    self.consider_pluginarg(parg)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 817, in consider_pluginarg
    self.import_plugin(arg, consider_entry_points=True)
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 867, in import_plugin
    raise ImportError(
  File "/home/pawamoy/data/dev/griffe/.venv/lib/python3.12/site-packages/_pytest/config/__init__.py", line 865, in import_plugin
    __import__(importspec)
ImportError: Error importing plugin "vscode_pytest": No module named 'vscode_pytest'

Without the vscode_pytest plugin:

% .venv/bin/python -m pytest --collect-only --config-file=config/pytest.ini 
========================================================================================= test session starts =========================================================================================
platform linux -- Python 3.12.1, pytest-8.2.0, pluggy-1.5.0
Using --randomly-seed=153736517
rootdir: /home/pawamoy/data/dev/griffe/config
configfile: pytest.ini
plugins: cov-5.0.0, randomly-3.15.0, xdist-3.6.1
collected 777 items
... (truncated)
===== 777 tests collected in 2.97s =====

Any idea why the process hangs and consumes an entire CPU?

Could be related to microsoft/pylance-release#4218 🤷


As mentioned in the discussion, I isolated the issue. It only happens when my pytest config file is in a subdirectory.

This works fine:

    "python.testing.pytestArgs": [
        "--config-file=pytest.ini"
    ],

While this consumes a CPU and never finishes:

    "python.testing.pytestArgs": [
        "--config-file=config/pytest.ini"
    ],
@github-actions github-actions bot added the triage-needed Needs assignment to the proper sub-team label May 11, 2024
@eleanorjboyd
Copy link
Member

Hello! Thanks for this detailed bug report, I will investigate. Could you include your logs from the python output channel? For your logs, can you first set your log level to trace via theDeveloper: set log level command in the command palette then rerun the tests. I am curious what point in the process it gets stuck at and if the logs.

Also are you running the command yourself from the root of the repo right?

Finally are you able to see the same behavior for an arbitrary test- ie if you just write a pytest test which always passes and run with the config arg do you get the same behavior?

Sorry for all the questions! Thanks

@github-actions github-actions bot added the info-needed Issue requires more information from poster label May 13, 2024
@pawamoy
Copy link
Author

pawamoy commented May 13, 2024

Thanks a lot for your help!

Could you include your logs from the python output channel? For your logs, can you first set your log level to trace via theDeveloper: set log level command in the command palette then rerun the tests.

Sure. Note that I don't even run the tests, I only try to "collect" them (refreshing them in the Testing tab).

Trace logs:

2024-05-13 19:36:05.969 [debug] Testing: Manually triggered test refresh
2024-05-13 19:36:05.969 [debug] Testing: Clearing all discovered tests
2024-05-13 19:36:05.970 [debug] Testing: Forcing test data refresh
2024-05-13 19:36:05.970 [debug] Testing: Refreshing all test data
2024-05-13 19:36:05.971 [debug] Found cached env for /tmp/repro23411/.venv/bin/python
2024-05-13 19:36:05.975 [info] Discover tests for workspace name: repro23411 - uri: /tmp/repro23411
2024-05-13 19:36:05.975 [info] Running discovery for pytest using the new test adapter.
2024-05-13 19:36:05.976 [info] All environment variables set for pytest discovery for workspace /tmp/repro23411: REDACTED
2024-05-13 19:36:05.978 [debug] Found cached env for /tmp/repro23411/.venv/bin/python
2024-05-13 19:36:06.021 [debug] Running pytest discovery with command: -m pytest -p vscode_pytest --collect-only --config-file=config/pytest.ini for workspace /tmp/repro23411.
2024-05-13 19:36:06.025 [info] > ./.venv/bin/python -m pytest -p vscode_pytest --collect-only --config-file=config/pytest.ini
2024-05-13 19:36:06.025 [info] cwd: .
2024-05-13 19:36:06.343 [info] ============================= test session starts ==============================

2024-05-13 19:36:06.343 [info] platform linux -- Python 3.12.1, pytest-8.2.0, pluggy-1.5.0

2024-05-13 19:36:06.343 [info] rootdir: /tmp/repro23411/config
configfile: pytest.ini

2024-05-13 19:36:06.351 [info] collected 1 item

2024-05-13 19:36:06.352 [info] 
<Dir repro23411>
  <Dir tests>
    <Module test_hello.py>
      <Function test_hello>

2024-05-13 19:36:16.730 [debug] Found cached env for /tmp/repro23411/.venv/bin/python
2024-05-13 19:36:21.469 [error] Subprocess exited unsuccessfully with exit code null and signal SIGTERM on workspace /tmp/repro23411.
2024-05-13 19:36:21.470 [error] Subprocess exited unsuccessfully with exit code null and signal SIGTERM on workspace /tmp/repro23411. Creating and sending error discovery payload
2024-05-13 19:36:21.470 [error] pytest test discovery error for workspace:  /tmp/repro23411 
  
 The python test process was terminated before it could exit on its own, the process errored with: Code: null, Signal: SIGTERM for workspace /tmp/repro23411
2024-05-13 19:36:21.470 [info] ResultResolver EOT received for discovery.
2024-05-13 19:36:21.470 [debug] deferredTill EOT resolved for /tmp/repro23411
2024-05-13 19:36:21.470 [info] Disposing data receiver for /tmp/repro23411 and deleting UUID; pytest discovery.
2024-05-13 19:36:23.232 [debug] Found cached env for /tmp/repro23411/.venv/bin/python

The logs hang at <Function test_hello>, then I interrupt the pytest process and the logs continue with Subprocess exited unsuccessfully etc.

Also are you running the command yourself from the root of the repo right?

No, I'm not running anything myself: it's vscode-python which runs the pytest process to collect the tests. I can trigger it by clicking on the "Refresh tests" button in the "Testing" tab.

Finally are you able to see the same behavior for an arbitrary test- ie if you just write a pytest test which always passes and run with the config arg do you get the same behavior?

Yep, I reproduced this in a clean, minimal project:

% tree -a -L 2
.
├── config
│   ├── .pytest_cache
│   └── pytest.ini
├── tests
│   ├── __pycache__
│   └── test_hello.py
├── .venv
│   ├── bin
│   ├── CACHEDIR.TAG
│   ├── .gitignore
│   ├── lib
│   ├── .lock
│   └── pyvenv.cfg
└── .vscode
    └── settings.json

9 directories, 7 files
% cat .vscode/settings.json 
{
    "files.watcherExclude": {
        "**/.venv*/**": true,
        "**/venv*/**": true
    },
    "python.testing.unittestEnabled": false,
    "python.testing.pytestEnabled": true,
    "python.testing.pytestArgs": [
        "--config-file=config/pytest.ini"
    ]
}
% cat config/pytest.ini 
[pytest]
python_files =
  test_*.py
testpaths =
  tests
% cat tests/test_hello.py 
def test_hello():
    assert True

Running the pytest command myself works fine:

% ./.venv/bin/python -m pytest --collect-only --config-file=config/pytest.ini 
==== test session starts ====
platform linux -- Python 3.12.1, pytest-8.2.0, pluggy-1.5.0
rootdir: /tmp/repro23411/config
configfile: pytest.ini
collected 1 item                                                                                                                                                                                              

<Dir repro23411>
  <Dir tests>
    <Module test_hello.py>
      <Function test_hello>

==== 1 test collected in 0.01s ====

@github-actions github-actions bot removed the info-needed Issue requires more information from poster label May 13, 2024
@eleanorjboyd
Copy link
Member

Hello! I have found the issue here. One thing I do to build the test tree seen in the test explorer is create nodes between session node (root) and test nodes. In this instance you can see in your logs that the root is within the config folder rootdir: /tmp/repro23411/config while the test files are inside the tests folder. Therefore the root dir is not a parent of the tests which was an assumption the plugin made. I will update it to check if the rootdir is not a parent and if so find the closest common path and making that the root (therefore making the tree look like below). Im putting together a fix but need a bit longer to create a test for this scenario.

image

@eleanorjboyd eleanorjboyd added bug Issue identified by VS Code Team member as probable bug area-testing needs PR Ready to be worked on and removed triage-needed Needs assignment to the proper sub-team labels May 13, 2024
@eleanorjboyd eleanorjboyd added this to the May 2024 milestone May 13, 2024
@pawamoy
Copy link
Author

pawamoy commented May 13, 2024

That's awesome, thank you so much! Take all the time you need, no worries 😄 And thanks for the explanation, that makes total sense 🙂

@eleanorjboyd
Copy link
Member

Hi! This is merged and will be out on vscode insiders tomorrow so you can give it a try and let me know if it is fixed or if an issue persists.

@pawamoy
Copy link
Author

pawamoy commented May 14, 2024

Thank you very much! Will let you know 🙂

@eleanorjboyd
Copy link
Member

Hello! Were you able to test to see if this is working?

@pawamoy
Copy link
Author

pawamoy commented May 23, 2024

I just installed the insiders version, and the issue still occurs 🤔

Version: 1.90.0-insider
Commit: 81c89c4d00663e1718871bab2f9bf2064a060b63
Date: 2024-05-22T09:30:48.825Z
Electron: 29.3.1
ElectronBuildId: 9464424
Chromium: 122.0.6261.156
Node.js: 20.9.0
V8: 12.2.281.27-electron.0
OS: Linux x64 6.9.1-arch1-1

Ah, maybe I needed to switch the pre-release version of the Python extension?

UPDATE: yep, it works fine with the pre-release 😄

@roblourens roblourens added the verified Verification succeeded label May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-testing bug Issue identified by VS Code Team member as probable bug needs PR Ready to be worked on verified Verification succeeded
Projects
None yet
3 participants