Skip to content

Tests are discovered correctly but then an incorrect argument is supplied to PyTest, leading test-result discovery to fail (Git Submodules?) #17903

@Mithrandir2k18

Description

@Mithrandir2k18

Environment data

  • VS Code version:
    Version: 1.61.2
    Commit: 6cba118ac49a1b88332f312a8f67186f7f3c1643
    Date: 2021-10-19T14:58:13.605Z
    Electron: 13.5.1
    Chrome: 91.0.4472.164
    Node.js: 14.16.0
    V8: 9.1.269.39-electron.0
    OS: Linux x64 5.4.0-90-generic snap
  • Extension version (available under the Extensions sidebar): v2021.10.1365161279
  • OS and version: Ubuntu 20.04
  • Python version (& distribution if applicable, e.g. Anaconda): 3.8.10
  • Type of virtual environment used (N/A | venv | virtualenv | conda | ...): None
  • Relevant/affected Python packages and their versions: PyTest 6.0.0
  • Relevant/affected Python-related VS Code extensions and their versions: VSCode Native Test Explorer
  • Value of the python.languageServer setting: Pylance

Expected behaviour

Given the following repo structure:

/work/repos/my_repo
|
-src/
    |
    - main.py
 - dependencies/
      |
     -my_git_submodule/
            - tests/
                   |
                   - test_other_main.py
           ...
-tests/
    |
    - conftest.py
    - test_sometests.py
    - integration_tests/
            |
            - test_integration_main.py

Discovered tests in workspace are run and their results displayed when doing CTRL+SHIFT+P: Test: Run All Tests.
Expect to run the command using the supplied path to the test folder.

Actual behaviour

Pytest fails to run, using the following command:

cwd: /work/repos/my_repo

Causing the error: ERROR - ModuleNotFoundError: No module named 'tests.integration_tests'
But it runs correctly when only running the tests in the tests folder and that can be achieved by clicking "Run All" on the "tests" subgroup:

> /usr/bin/python3 -m pytest --rootdir /work/repos/my_repo --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-14347RaAiZqepCFUO.xml ./tests
cwd: /work/repos/my_repo

This works without any issues, however this is not run when doing the VSCode Run All tests function and I have not been able to configure this working to command to be used.

Another way to get it working is ignoring the submodules in dependencies(they should be unit-tested using their own repos workflows) using --ignore-glob=dependencies/* in python.testing.pytestArgs. However I don't see why not manually ignoring submodules should break test(result) discovery.

> /usr/bin/python3 -m pytest --rootdir /work/repos/my_repo --override-ini junit_family=xunit1 --junit-xml=/tmp/tmp-14347jcCkQFFhe9c0.xml --ignore-glob=dependencies/* ./tests
cwd: /work/repos/my_repo

So the first command fails while the two other commands work. I am not sure if it is intended for the first command to fail.
When using pytest from the terminal, all that's required to get it to run is a simple pytest.ini like this:

[pytest]
testpaths = 
    tests

and the simply doing pytest with the repo as cwd, no ignoring the dependencies folder required.

Steps to reproduce:

Replicate above repo structure with Pytest files in /tests and in /tests/integration_tests and include another repo as a submodule that also contains its own tests.

Metadata

Metadata

Assignees

Labels

area-testingbugIssue identified by VS Code Team member as probable bug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions