Python unittest discovery error

[dantebarba]

Hi

I'm dealing with an issue that I'm currently unable to resolve. The vscode language server is unable to discover the python tests from my project. This issue was quite sudden I didn't update vs code nor the python extension, it was in fact working. I've also configured the same environment in multiple platforms (debian, different versions of macos) and this never happened. I even have the same project running with pyenv in another platform and it works ok.

Current environment:

OS: 22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Nov 2 18:01:13 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

VSCode version: 1.82.3 was rolled back from latest but the issue was present on the latest versions

Python extension: 2023.10.1 also downgraded from latest to test if this was an update issue.

Python: 3.7.13 with pyenv

Troubleshooting

• Tried deleting the .vscode and .config/Code directories and downgrading VSCode
• Tried downgrading the extension alongside the VSCode version with all the configuration reset to defaults. The only difference here is that the error was directly displayed on test discovery instead of debug run.
• With the latest VSCode version and extension version the tests work ok if ran without debug. The Test Results tab populates correctly with the log
• Tried playing around with PYTHONPATH. This eventually solved the issue (adding the python path at the top of the imports in my test case): https://stackoverflow.com/questions/68584302/modulenotfounderror-no-module-named-when-trying-to-make-unit-tests-in-pyt. None of the other PYTHONPATH modification solutions worked.

import sys
sys.path.insert(0, './')

I'm also getting these first two lines now every time I debug a test that did not show up in the log before, when everything worked ok:

CLIENT: Server listening on port 40565...
Received JSON data: ['analytics_endpoints_test.AnalyticsEndpointTest.test_get_gross_revenue']

The error

2023-12-07 11:52:39.910 [info] Discover tests for workspace name: myproject - uri: /home/myuser/myproject
2023-12-07 11:52:39.910 [info] Running discovery for unittest using the new test adapter.
2023-12-07 11:52:39.910 [info] Sending discover unittest script to server.
2023-12-07 11:52:39.927 [info] Running unittests with arguments: /home/myuser/.vscode/extensions/ms-python.python-2023.10.1/pythonFiles/unittestadapter/discovery.py,--port,41847,--uuid,82406961-fb55-4b19-be45-f0b8b3f6e115,--udiscovery,-v,-s,./tests,-p,*_test.py

2023-12-07 11:52:39.927 [info] > ./venv/bin/python ~/.vscode/extensions/ms-python.python-2023.10.1/pythonFiles/unittestadapter/discovery.py --port 41847 --uuid 82406961-fb55-4b19-be45-f0b8b3f6e115 --udiscovery -v -s ./tests -p *_test.py
2023-12-07 11:52:39.927 [info] cwd: .
2023-12-07 11:52:40.051 [info] Test server connected to a client.
2023-12-07 11:52:40.052 [error] Unittest test discovery error 
 Failed to import test module: admin_endpoints_test
Traceback (most recent call last):
  File "/home/myuser/.pyenv/versions/3.7.13/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/myuser/.pyenv/versions/3.7.13/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/myuser/myproject/tests/admin_endpoints_test.py", line 8, in <module>
    from create_app import create_app
ModuleNotFoundError: No module named 'create_app'

Thanks

[dantebarba]

Solved by adding this line on my global settings.json.

"python.experiments.optOutFrom": ["pythonTestAdapter"]

I'll keep this opened since it would be great to know why this happened and why this "adapter" was activated without my consent.

[brettcannon]

We have rewritten our testing code; that's what the adapter represents and it has now been activated for all users. We now use unittest to discover its own tests, and it imports code for discovery. I'm not sure how you were able to run tests previously with an ImportError.

And FYI you appear to be using a version of Python that we don't officially support.

[dantebarba]

This is the current log with the working configuration.

Idk the difference between unittest_discovery.py and discovery.py but when using the new method the PYTHONPATH variable is not set with the project root hence causing the import issues. The legacy method also activates the venv which the new adapter doesn't. If I run the discovery directly from my venv it works correctly and python tests are executed successfully.

2023-12-07 16:58:02.572 [info] > . ./venv/bin/activate && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python ~/.vscode/extensions/ms-python.python-2023.10.1/pythonFiles/printEnvVariables.py
2023-12-07 16:58:02.572 [info] shell: bash
2023-12-07 16:58:02.610 [info] > ./venv/bin/python ~/.vscode/extensions/ms-python.python-2023.10.1/pythonFiles/testing_tools/unittest_discovery.py ./tests *_test.py
2023-12-07 16:58:02.610 [info] cwd: .
2023-12-07 16:58:05.974 [info] DAP Server launched with command: /home/myuser/myproject/venv/bin/python /home/myuser/.vscode/extensions/ms-python.python-2023.10.1/pythonFiles/lib/python/debugpy/adapter

About the python version: bumping to 3.8.13 did not make any difference.

[brettcannon]

We don't set PYTHONPATH; we simply execute Python with the cwd set to the workspace or whatever you have set in your settings. If you still feel there's a bug, please open an issue using the "Report Issue" command.

[gxpd-jjh]

FYI for anyone else that stumbles on this - #21962 (comment)

[dantebarba]

I just wanted to bump this because recently I was working on an opensource project based on python 3.10 and flask 2.2. Specifically I was using github codespaces and the same issue came up.

This is the project, you can clone it and run the tests: https://github.com/dantebarba/autoscan-adapter.

The problem was solved the same way as I did the first time but maybe I'm missing something or I'm not configuring the environment correctly.

[brettcannon]

As I said in #22604 (reply in thread) , if you think there's a bug or a feature request you would like to make, please open an issue.

[gxpd-jjh]

Can you at least please ask the Doc team to remove that v3.7 is still supported from the intro paragraph? #21962 (comment)

[brettcannon]

Can you at least please ask the Doc team to remove that v3.7 is still supported from the intro paragraph? #21962 (comment)

That comes from https://github.com/microsoft/vscode-python/blob/main/README.md , so no docs team; just us 😅. Did you want to submit a PR? Otherwise we will fix it as part of #22946 .