Python Tests fails to load PYTHONPATH

[yygrechka]

Type: Bug

Behaviour

Expected vs. Actual

I have this in my settings file: "terminal.integrated.inheritEnv": true
This seems to have correctly imported my PYTHONPATH that I set in my .bashrc for the purposes of development. (i.e. code completion, jump to definition etc...) all work as expected. However when I write a test file, the test functionality in VSCode fails. I'm able workaround the failure by manually including the path of my custom module via sys.path.append, however, this should not be necessary.

My workspace:
bug_test
-- classA.py
-- tests
---- test_A.py

classA:

class A:
    pass

test_A:

import sys
import unittest

from bug_test import classA
class Test1(unittest.TestCase):
    def test1(self):
        self.assertEqual(4,4)

The expected behavior is for the test to pass.
The actual behavior is:

ModuleNotFoundError: No module named: bug_test

Workaround:
Adding this line in test_A.py : sys.path.append('/home/grechkay/yyg_projects')

fixes the issue and the test passes.

However, this has already been added to the PYTHONPATH in the bashrc file, which is recognized by all of the other VSCode functionality except for the tests.

Steps to reproduce:

1. Add: export PYTHONPATH='{path_to_module}':$PYTHONPATH in .bashrc
2. structure your workspace as above
3. run the tests

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.8.10
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Global
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

XXX

User Settings

languageServer: "Pylance"

testing
• unittestArgs: "<placeholder>"
• unittestEnabled: true

Extension version: 2023.12.0
VS Code version: Code 1.80.1 (74f6148eb9ea00507ec113ec51c489d6ffb4b771, 2023-07-12T17:22:25.257Z)
OS version: Linux x64 5.15.0-76-generic
Modes:

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz (16 x 3800)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	0, 1, 1
Memory (System)	31.01GB (11.41GB free)
Process Argv	. --crash-reporter-id cbbfbe56-bd2c-4171-b7c8-8bf66b35fe73
Screen Reader	no
VM	0%
DESKTOP_SESSION	ubuntu
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	ubuntu
XDG_SESSION_TYPE	x11

A/B Experiments

vsliv368cf:30146710
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
pythonvs932:30410667
py29gd2263cf:30789497
vsclangdc:30486549
c4g48928:30535728
dsvsc012cf:30540253
pynewext54:30695312
azure-dev_surveyone:30548225
282f8724:30602487
pyind779:30671433
f6dab269:30613381
pythonsymbol12:30671437
2i9eh265:30646982
showlangstatbar:30737416
vsctsb:30748421
pythonms35:30701012
03d35959:30757346
pythonfmttext:30731395
pythoncmvfstrcf:30756944
fixshowwlkth:30771522
hideindicator:30785051
pythongtdpath:30769146
i26e3531:30780429
pythonnosm12tcf:30779713
pythonidxpt:30784022
pythonnoceb:30776495
h7j2d465:30786200
dsvsc013:30789517

[eleanorjboyd]

Thank you for your issue report. We are looking into this now! In the meantime, you are likely on the new testing rewrite and this is why you saw a change in behavior since we just moved to this experiment to a larger group of people receiving it. You can opt out of the rewrite as I get this fix in by setting this in your user settings: "python.experiments.optOutFrom": ["pythonTestAdapter"],. If this doesn't work let me know as this will mean it is a different issue. Thank you and I will send updates in this thread as I get a fix in.

[eleanorjboyd]

Hello quick question, is this behavior a regression, ie have you had this work in the past and it no longer does? Secondly, can you check to see if you can see this in your logs Experiment 'pythonTestAdapter' is active? Finally, can you provide all the settings related to testing that you have set? It should be either pytest or unittest args in your workspace settings.json. Thanks

[gyandeeps]

Had the same issue and adding "python.experiments.optOutFrom": ["pythonTestAdapter"] fixed the issue. It did not log Experiment 'pythonTestAdapter' is active in logs.

[yygrechka]

As far as I know this is not a regression; I have not had this work before. I'm not sure how to check my logs for Experiment 'pythongTestAdapter'

The only settings I have related to testing is: "python.testing.pytestEnabled": true,

However I just checked today and reran the test and everything somehow works now. I can no longer reproduce the error. I'm pretty confused as to what happened, everything even works when I remove the "terminal.integrated.inheritEnv": true," setting. I'll attempt to reproduce a similar error tonight.

[yygrechka]

Yea... now VSCode seems to inherit the PYTHONPATH variable that I exported in .bashrc even with an empty settings file. So, I guess I don't have any issues relating to this anymore, although I'm not really sure what happened between when I opened the issue and now.

[joelhed]

I had the same problem originally reported here, except that it was solved by adding "python.experiments.optOutFrom": ["pythonTestAdapter"], to my user config.

[eleanorjboyd]

Thank you, for your responses. Can someone else check to see if they also only see this issue on the rewrite?

You can opt out of the rewrite as I get this fix in by setting this in your user settings: "python.experiments.optOutFrom": ["pythonTestAdapter"], to join the experiment add this setting to your users settings.json "python.experiments.optInto": ["pythonTestAdapter"]. (so depending on if it is working or not you can try the other boolean)

You can confirm you have the rewrite enabled by setting "python.analysis.logLevel": "Trace", in your settings and from the command palette "developer: set log level..." and select trace. Reload your vscode window then in your python output panel check for Experiment 'pythonTestAdapter' (should be at the very top of the python output)

Let me know if the rewrite fixes your issue. Thanks!

[andypexai]

I'm pretty sure this happens to me as well, but toggling the "python.experiments.optInto": ["pythonTestAdapter"] setting either way doesn't have an effect. I have a multi-root workspace where different roots are different python libraries, each with their own virtual env. So if I have library A and library B and library B depends on A, I get a ModuleNotFound error [trying to] test B (unittest discovery error). PYTHONPATH set with launch.json or .env doesn't help running tests but seems to help the debugger. My workaround has been to install library A's wheel file into library B's virtual environment. But that's not really ideal. I'd like to be making live changes in both places.

[eleanorjboyd]

@andypexai I will investigate further. When you say setting the .env file helps the debugger are you saying that this setup works when you debug but not any other way?

[woodyza]

Looks like I have this issue too, when using "python.envFile": "${workspaceFolder}/.vscode/.env" to prepend local paths to the PYTHONPATH. When pytest discovery is running the value of PYTHONPATH is /Users/me/.vscode/extensions/ms-python.python-2023.12.0/pythonFiles and discovery fails with module load errors. If I opt out of the rewrite it works as it used to.

Double checked 'pythonTestAdapter' is active is showing in the log too.

[FRidh]

Another case here. I described my situation extensively in #21872.

[github-actions]

Hey @eleanorjboyd, this issue might need further attention.

@yygrechka, you can help us out by closing this issue if the problem no longer exists, or adding more information.

[andypexai]

@eleanorjboyd sorry for not answering sooner. Not long after, I stopped having my specific problem.

[yygrechka]

@eleanorjboyd I'm fine to close out the issue if you are.

[eleanorjboyd]

Great thank you!

[woodyza]

@eleanorjboyd I still see this issue if I don't opt out of the new test adapter experiment, and we keep having to opt out team members if they've been included. Is there any more info I can provide?
#21642 (comment)

[eleanorjboyd]

Hi @woodyza, what version of vscode and the python extension are you on? Can you also describe your project setup more so I can look to replicating it?

[woodyza]

Hi @eleanorjboyd

I actually inadvertently found a solution to my issue while I was setting up a simple repo to reproduce. Basically it's a monorepo using a src layout, with multiple apps and some shared packages all in one repo. Something like:

src/
  app1/
    __init__.py
   # app code
  app2/
    __init__.py
   # app code
  library1/
    __init__.py
   # shared lib code
  library2/
    __init__.py
   # shared lib code
tests/
  test_stuff.py

Previously we were using "python.envFile" in settings.json to prepend the src folder to PYTHONPATH eg PYTHONPATH="./src:$PYTHONPATH" and that was enough for the test explorer (using pytest) to handle finding those imports. After enabling the new experimental adapter that trick doesn't work anymore.

What does work however is using pytests pythonpath argument to do the same thing, eg in pyptoject.toml:

[tool.pytest.ini_options]
pythonpath = [
  "src",
]

FWIW my vscode info is:

Version: 1.83.0 (Universal)
Commit: e7e037083ff4455cf320e344325dacb480062c3c
Date: 2023-10-03T16:13:15.449Z (1 day ago)
Electron: 25.8.4
ElectronBuildId: 24154031
Chromium: 114.0.5735.289
Node.js: 18.15.0
V8: 11.4.183.29-electron.0
OS: Darwin arm64 22.5.0

And the python extension is at v2023.16.0

[eleanorjboyd]

found the issue- seems we are not referencing the user-defined env file as part of the environment when configuring the sub-process for the test run. I will add it now but this will require a PR. In the meantime hopefully, your workaround can suffice and I will close this issue when the fix is merged. Thanks

[woodyza]

Thanks @eleanorjboyd!

[eleanorjboyd]

Is anyone able to verify if this is fixed? If not I can have a team member do so before our next stable release. Thanks!

[woodyza]

Using the v2023.19.12961014 pre-release of the extension it works for me, using that simple test case and without setting tool.pytest.ini_options. Thanks!