Adding markers (@mark.asyncio) make discovery fails

[titouanfreville]

Type: Bug

Hello,

I have a strange issue and test discovery. While making new test asynchronus, the test discovery began to broke.
With the exact same test case, adding only async markers and async test methods, the discovery fails with asyncs on, and pass correctly wihout them.

I looked at the output of the discovery command, both are exactly the same expect for the line number due to markers being added.

Test run correctly directly from command line.

Behaviour

Expected vs. Actual

Expected

Discovery works

Actual

Discovery fails without any error message

Discovery outputs

Ok: https://drive.google.com/file/d/1Zqsxk8pW_hcttCnAFydbOqyNX_KU1-ul/view?usp=share_link

Ko: https://drive.google.com/file/d/18aocozebcQ1Rfwsz_g9_CVtiTs40Zxzf/view?usp=share_link

Steps to reproduce:

I do not know how to reproduce on a simpler code base. Already added some async markers without issue so it feel strange.

I will try to reproduce on simpler project

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.10.7
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Venv
• Value of the python.languageServer setting: Pylance

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

 ~/venv/bin/python ~/.vscode-server/extensions/ms-python.python-2022.18.2/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear
cwd: .
[ERROR 2022-10-29 19:43:10.76]: Error discovering pytest tests:
 [n [Error]: Waiting up to 5 seconds.
Sent all pending logs.

	at ChildProcess.<anonymous> (/home/vscode/.vscode-server/extensions/ms-python.python-2022.18.2/out/client/extension.js:2:242843)
	at Object.onceWrapper (node:events:646:26)
	at ChildProcess.emit (node:events:526:28)
	at maybeClose (node:internal/child_process:1092:16)
	at Process.ChildProcess._handle.onexit (node:internal/child_process:302:5)]

User Settings

languageServer: "Pylance"

linting
• flake8Enabled: true
• banditArgs: "<placeholder>"
• banditEnabled: true
• mypyEnabled: true

formatting
• provider: "black"
• blackArgs: "<placeholder>"

testing
• cwd: "<placeholder>"
• pytestEnabled: true

Extension version: 2022.18.2
VS Code version: Code 1.73.1 (6261075646f055b99068d3688932416f2346dd3b, 2022-11-09T03:54:53.913Z)
OS version: Linux x64 6.0.6-76060006-generic
Modes:
Sandboxed: No
Remote OS version: Linux x64 6.0.6-76060006-generic

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz (12 x 4299)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: disabled_software video_encode: disabled_software vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: disabled_off
Load (avg)	1, 1, 1
Memory (System)	31.27GB (23.03GB free)
Process Argv	--unity-launch --crash-reporter-id 9193f995-a3c7-4d94-9dd7-1fdc933506b2
Screen Reader	no
VM	0%
DESKTOP_SESSION	pop
XDG_CURRENT_DESKTOP	Unity
XDG_SESSION_DESKTOP	pop
XDG_SESSION_TYPE	x11

Item	Value
Remote	Dev Container: Bet On You API
OS	Linux x64 6.0.6-76060006-generic
CPUs	Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz (12 x 4299)
Memory (System)	31.27GB (23.03GB free)
VM	0%

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscorecescf:30445987
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30581797
vsaa593:30376534
pythonvs932:30410667
cppdebug:30492333
vscaac:30438847
vsclangdf:30486550
c4g48928:30535728
dsvsc012:30540252
azure-dev_surveyonecf:30548226
pyindex848:30577860
nodejswelcome1:30587005
3biah626:30602489
gswce1:30612156
iaj6b796:30613358
dbltrim-noruby:30604474
f6dab269:30613381

[titouanfreville]

🌵 Found issue. (Sorry, already looked for hours and did not found previously). An import in test code missed env configuration at discovery time and made it failed without any indication.
Still strange cause pytest --collect-only worked fined (usually failed when discovery failed), manual command run was ok, and discovery only failed from plugin directly without logs to help :/

[eleanorjboyd]

Hello @titouanfreville! Thank you for all your careful investigation, this is very useful to us! We are sorry that this issue occurred. To clarify, does this issue still exist for you or were you able to reconfigure to have the import get the env configuration at discovery time? Could you send over your env configuration that caused the error? I will take a closer look at this and appreciate you submitted this issue to our repo!

[titouanfreville]

Hello, @eleanorjboyd.
The issue still exists but is not linked to what I though (and it makes more sense now). It is direcly linked to the already called issue: #18431
While testing various think, I added a direct import of the fastAPI server in my tests witch requires some envs to be sat to run. Then it probably raises indirectly an error on discovery. Removing the import solve the issue. I already had this issue when creating the tests cases initially and used a local import in method to solve it.

The strange thing is that there are no line related to those issue when running the command manually and it provides a correct result 💫

An easy reproduction would be to have a file that depends on some env being loaded and import it without setuping env in the test files.

[eleanorjboyd]

@titouanfreville, thank you for the extra info! I will look into this.

[eleanorjboyd]

Hello @titouanfreville! Sorry for the delay in circling back. Yes you are correct about it relating to #18431 and therefore the fix for it is the feature request, #17171. Given we already have an issue tracking this I am going to close this issue in favor of the feature request. Thank you again for all your work debugging and I am sorry you encountered this lack of functionality with environment variables.