Debug output for test discovery has misleading argument

[JamesHutchisonCarta]

Type: Bug

Behaviour

Expected vs. Actual

Expected:
The output found in "Python" when refreshing tests produces a verbatum command that can be copy and pasted into a terminal to reproduce an issue.

> /usr/local/bin/python ~/.vscode-remote/extensions/ms-python.python-2022.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir /workspaces/my-project -s --cache-clear --reuse-db
cwd: .

Actual:
The output has . as the --rootdir argument, but the actual argument passed in is a directory (likely the workspace one) with an absolute path

Example:

> /usr/local/bin/python ~/.vscode-remote/extensions/ms-python.python-2022.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear --reuse-db
cwd: .

Steps to reproduce:

1. Update the run_adapter.py file to add print(sys.argv). You might have to add the sys import
2. Refresh tests. This print statement will bust the parser and it will display all output. You can see that sys.argv does not have . as the rootdir argument.

> /usr/local/bin/python ~/.vscode-server/extensions/ms-python.python-2022.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear tests
cwd: .
[ERROR 2022-8-13 1:55:32.617]: Error discovering pytest tests:
 [SyntaxError: ['/home/vscode/.vscode-server/extensions/ms-python.python-2022.14.0/pythonFiles/testing_tools/run_adapter.py', 'discover', 'pytest', '--', '--rootdir', '/workspaces/my-project', '-s', '--cache-clear', 'tests']

Significance

When writing custom logic, the cwd . and the actual directory argument provided may technically be equivalent, but the test explorer will insert the absolute path in front of the source file's location when it parses the results as if it was a relative path if the path returned does not start with .

Example:
/workspaces/my-project/src/my-file.py

becomes

/workspaces/my-project/workspaces/my-project/src/my-file.py

Which then fails to open.

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.8.13
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Global
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

> /usr/local/bin/python ~/.vscode-remote/extensions/ms-python.python-2022.14.0/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir . -s --cache-clear --reuse-db
cwd: .

User Settings

languageServer: "Pylance"

linting
• flake8Enabled: true
• flake8Path: "<placeholder>"
• banditPath: "<placeholder>"
• mypyPath: "<placeholder>"
• pycodestylePath: "<placeholder>"
• pydocstylePath: "<placeholder>"
• pylintPath: "<placeholder>"

formatting
• autopep8Path: "<placeholder>"
• provider: "black"
• blackPath: "<placeholder>"
• yapfPath: "<placeholder>"

testing
• cwd: "<placeholder>"
• debugPort: 5678
• pytestArgs: "<placeholder>"
• pytestEnabled: true
• autoTestDiscoverOnSaveEnabled: false

Extension version: 2022.14.0
VS Code version: Code 1.70.2 (Universal) (e4503b30fc78200f846c62cf8091b76ff5547662, 2022-08-16T05:36:37.829Z)
OS version: Darwin x64 21.6.0
Modes:
Remote OS version: Linux x64 5.10.76-linuxkit
Remote OS version: Linux x64 5.4.0-1089-azure
Remote OS version: Linux x64 5.10.76-linuxkit
Remote OS version: Linux x64 5.10.76-linuxkit

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vsdfh931cf:30280410
vshan820:30294714
vstes263cf:30335440
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
cmake_vspar411:30557514
vsaa593cf:30376535
pythonvs932:30410667
cppdebug:30492333
pylanb8912:30545647
vsclangdf:30486550
c4g48928:30535728
hb751961:30553087
dsvsc012:30540252
azure-dev_surveyone:30548225
i497e931:30553904

[eleanorjboyd]

Hello! Thank you for submitting this bug, we appreciate your help. Do you notice this bug when you run both pytests and unittests or is it present on only one?

[JamesHutchisonCarta]

I haven't tried unittests

[eleanorjboyd]

Hello and sorry for the delay, this looks to be a bug in the test adapter that is likely related to the ongoing buggy behavior on pytest discovery and running in vscode. We are currently work on a rewrite of how pytest works in vscode and hope this rewrite will fix this and similar bugs. The main issue which will be working on is here: #17242, and progress will be updated on this issue. Thank you!

[eleanorjboyd]

Hi! This issue should be fixed with the rewrite that is released via an experiment. I would like to expand this to make it more comprehensive. I would like the logs to include a specific command you can copy and paste to run in the terminal. This will need to be delayed for an other fix which needs to be done first but I made this issue to track it.