python.testing.pytestArgs ignores multiple instances of the same arg

[DScott-2]

Type: Bug

Behaviour

Expected vs. Actual

This is a new issue that began appearing as of v2024.2

Expected:
pytestArgs which allow multiple of the same arg (e.g. --ignore=path/to/ignored/dir) pass all instances of the arg in the list to pytest
Actual:
Only the last instance of multiple args in the list are passed to pytest

Steps to reproduce:

1. Modify the python.testing.pytestArgs setting with multiple --ignore=path options
2. collect tests
3. Only the last ignored path in python.testing.pytestArgs will be ignored

Diagnostic data

• Python version (& distribution if applicable, e.g. Anaconda): 3.12.1
• Type of virtual environment used (e.g. conda, venv, virtualenv, etc.): Unknown
• Value of the python.languageServer setting: Default

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

Actual command: sent:

2024-02-29 12:03:43.223 [info] > ./shared/lib_ext/python/python.exe -m pytest -p vscode_pytest --collect-only . --ignore=./__temp__/ --basetemp=./__temp__
2024-02-29 12:03:43.224 [info] cwd: .
2024-02-29 12:03:43.868 [info] Discover tests for workspace name: pmu - uri: c:\Users\dscott2\source\pmu
2024-02-29 12:03:43.868 [info] Running discovery for pytest using the new test adapter.

Expected command sent:

2024-02-29 12:03:43.223 [info] > ./shared/lib_ext/python/python.exe -m pytest -p vscode_pytest --collect-only . --ignore=./web_ui/ --ignore=./__deploy__/ --ignore=./__temp__/ --basetemp=./__temp__

User Settings

languageServer: "Pylance"

testing
• cwd: "${workspaceFolder}"
• pytestArgs: [
".",
"--ignore=${workspaceFolder}/web_ui/",
"--ignore=${workspaceFolder}/__deploy__/",
"--ignore=${workspaceFolder}/__temp__/",
"--basetemp=${workspaceFolder}/__temp__/",
]
• pytestEnabled: true

Extension version: 2024.2.0
VS Code version: Code 1.86.1 (31c37ee8f63491495ac49e43b8544550fbae4533, 2024-02-07T09:08:20.941Z)
OS version: Windows_NT x64 10.0.19045
Modes:

[drothlis]

I think this was introduced by #22885 here:

75ed73e#diff-19ce457a2aaf03c9ce746ac804e645758ead9dccd86e7fdd4a3367116be9b769L69-R114

[drothlis]

It's worse than "ignored" — this causes an error for me. We use a custom pytest plugin which we enable with these VSCode settings:

    "python.testing.pytestArgs": [
        "-p",
        "stbt_rig",
        "-p",
        "no:python",
        ...
    ]

and since 2024.2.0 this causes my pytest collection to fail with "ERROR: file or directory not found: no:python".

This is the command line VS Code is using:

./.venv/bin/python -m pytest -p vscode_pytest --collect-only -p stbt_rig no:python ...

Note that it's missing the second "-p".

[pyropenguin]

I'm seeing this issue when ignoring multiple directories. Only the last ignore is used:

settings.json:

    "python.testing.pytestArgs": [
        "--ignore=src/application1/test",
        "--ignore=src/application2/test",
        "--ignore=src/application3/test",
        "./src"
    ],

Python Output

2024-02-29 09:37:39.408 [info] > /bin/python3 -m pytest -p vscode_pytest --collect-only --ignore=src/application3/test ./src

[eleanorjboyd]

thank you for the bug report- yes I think you are right with the commit id that introduces this. I will work on a fix- thanks

[PierreKiwi]

Hello,
Not sure if it is related, but I have the following setup:

./libs/libA
./libs/libB

When discovering the tests, I want to ignore libA tests so I added --ignore=libs/libA, but now nothing is discovered.

When running the command in the terminal, it does work just fine.

......run_adapter.py discover pytest -- -s --cache-clear --ignore=libs/libA --root-dir .

[eleanorjboyd]

Hello! The fix for this has been merged and is out on the most recent release of vscode insiders. Is someone able to verify that the bug is fixed for them? If so, we can then put a point release out for stable with the fix. Thanks

@anthonykim1 for visibility

[drothlis]

@eleanorjboyd does that include the code-insiders snap for linux?

❯ code-insiders --version
1.88.0-insider
d73fa8b14a6c873958d00a7d7ad13fcb540a052c
x64

I also installed the pre-release version of the Python extension: v2024.3.10651007 (pre-release)

With these versions I still get the same symptoms reported in my comment above.

[drothlis]

From code inspection it seems to me that #23024 fixes this case:

--key=value --key=value2

...but it doesn't handle these cases (bug introduced in #22885 / #22076):

--key value --key value2
-k value -k value2

[DScott-2]

@eleanorjboyd
I can confirm that it fixes the issue with --ignore=path/ that I was seeing previously on 2024.3.10651007
Can't speak to the other issues mentioned above