Debugging pytest fails

[WindsOfHades]

Environment data

VS Code version: 1.20.1
Python Extension version: Latest (can't even find the version number from git hub or within vscode)
Python Version: Python 3.6.4 :: Anaconda, Inc.
OS and version: Windows 7

Actual behavior

Cannot configure debugging for pytest. Tried different launch configuration but no success.

1. I tried with the default launch configuration:
   {
   "name": "Python",
   "type": "python",
   "request": "launch",
   "stopOnEntry": false,
   "pythonPath": "${config:python.pythonPath}",
   "program": "${file}",
   "cwd": "${workspaceFolder}",
   "env": {},
   "envFile": "${workspaceFolder}/.env",
   "debugOptions": [
   "RedirectOutput"
   ]
   },

A. Pressing the "Debug test" on top of test case:
"Debug adapter process has terminated unexpectedly"
Then I will end up in a lingering debug mode that if I press the following I get the listed results

---

pause:
no effect

---

restart:
There was an error in starting the debug server. Error = {"code":"ECONNREFUSED","errno":"ECONNREFUSED","syscall":"connect","address":"127.0.0.1","port":3000}

---

disconnect:
No effect

---

selecting and starting debug configuration from the status bar (at the bottom):
Traceback (most recent call last):
File "c:\workspace\ver-virtual-vehicle\test\test_fdx_decoder.py", line 7, in
from vv.fdx import decoder
ModuleNotFoundError: No module named 'vv'

Module import errors all of a sudden!!!

my directory structure is:
(workspace repo)/
src/
test/

---

B. Pressing F5

Traceback (most recent call last):
File "c:\workspace\ver-virtual-vehicle\test\test_fdx_decoder.py", line 7, in
from vv.fdx import decoder
ModuleNotFoundError: No module named 'vv'

And no lingering debugger!

---

2. Using integrated terminal:

same as above!
{
"name": "Python: Terminal (integrated)",
"type": "python",
"request": "launch",
"stopOnEntry": false,
"pythonPath": "${config:python.pythonPath}",
"program": "${file}",
"cwd": "${workspaceFolder}",
"console": "integratedTerminal",
"env": {},
"envFile": "${workspaceFolder}/.env",
"debugOptions": []
},

First it fails:
"Debug adapter process has terminated unexpectedly"

Second time when I press in the bar and select the launcher again it outputs the following in the terminal:
$ cd "C:\workspace\ver-virtual-vehicle" ; env "PYTHONIOENCODING=UTF-8" "PYTHONUNBUFFERED=1" python "C:\Users\A281180.vscode\extensions\ms-python.python-2018.1.0\pythonFiles\PythonTools\visualstudio_
py_launcher.py" "C:\workspace\ver-virtual-vehicle" 50990 34806ad9-833a-4524-8cd6-18ca4aa74f14 DUMMYVALUE "c:\workspace\ver-virtual-vehicle\test\test_fdx_decoder.py"
Traceback (most recent call last):
File "c:\workspace\ver-virtual-vehicle\test\test_fdx_decoder.py", line 7, in
from vv.fdx import decoder
ModuleNotFoundError: No module named 'vv'

Can't import modules properly.

---

3. Using a custom launcher:
   {
   "name": "PyTest",
   "type": "python",
   "request": "launch",
   "stopOnEntry": false,
   "pythonPath": "${config:python.pythonPath}",
   "module": "pytest",
   "args": [
   "-sv"
   ],
   "program": "${file}",
   "cwd": "${workspaceFolder}",
   "env": {},
   "envFile": "${workspaceFolder}/.env",
   "debugOptions": [
   "RedirectOutput"
   ]
   },

---

A. Pressing the "Debug test" on top of test case:
Lingering issue for days but after selecting the debugger again it eventually run all the test cases (more explanation below)!

---

B. Pressing F5
This works but it will run all the tests until it reaches the test case that I have a breakpoint on!
And of course it will continue running the rest of test cases after I am finished debugging my selected test case!

---

4. Inconsistencies

I am pretty sure I tried one combination with the same launch config, that I managed to get what I want (just debug one test case!). After all, I tried so many things that I can't reproduce that behavior again.
It could be that the first time I select any launcher it works and stop of working after that!

---

P.s. The lack of proper documentation for debugging pytest is just wasting hours of developer times
Would you please write a step by step guide once and for all that will not lead to all these questions and github issues, this will save time for everyone (users and you guys)?

Expected behavior

I just want to add a break point on the first entry line of one of the my test cases and just press F5 and it should automatically stop on my breakpoint without running other test cases!

I do not want to add any files to my project just to satisfy my IDE needs! This means that the following solution is not ok IMHO:
https://donjayamanne.github.io/pythonVSCodeDocs/docs/debugging_debugging-pytest/

Steps to reproduce:

Described above!

Logs

Output from Python output panel

Output from Console window (Help->Developer Tools menu)

[brettcannon]

We do have docs on debugging tests, they are just not pytest-specific as there is nothing special from the perspective of the extension between e.g. pytest and unittest. (The docs you link to are from before Microsoft took over development of the extension.)

As for the import error, that's because your code is in a src directory which isn't at the root of your workspace, so Python doesn't know about your vv package when you directly run your tests. If you add a .env file that sets PYTHONPATH then you won't get that issue, e.g.:

PYTHONPATH=src:test
# If you're on Windows, replace the `:` with `;`

I have opened #924 to track the idea of making this more obvious.

Otherwise I can't reproduce since I can run an individual test both from the code lens and command palette and have it stop on a breakpoint as expected.

[WindsOfHades]

Hi Brett,

I have followed the new issue that you opened and tried to clarify there, that this is not about finding the test case. But I've been asked to not post comments there.
So I will post it again here:

There is no problem with detection of the test! I can run the test by simply clicking on "Run Test" or typing "pytest" from shell and there is no need of adding any .env file.

The problem arises when you want to debug your test:

1. If you just use the default "Python" lanucher and press F5, suddenly it cannot import the target module.
2. This can be solved by the following launcher, but the problem is that it will run all the test cases that you have until it reaches the one you are trying to debug.

{
            "name": "PyTest",
            "type": "python",
            "request": "launch",
            "stopOnEntry": false,
            "pythonPath": "${config:python.pythonPath}",
            "module": "pytest",
            "args": [
                "-sv"
            ],
            "program": "${file}",
            "cwd": "${workspaceFolder}",
            "env": {},
            "envFile": "${workspaceFolder}/.env",
            "debugOptions": [
                "RedirectOutput"
            ]
},

3. You can still use the "Debug test" (link/button) on top of the test case (using default python launcher) and it will end up on your break point without adding .env file. (Do you see the inconsistency? it is some how able to import the module unlike pressing F5)
   But of course it still has problems (you can see it in python test log):

Exception ignored in: <async_generator object _ag at 0x04B81BF0>
Traceback (most recent call last):
  File "C:\Python36\lib\types.py", line 27, in _ag
  File "C:\Users\Amir\.vscode\extensions\ms-python.python-2018.1.0\pythonFiles\PythonTools\ptvsd\visualstudio_py_debugger.py", line 971, in trace_func
  File "C:\Users\Amir\.vscode\extensions\ms-python.python-2018.1.0\pythonFiles\PythonTools\ptvsd\visualstudio_py_debugger.py", line 1014, in handle_call
  File "C:\Users\Amir\.vscode\extensions\ms-python.python-2018.1.0\pythonFiles\PythonTools\ptvsd\visualstudio_py_debugger.py", line 576, in should_debug_code
TypeError: 'NoneType' object is not callable

If you try it several times you will end up in a dangling debugger that you cannot even proceed with debugging!

---

I have attached my simple try out structure along with my lanuch.json so you can reproduce this:
I am using Windows 10 and latest VScode and ms-python extension.

testing_pytest_debugger.zip

[DonJayamanne]

@WindsOfHades

If you just use the default "Python" lanucher and press F5, suddenly it cannot import the target module.

What do yo umean by target module
If you're referring to pytest, then the problem is you do not have pytest in the python interpreter selected in VS Code.

is that it will run all the test cases

It is upto you to configure the debugger to run a specific test file/test case. You'll need to check the documentation of pytest on how you can run a specific test file/test case.

You can still use the "Debug test" (link/button) on top of the test case (using default python launcher) and it will end up on your break point without adding .env file. (Do you see the inconsistency?

No inconsistency here. It works perfectly fine for me, without any .env files.
Once again pressing F5 will launch the debugger. We have preconfigured debug configurations in launch.json for programs, modules, django, flask, etc. If you would like to debug something else using the F5 key, then its upto you (the user) to configure launch.json accordingly.
E.g. you have managed to get it to debug unit tests, but complain it runs all tests, in this case its upto you to configure it to run a specific test (please check pytest documentation).

If you try it several times you will end up in a dangling debugger that you cannot even proceed with debugging!

That's yet another issue, please do create a separate issue for this with instructions (along with the sample code and files such as launch.json and settings.json) to replicate this.