Disabling "justMyCode" does not allow me to step into stdlib functions

[calebho]

Environment data

• VS Code version: 1.42.1
• Extension version (available under the Extensions sidebar): 2020.2.64397
• OS and version: Darwin x64 19.3.0
• Python version (& distribution if applicable, e.g. Anaconda): Anaconda 3.7
• Type of virtual environment used (N/A | venv | virtualenv | conda | ...): conda
• Relevant/affected Python packages and their versions:
• Relevant/affected Python-related VS Code extensions and their versions:
• Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info How to update the language server to the latest stable version vscode-python#3977): Jedi
• Value of the python.languageServer setting: Microsoft

Expected behaviour

Debugger steps into standard library function

Actual behaviour

Debugger steps over standard library function

Steps to reproduce:

I think this is similar to microsoft/vscode-python#7347 so I'm copying the repro steps except for launch.json

1. launch.json (note that I tried with and without "console": "integratedTerminal"):

{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Debug Tests",
      "type": "python",
      "request": "test",
      // "console": "integratedTerminal",
      "justMyCode": false
    }
  ]
}

2. test_main.py:

import subprocess


def test_main():
    a = subprocess.call(["ls"])  # set breakpoint here
    print(a)

3. Configure pytest as the testing framework
4. Click the "Debug Test" lens prompt over test_main
5. At the breakpoint, click step into
6. Observe that instead the debugger steps over the subprocess.call call

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

n/a

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)

[Extension Host] Info Python Extension: 2020-02-26 15:28:03: Cached data exists getEnvironmentVariables, extension-output-#4

[karthiknadig]

@calebho Can you share the lines from the Python Output panel?

[Aniket-Singla]

I am also getting exceptions when justMyCode is disabled :

Exception has occurred: FileNotFoundError
[WinError 2] The system cannot find the file specified: 'C:\\Users\\asingla\\AppData\\Local\\Continuum\\anaconda3\\envs\\codprog_website\\python37.zip'
  File "C:\Users\asingla\AppData\Local\Continuum\anaconda3\envs\codprog_website\Lib\genericpath.py", line 19, in exists
    os.stat(path)
  File "C:\Users\asingla\AppData\Local\Continuum\anaconda3\envs\codprog_website\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\asingla\AppData\Local\Continuum\anaconda3\envs\codprog_website\Lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)

It works fine when justMyCode is set to true.
My Configuration is :

{
            "name": "Python: Django",
            "type": "python",
            "request": "launch",
            "cwd": "${workspaceFolder}/app",
            "program": "${workspaceFolder}/app/manage.py",
            "console": "integratedTerminal",
            "args": [
                "runserver",
                "--noreload",
                "--nothreading"
            ],
            "justMyCode": false,
            "django": true,
            // "envFile": ""
        }

[karthiknadig]

@calebho try the same thing after adding this to your user settings:

    "python.experiments.optInto": [
        "PtvsdWheels37 - experiment"
    ],

This should use the newer version of the debugger that should handle subprocess better.

@Aniket-Singla This is unrelated to what you are seeing. Thanks for creating a separate issue.

[ashleybartlett]

I too am experiencing a similar issue, using a Django project. This was working as expected, and a recent update caused it to not be respecting the justMyCode flag. It stopped working last week.

what is in my launch.json

{
      "name": "Python: Debug Unit Tests",
      "type": "python",
      "request": "test",
      "console": "integratedTerminal",
      "justMyCode": false
    }

If I set a breakpoint in my code base, and try to step into a django lib call, I get the error

Frame skipped from debugging during step-in.
Note: may have been skipped because of "justMyCode" option (default == true).

This is from the VSCode console

[Extension Host] Info Python Extension: 2020-03-03 16:41:59: Cached data exists getEnvironmentVariables, /Users/abartlett/src/ssam/src/accessmgr/tests/test_views_access_access_level_v2.py
2console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:41:59: > ~/.virtualenvs/ssam/bin/python ~/.vscode/extensions/ms-python.python-2020.2.64397/pythonFiles/symbolProvider.py ~/src/ssam/src/accessmgr/tests/test_views_access_access_level_v2.py
console.ts:137 [Extension Host] Warn Python Extension: 2020-03-03 16:42:02: Unknown command line option passed into args parser for tests '-vv'. Please report on https://github.com/Microsoft/vscode-python/issues/new
t.log @ console.ts:137
$logExtensionHostMessage @ mainThreadConsole.ts:39
_doInvokeHandler @ rpcProtocol.ts:398
_invokeHandler @ rpcProtocol.ts:383
_receiveRequest @ rpcProtocol.ts:299
_receiveOneMessage @ rpcProtocol.ts:226
(anonymous) @ rpcProtocol.ts:101
fire @ event.ts:582
fire @ ipc.net.ts:453
_receiveMessage @ ipc.net.ts:733
(anonymous) @ ipc.net.ts:592
fire @ event.ts:582
acceptChunk @ ipc.net.ts:239
(anonymous) @ ipc.net.ts:200
t @ ipc.net.ts:28
emit @ events.js:200
addChunk @ _stream_readable.js:294
readableAddChunk @ _stream_readable.js:275
Readable.push @ _stream_readable.js:210
onStreamRead @ internal/stream_base_commons.js:166
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: Shell path 'Python Debug Console'
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: Shell path identified as shell 'other'
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: Terminal name 'Python Debug Console' identified as shell 'other'
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: [object Object]. Shell identified as other (Terminal name is Python Debug Console)
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: Shell path '/usr/local/bin/zsh'
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: Shell path identified as shell 'zsh'
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: Terminal shell path '/usr/local/bin/zsh' identified as shell 'zsh'
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: [object Object]. Shell identified as zsh (Terminal name is Python Debug Console)
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: Shell identified as 'zsh'
2console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:03: Cached data exists getEnvironmentVariables, /Users/abartlett/src/ssam/src/accessmgr/tests/test_views_access_access_level_v2.py
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:15: Cached data exists getEnvironmentVariables, /Users/abartlett/src/ssam
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:23: Cached data exists getEnvironmentVariables, /Users/abartlett/src/ssam
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:23: Cached data exists getEnvironmentVariables, /Users/abartlett/src/ssam/src/accessmgr/rules.py
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:23: > ~/.virtualenvs/ssam/bin/python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s ~/src/ssam/src/accessmgr/rules.py
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:23: cwd: ~/src/ssam
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:23: > ~/.virtualenvs/ssam/bin/python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s ~/src/ssam/src/accessmgr/rules.py
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:23: cwd: ~/src/ssam
console.ts:137 [Extension Host] Info Python Extension: 2020-03-03 16:42:29: Cached data exists getEnvironmentVariables, /Users/abartlett/src/ssam

VS Code version: 1.42.1
Extension version (available under the Extensions sidebar): 2020.2.64379
OS and version: Darwin x64 19.3.0
Python version (& distribution if applicable, e.g. Anaconda): Python 3.6.8
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): virtualenv
Relevant/affected Python packages and their versions:

• pytest 5.3.2,
• pytest-django-3.7.0
  Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info How to update the language server to the latest stable version vscode-python#3977): Jedi (setting is not set)
  Value of the python.languageServer setting: Not Set

[calebho]

@calebho Can you share the lines from the Python Output panel?

Yeah sure @karthiknadig:

User belongs to experiment group 'ShowPlayIcon - start'
User belongs to experiment group 'ShowExtensionSurveyPrompt - control'
User belongs to experiment group 'DebugAdapterFactory - experiment'
User belongs to experiment group 'PtvsdWheels37 - experiment'
User belongs to experiment group 'UseTerminalToGetActivatedEnvVars - control'
User belongs to experiment group 'AA_testing - control'
User belongs to experiment group 'WebHostNotebook - control'
> conda --version
> pyenv root
> python3.7 -c "import sys;print(sys.executable)"
> python3.6 -c "import sys;print(sys.executable)"
> python3 -c "import sys;print(sys.executable)"
> python2 -c "import sys;print(sys.executable)"
> python -c "import sys;print(sys.executable)"
> ~/miniconda3/bin/python -c "import sys;print(sys.executable)"
> conda info --json
> ~/miniconda3/bin/python -c "import notebook"
> ~/miniconda3/bin/python -c "import jupyter"
> ~/miniconda3/bin/python -m jupyter kernelspec --version
> ~/miniconda3/bin/python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s ~/scratch/vscode-test/test_main.py
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s ~/scratch/vscode-test/test_main.py
cwd: ~/scratch/vscode-test
Starting Jedi Python language engine.
##########Linting Output - flake8##########
> conda env list
> ~/miniconda3/bin/python -c "import sys;print(sys.prefix)"
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -c "import sys;print(sys.prefix)"
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -c "import sys;print(sys.executable)"
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -c "import sys;print(sys.executable)"
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -m site --user-site
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -m site --user-site
cwd: ~/scratch/vscode-test
> conda env list
> ~/miniconda3/bin/python completion.py
cwd: ~/.vscode-server/extensions/ms-python.python-2020.2.64397/pythonFiles
> ~/miniconda3/bin/python completion.py
cwd: ~/.vscode-server/extensions/ms-python.python-2020.2.64397/pythonFiles
> ~/miniconda3/bin/python ~/.vscode-server/extensions/ms-python.python-2020.2.64397/pythonFiles/testing_tools/run_adapter.py discover pytest -- --rootdir ~/scratch/vscode-test -s --cache-clear .
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s ~/scratch/vscode-test/main.py
cwd: ~/scratch/vscode-test
> ~/miniconda3/bin/python -m flake8 --format=%(row)d,%(col)d,%(code).1s,%(code)s:%(text)s ~/scratch/vscode-test/main.py
cwd: ~/scratch/vscode-test
##########Linting Output - flake8##########

[calebho]

@calebho try the same thing after adding this to your user settings:

    "python.experiments.optInto": [
        "PtvsdWheels37 - experiment"
    ],

This should use the newer version of the debugger that should handle subprocess better.

I don't think subprocess is relevant here. The same thing happens if I use an os function e.g.

import os

def test_main():
    os.listdir()  # set breakpoint here

Nothing changes if I add the setting you suggested to my user settings

[kvdveer]

I have found a workaround: Adding "debugStdLib": true, to the launch.json, allows me to debug outside of my code. (tested with django, not actual stdlib).

launch.json editor told me that it's not a valid property, but it does actually have an effect.

[fabioz]

This should be fixed by microsoft/debugpy@6b335f1

[fabioz]

As noted, until a new release is done, the workaround is using "debugStdLib": true.

[ashleybartlett]

This workaround worked for me.