Developer: inspect editor tokens and scopes

[tythi]

Type: Bug

Developer: inspect editor tokens and scopes does not work when the Pylance extension is enabled.
Don't know if the rest of my extensions will be sent with this report but just to be sure I have Flake8, Python and WSL installed as extensions.
Opened a .py file from a wsl folder (not well versed enough to know if that is an accurate description but it's the best I can do) and tried to run the command 'Developer: Inspect Editor Tokens and Scopes' but did not work. Ran extension bisect to find that the 'Pylance' extension was at fault. Can still code fine without it so not to worried about knowing if it gets fixed. This is all I know, hope this helps :)

Extension version: 2023.10.10
VS Code version: Code 1.83.0 (e7e037083ff4455cf320e344325dacb480062c3c, 2023-10-03T16:12:16.321Z)
OS version: Windows_NT x64 10.0.19045
Modes:

System Info

Item	Value
CPUs	AMD Ryzen 5 5600 6-Core Processor (12 x 3493)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled
Load (avg)	undefined
Memory (System)	31.91GB (17.56GB free)
Process Argv	--crash-reporter-id 1835f4fa-d811-4e17-ac75-e306d8cdcbd4
Screen Reader	no
VM	0%

A/B Experiments

vsliv368:30146709
vsreu685:30147344
python383cf:30185419
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492cf:30256860
vslsvsres303:30308271
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscoreces:30445986
vscod805cf:30301675
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593cf:30376535
pythonvs932:30410667
vscaat:30438848
vsclangdc:30486549
c4g48928:30535728
dsvsc012:30540252
pynewext54:30695312
azure-dev_surveyone:30548225
3biah626:30602489
89544117:30613380
showlangstatbar:30737416
03d35959:30757346
24365598:30736109
pythonfmttext:30731395
fixshowwlkth:30771522
showindicator:30805244
pythongtdpath:30769146
i26e3531:30792625
pythonnosmt12:30797651
pythonidxpt:30805730
pythonnoceb:30805159
copilotsettingt:30839829
dsvsc013:30795093
dsvsc014:30804076
diffeditorv2:30821572
dsvsc015:30845448

[debonte]

I'm unable to repro this.

Can you please set "python.trace.server": "verbose" (so we include all the LSP messages in the log) and "python.analysis.logLevel": "Trace" (to get even our most verbose log output). Then reproduce the problem, and provide a the "Python Language Server" log?

[tythi]

Have put "python.trace.server": "verbose" and "python.analysis.logLevel": "Trace" in my settings.json.
Presume this is what you're looking for:

2023-10-07 14:30:09.178 [info] Params: {
    "textDocument": {
        "uri": "file://wsl.localhost/Ubuntu/home/../something.py"
    },
    "position": {
        "line": 63,
        "character": 11
    }
}


2023-10-07 14:30:12.583 [info] [Trace - 2:30:12 PM] Sending request 'textDocument/codeAction - (15)'.
2023-10-07 14:30:12.583 [info] Params: {
    "textDocument": {
        "uri": "file://wsl.localhost/Ubuntu/home/../something.py"
    },
    "range": {
        "start": {
            "line": 63,
            "character": 11
        },
        "end": {
            "line": 63,
            "character": 11
        }
    },
    "context": {
        "diagnostics": [],
        "triggerKind": 2
    }
}


2023-10-07 14:30:14.566 [info] [Trace - 2:30:14 PM] Sending request 'textDocument/semanticTokens/full - (16)'.
2023-10-07 14:30:14.566 [info] Params: {
    "textDocument": {
        "uri": "file://wsl.localhost/Ubuntu/home/../something.py"
    }
}

Hovered over a function from which I'd guess the first log came, then clicked on the function and lastly through F1 ran Developer: Inspect Tokens and Scopes. Got the loading box but it never finishes.

[tythi]

After turning Pylance back off got this doing the same actions:

2023-10-07 14:35:44.241 [info] [Trace - 2:35:44 PM] Sending request 'textDocument/hover - (10)'.
2023-10-07 14:35:44.241 [info] Params: {
    "textDocument": {
        "uri": "file://wsl.localhost/Ubuntu/home/../something.py"
    },
    "position": {
        "line": 63,
        "character": 16
    }
}


2023-10-07 14:35:44.258 [info] [Trace - 2:35:44 PM] Received response 'textDocument/hover - (10)' in 17ms.
2023-10-07 14:35:44.258 [info] Result: {
    "contents": {
        "kind": "markdown",
        "value": "```python\ndef a_func(args)\n```\n---\n**Full name:** `something.a_func`"
    },
    "range": {
        "start": {
            "line": 63,
            "character": 4
        },
        "end": {
            "line": 63,
            "character": 20
        }
    }
}


2023-10-07 14:35:46.556 [info] [Trace - 2:35:46 PM] Sending request 'textDocument/documentHighlight - (11)'.
2023-10-07 14:35:46.556 [info] Params: {
    "textDocument": {
        "uri": "file://wsl.localhost/Ubuntu/home/../something.py"
    },
    "position": {
        "line": 63,
        "character": 16
    }
}


2023-10-07 14:35:46.573 [info] [Trace - 2:35:46 PM] Received response 'textDocument/documentHighlight - (11)' in 17ms.
2023-10-07 14:35:46.573 [info] Result: [
    {
        "range": {
            "start": {
                "line": 63,
                "character": 4
            },
            "end": {
                "line": 63,
                "character": 20
            }
        }
    },
    {
        "range": {
            "start": {
                "line": 129,
                "character": 27
            },
            "end": {
                "line": 129,
                "character": 43
            }
        }
    }
]


2023-10-07 14:35:46.822 [info] [Trace - 2:35:46 PM] Sending request 'textDocument/codeAction - (12)'.
2023-10-07 14:35:46.822 [info] Params: {
    "textDocument": {
        "uri": "file://wsl.localhost/Ubuntu/home/../something.py"
    },
    "range": {
        "start": {
            "line": 63,
            "character": 16
        },
        "end": {
            "line": 63,
            "character": 16
        }
    },
    "context": {
        "diagnostics": [],
        "triggerKind": 2
    }
}


2023-10-07 14:35:49.375 [info] [Trace - 2:35:49 PM] Received response 'textDocument/codeAction - (12)' in 2553ms.
2023-10-07 14:35:49.375 [info] No result returned.

This worked fine and could see the information.

[debonte]

After turning Pylance back off got this doing the same actions:

By "off", do you mean that you switched to Jedi?

2023-10-07 14:30:14.566 [info] [Trace - 2:30:14 PM] Sending request 'textDocument/semanticTokens/full - (16)'.

Are you not seeing a response to this semantic tokens request? On my machine, I see the following:

2023-10-09 15:47:56.192 [info] [Trace - 3:47:56 PM] Sending request 'textDocument/semanticTokens/full - (17)'.
2023-10-09 15:47:56.193 [info] Params: {
    "textDocument": {
        "uri": "file:///mnt/c/users/erikd/issues/4924/test2.py"
    }
}


2023-10-09 15:47:56.230 [info] [Trace - 3:47:56 PM] Received response 'textDocument/semanticTokens/full - (17)' in 38ms.
2023-10-09 15:47:56.230 [info] Result: {
    "resultId": "1696891676198",
    "data": [
        0,
        4,
        3,
        12,
        1,
        ...

[tythi]

With off I mean disabling Pylance, after which the only extensions that I have and are enabled are Flake8, Python and WSL.
When I do 'Developer: Inspect Tokens and Scopes', then hover over a function and click the function, this is all I get:

2023-10-10 08:36:34.039 [info] Params: {
    "textDocument": {
        "uri": "file://wsl.localhost/../something.py"
    },
    "position": {
        "line": 37,
        "character": 11
    }
}


2023-10-10 08:36:34.058 [info] [Trace - 8:36:34 AM] Received response 'textDocument/hover - (26)' in 19ms.
2023-10-10 08:36:34.058 [info] Result: {
    "contents": {
        "kind": "markdown",
        "value": "```python\ndef a_func(args)\n```\n---\n**Full name:** `something.a_func`"
    },
    "range": {
        "start": {
            "line": 37,
            "character": 4
        },
        "end": {
            "line": 37,
            "character": 20
        }
    }
}


2023-10-10 08:36:35.358 [info] [Trace - 8:36:35 AM] Sending request 'textDocument/documentHighlight - (27)'.
2023-10-10 08:36:35.358 [info] Params: {
    "textDocument": {
        "uri": "file://wsl.localhost/../something.py"
    },
    "position": {
        "line": 37,
        "character": 12
    }
}


2023-10-10 08:36:35.377 [info] [Trace - 8:36:35 AM] Received response 'textDocument/documentHighlight - (27)' in 19ms.
2023-10-10 08:36:35.377 [info] Result: [
    {
        "range": {
            "start": {
                "line": 37,
                "character": 4
            },
            "end": {
                "line": 37,
                "character": 20
            }
        }
    },
    {
        "range": {
            "start": {
                "line": 118,
                "character": 27
            },
            "end": {
                "line": 118,
                "character": 43
            }
        }
    }
]


2023-10-10 08:36:35.621 [info] [Trace - 8:36:35 AM] Sending request 'textDocument/codeAction - (28)'.
2023-10-10 08:36:35.621 [info] Params: {
    "textDocument": {
        "uri": "file://wsl.localhost/../something.py"
    },
    "range": {
        "start": {
            "line": 37,
            "character": 12
        },
        "end": {
            "line": 37,
            "character": 12
        }
    },
    "context": {
        "diagnostics": [],
        "triggerKind": 2
    }
}


2023-10-10 08:36:38.538 [info] [Trace - 8:36:38 AM] Received response 'textDocument/codeAction - (28)' in 2917ms.
2023-10-10 08:36:38.538 [info] No result returned.

[debonte]

With off I mean disabling Pylance

Ok. With the Pylance extension disabled and python.languageServer set to Default you'll be using Jedi as the Python language server. Jedi doesn't support semantic tokens, so there's no semantic token request from VS Code in that case.

Can you please re-enable the Pylance extension, reproduce the problem, and provide the resulting full "Python Language Server" log? If it's too large to include in the text of a comment, you can copy it to a file and drag that file into the comment as an attachment.

[tythi]

Right so about one to two days ago my friend who had the same problem said that their syntax checking and variable colours had returned, though it was "not fast" like showing up when typing (I assume). Now when I enabled Pylance again I was not able to reproduce the problem.
text.txt
This is what was in the ouput. Though python.languageServer was not explicitly set to Default (I don't know what its default is) in settings.json.

[debonte]

Though python.languageServer was not explicitly set to Default (I don't know what its default is) in settings.json.

The default is Default.

Now when I enabled Pylance again I was not able to reproduce the problem.

Is VS Code prompting you to reload with a "Reload Required" button on Pylance in the extension pane? The contents of text.txt are either not the full "Python Language Server" log or not from Pylance (i.e. Jedi is still running instead).

[tythi]

No reload was required when I enabled Pylance.
text.txt
Found this in a read only file so hope this is the right one.

[debonte]

What I'd like to see is the contents of the "Python Language Server" log in the Output pane.

The first line should be something like:

Pylance (sync) language server 2023.10.11 (pyright 71ed1178) starting

[tythi]

For whatever reason your last response wasn't showing up until I commented myself.
Up until text.txt I have been copying text from that output tab.
text.txt
This is what I got when doing Ctrl + A.

[debonte]

I'm still unable to reproduce this. Can you see if it repros for you on the most recent prerelease build of Pylance (2023.12.102)?

[github-actions]

This issue has been closed automatically because it needs more information and has not had recent activity. If the issue still persists, please reopen with the information requested. Thanks.