semantic_tokens.lua:111: attempt to index local 'request' (a nil value)

[petersid2022]

Problem

Hello everyone, I have been getting this error message:

vim.schedule callback: ...local/share/nvim/runtime/lua/vim/lsp/semantic_tokens.lua:322: ...local/share/nvim/runti
me/lua/vim/lsp/semantic_tokens.lua:111: attempt to index local 'request' (a nil value)
stack traceback:
        [builtin#36]: at 0x6302c8b64250
        ...local/share/nvim/runtime/lua/vim/lsp/semantic_tokens.lua:322: in function 'handler'
        /usr/local/share/nvim/runtime/lua/vim/lsp/client.lua:752: in function ''
        vim/_editor.lua: in function <vim/_editor.lua:0>

multiple times per session (mainly when I use the language server but once it gets triggered once it keeps happening even when I scroll around).

It seems to be related to this 23ddb20

My nvim config is very simple:
https://gist.github.com/petersid2022/c17ce2adeb77f358d215df5e23885c04

Thanks in advance!

Steps to reproduce

• Build neovim on 5299967
• Have a working language server config.
• Open file, let the language server attach to it properly and do something like vim.lsp.buf.definition()

Expected behavior

This shouldn't be happening :)

Nvim version (nvim -v)

v0.12.0-dev-1733+g5299967551

Vim (not Nvim) behaves the same?

no

Operating system/version

Ubuntu 24.04.3 LTS (wsl2, kernel 6.6.87.2-microsoft-standard-WSL2)

Terminal name/version

Windows Terminal Preview using tmux next-3.7

$TERM environment variable

tmux-256color

Installation

Build from source

[petersid2022]

Update, I can confirm that this issue persists even in older versions than the one that I originally thought was the culprit. Below is the output of nvim -v after I compiled on acfb9bc614

NVIM v0.12.0-dev-1731+gacfb9bc614
Build type: Release
LuaJIT 2.1.1764270282
Run "nvim -V1 -v" for more info

I'm still getting:

vim.schedule callback: ...local/share/nvim/runtime/lua/vim/lsp/semantic_tokens.lua:322: ...local/share/nvim/runti
me/lua/vim/lsp/semantic_tokens.lua:111: attempt to index local 'request' (a nil value)
stack traceback:
        [builtin#36]: at 0x5869182a01c0
        ...local/share/nvim/runtime/lua/vim/lsp/semantic_tokens.lua:322: in function 'handler'
        /usr/local/share/nvim/runtime/lua/vim/lsp/client.lua:752: in function ''
        vim/_editor.lua: in function <vim/_editor.lua:0>

[SanchayanMaity]

Also facing this issue when using clangd, but, not on rust-analyzer. May be the call around line 423 in semantic_tokens.lua which changed recently with that commit is the problem.

  local request_idx = range or FULL
 .....
  local highlights =
    tokens_to_ranges(tokens, self.bufnr, client, state.active_requests[request_idx])

Workaround for me here right now is,

  if client.name == 'clangd' then
    -- Opt out of semantic highlighting
    client.server_capabilities.semanticTokensProvider = nil
  end

[petersid2022]

Thank you! I can confirm this works in my setup as well. I forgot to mention earlier that I'm using clangd.
Though now, by disabling semanticTokensProvider and since I wasn't using any tree-sitter highlighting beforehand everything looks a bit plain now. that's on me!!

[seblyng]

Related to this: #36705

I have talked to @tris203, and we believe something like this might fix it:

diff --git a/runtime/lua/vim/lsp/semantic_tokens.lua b/runtime/lua/vim/lsp/semantic_tokens.lua
index b636d73eae..343377f1dd 100644
--- a/runtime/lua/vim/lsp/semantic_tokens.lua
+++ b/runtime/lua/vim/lsp/semantic_tokens.lua
@@ -108,7 +108,7 @@ local function tokens_to_ranges(data, bufnr, client, request)
         vim.schedule(function()
           coroutine.resume(co, util.buf_versions[bufnr])
         end)
-        if request.version ~= coroutine.yield() then
+        if not request or request.version ~= coroutine.yield() then
           -- request became stale since the last time the coroutine ran.
           -- abandon it by yielding without a way to resume
           coroutine.yield()

I have been running latest master with this patch for almost an hour, and I haven't gotten an error or any weird behavior with this patch at least

[petersid2022]

I can confirm that this patch works! Thank you so much!