x/tools/gopls: Highlighting/Semantic Tokens: Gopls is not generating tokens for constants/readonly or packages/namespaces.

[ChrisPittsley]

I am running Neovim v0.10.4 with gopls v0.17.1.

EDIT: manually updated the gopls version installed by mason to v0.18.1 and see the same behavior.

For example, in the code below the constant "myUrl" will be highlighted as a constant when it is declared (:Inspect = @constant) but not when it is used in main() (:Inspect = @variable). My understanding is that there's supposed to be a group/modifier called @lsp.mod.readonly that gopls will mark a constant with, but this does not appear in the list returned by :highlight.

The usages of http and log are also marked as @variable.go when they're used within the two functions, except when http (or any other package) is used to specify a type like in the function signature for printStatus() or the declaration of resp in main() (:Inspect = @module). My understanding is that gopls should be marking these as "namespace", which is listed by :highlight.

I know at least some information is making it from gopls to Treesitter/the neovim highlighter; if I disable gopls, most of the coloring of things like function calls and non-builtin types goes away.

Here's my LSP configuration, should be pretty much the same as the one suggested by LazyVim. Not sure if that workaround for semantic token support is still needed, but I see the same problem with the defaults (gopls.setup({})).

require('mason').setup({
    ensure_installed = {
        "goimports",
        "gofumpt",
        "gomodifytags",
        "impl",
        "delve"
    }
})

require('mason-lspconfig').setup({
    ensure_installed = {'gopls'}
})

local lspconfig = require("lspconfig")

lspconfig.gopls.setup({
  opts = {
    servers = {
      gopls = {
        settings = {
          gopls = {
            gofumpt = true,
              codelenses = {
                gc_details = false,
                generate = true,
                regenerate_cgo = true,
                run_govulncheck = true,
                test = true,
                tidy = true,
                upgrade_dependency = true,
                vendor = true,
          },
          hints = {
            assignVariableTypes = true,
            compositeLiteralFields = true,
            compositeLiteralTypes = true,
            constantValues = true,
            functionTypeParameters = true,
            parameterNames = true,
            rangeVariableTypes = true,
          },
          analyses = {
            nilness = true,
            unusedparams = true,
            unusedwrite = true,
            useany = true,
          },
          usePlaceholders = true,
          completeUnimported = true,
          staticcheck = true,
          directoryFilters = { "-.git", "-.vscode", "-.idea", "-.vscode-test", "-node_modules" },
          semanticTokens = true,
          },
        },
      },
    },
  },
  setup = {
    gopls = function(_, opts)
      -- workaround for gopls not supporting semanticTokensProvider
      -- https://github.com/golang/go/issues/54531#issuecomment-1464982242
      LazyVim.lsp.on_attach(function(client, _)
        if not client.server_capabilities.semanticTokensProvider then
          local semantic = client.config.capabilities.textDocument.semanticTokens
          client.server_capabilities.semanticTokensProvider = {
            full = true,
            legend = {
              tokenTypes = semantic.tokenTypes,
              tokenModifiers = semantic.tokenModifiers,
            },
            range = true,
          }
        end
      end, "gopls")
      -- end workaround
    end,
  },
})

[gabyhelp]

Related Issues

• x/tools/gopls: Missing semantic tokens in the declaration of function type variables #66809 (closed)
• x/tools/gopls: setting semanticTokenTypes has no effect #71964 (closed)
• x/tools/gopls: not returning semanticTokensProvider capability #54531 (closed)
• x/tools/gopls: semantictokens are disabled #43353 (closed)
• x/tools/gopls: semantic tokens appear to have broken since v0.13.2 #63415 (closed)
• x/tools/gopls: Labels are treated as parameters when assigning semantic tokens #65494 (closed)
• x/tools/gopls: Semantic Highlighting for builtin `true` and `false` broken in 0.17.0 #70934 (closed)
• x/tools/gopls: semantic tokens should report modifiers for the top-level type constructor of each symbol (e.g. struct, number, pointer, interface, etc) #68975 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[findleyr]

CC @pjweinb

[adonovan]

This is what gopls emits when supplied (by the CLI client) the default set of token types and modifiers (which are listed at the bottom):

$ cat a.go
package main

const myURL = "foo"

func main() {
	_ = myURL
}

$ gopls semtok ./a.go
/*⇒7,keyword,[]*/package /*⇒4,namespace,[]*/main

/*⇒5,keyword,[]*/const /*⇒5,variable,[definition readonly]*/myURL = /*⇒5,string,[]*/"foo"

/*⇒4,keyword,[]*/func /*⇒4,function,[definition signature]*/main() {
	/*⇒1,variable,[]*/_ /*⇒1,operator,[]*/= /*⇒5,variable,[readonly]*/myURL
}

In other words, the first occurrence (the def) of myURL is variable + [definition readonly], and the second (the use) is variable + [readonly]. This seems like the correct answer to me.

Defaults used by CLI client:

// TokenTypes is a slice of types gopls will return as its server capabilities.
var TokenTypes = []Type{
	TokNamespace,
	TokType,
	TokTypeParam,
	TokParameter,
	TokVariable,
	TokFunction,
	TokMethod,
	TokMacro,
	TokKeyword,
	TokComment,
	TokString,
	TokNumber,
	TokOperator,
	TokLabel,
}

// TokenModifiers is a slice of modifiers gopls will return as its server
// capabilities.
var TokenModifiers = []Modifier{
	// LSP 3.18 standard modifiers.
	ModDefinition,
	ModReadonly,
	ModDefaultLibrary,
	// Additional custom modifiers.
	ModArray,
	ModBool,
	ModChan,
	ModFormat,
	ModInterface,
	ModMap,
	ModNumber,
	ModPointer,
	ModSignature,
	ModSlice,
	ModString,
	ModStruct,
}

[ChrisPittsley]

Thanks @adonovan, I ran gopls from the command line on my example, and it is emitting the correct tokens and modifiers:

/*⇒7,keyword,[]*/package /*⇒4,namespace,[]*/main

/*⇒6,keyword,[]*/import (
        "log"/*⇐3,namespace,[]*/
        "net/http"/*⇐4,namespace,[]*/
)

/*⇒5,keyword,[]*/const /*⇒5,variable,[definition readonly]*/myUrl = /*⇒20,string,[]*/"http://example.com"

/*⇒4,keyword,[]*/func /*⇒4,function,[definition signature]*/main() {
        /*⇒3,keyword,[]*/var /*⇒3,variable,[definition interface]*/err /*⇒5,type,[defaultLibrary interface]*/error
        /*⇒3,keyword,[]*/var /*⇒4,variable,[definition pointer]*/resp /*⇒1,operator,[]*/*/*⇒4,namespace,[]*/http./*⇒8,type,[struct]*/Response
        /*⇒4,variable,[pointer]*/resp, /*⇒3,variable,[interface]*/err /*⇒1,operator,[]*/= /*⇒4,namespace,[]*/http./*⇒3,function,[signature]*/Get(/*⇒5,variable,[readonly]*/myUrl)
        /*⇒2,keyword,[]*/if /*⇒3,variable,[interface]*/err /*⇒2,operator,[]*/!= /*⇒3,variable,[readonly defaultLibrary]*/nil {
                /*⇒3,namespace,[]*/log./*⇒5,function,[signature]*/Fatal(/*⇒3,variable,[interface]*/err)
        }
        /*⇒11,function,[signature]*/printStatus(/*⇒4,variable,[pointer]*/resp)
}

/*⇒4,keyword,[]*/func /*⇒11,function,[definition signature]*/printStatus(/*⇒4,parameter,[definition pointer]*/resp /*⇒1,operator,[]*/*/*⇒4,namespace,[]*/http./*⇒8,type,[struct]*/Response) {
        /*⇒3,namespace,[]*/log./*⇒5,function,[signature]*/Print(/*⇒4,parameter,[pointer]*/resp./*⇒6,variable,[string]*/Status)
}

So this must be an issue with the nvim lsp plugin.

[adonovan]

I'll close this issue then. Thanks.