x/tools/gopls: allow for semanticTokens maxFullFileSize to be user configurable

[Jay-Madden]

gopls version

❯ gopls version
golang.org/x/tools/gopls v0.22.0

go env

AR='ar'
CC='cc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='c++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/Users/jaymadden/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/jaymadden/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/g7/q92rbdc954j16ntmpvgt9g2h0000gn/T/go-build1901075686=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/dev/null'
GOMODCACHE='/Users/jaymadden/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/jaymadden/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.24.5/libexec'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/jaymadden/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.24.5/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.24.5'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

Generated this file https://codeberg.org/Jay-Madden/toobig/src/branch/master/big.go then loaded it with gopls in neovim.

What did you see happen?

❯ nvim --clean --headless -u scratch.lua big.go
lsp_err_code=0
lsp_err_message=semantic tokens: range /Users/jaymadden/temp/toobig/big.go too large (271612 > 100000)
result=nil

What did you expect to see?

Above^ It comes from here https://github.com/golang/tools/blob/master/gopls/internal/golang/semtok.go#L69 which is a hardcoded constant in gopls.

It would be nice to expose that value as a configurable option so we could get allow larger file sizes and take the loading hit upfront?

Editor and settings

    vim.lsp.config("gopls", {
      cmd = { "gopls" },
      filetypes = { "go", "gomod", "gowork", "gotmpl" },
      settings = {
        gopls = {
          experimentalPostfixCompletions = true,
          analyses = {
            unusedparams = true,
            shadow = false,
          },
          staticcheck = false,
          gofumpt = true,
          completeUnimported = true,
        },
      },
      init_options = {
        usePlaceholders = false,
        semanticTokens = true,
      },
    })
    vim.lsp.enable("gopls")

Logs

No response

[gabyhelp]

Related Issues

• x/tools/gopls: make file size limit configurable #57660 (closed)
• x/tools/gopls: semantic tokens no longer returned (regression in v0.22.0) #79524
• x/tools/gopls: disabling semantic token types in ui.semanticTokenTypes can break token locations #78269 (closed)
• x/tools/gopls: semantictokens are disabled #43353 (closed)
• x/tools/gopls: gopls causes bad vscode performance on large projects due to excessive LSP traffic #74876
• x/tools/gopls: can't enable semantic tokens from Emacs lsp-mode #52385

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[aputman]

This was also requested in #57660.

Talked with @adonovan about this, and we think a path forward here:

1. Do away with the range too large error message and just silently truncate the semantic token analysis to the first N tokens.
2. Make that N configurable for the likely small subset of users who run into this issue. Because as you mentioned, it would cause a large upfront loading cost which most users wouldn't want.

[Jay-Madden]

I think that makes sense with the caveat that i believe that would silently break clients that rely on the error to know if they SHOULD switch to using range based?

Ideally they'd be doing both eitherway but i dont think they all do?

[pjweinb]

How big is the loading hit? There is a substantial delay if the file is open during the initial workspace load, but I believe that is because gopls is waiting for all the type checking to be done before it responds to any LSP requests. I think that generating semantic tokens for type-checked files is quite fast.

As the comment in the code indicates, returning an error is an abuse of the LSP specification, so I'd be happy to remove the check.

[Jay-Madden]

If we could remove it that would be nice