x/tools/gopls: analyze workspace without opening code editor

[zhdaniel]

gopls version

golang.org/x/tools/gopls v0.20.0

go env

AR='ar'
CC='clang'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='clang++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='$USER/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='$USER/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/00/51v9d1hx0cz8r36v_s077_xw0000gn/T/go-build71646679=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='$USER/sandbox/golang-tools/go.mod'
GOMODCACHE='$USER/go/pkg/mod'
GOOS='darwin'
GOPATH='$USER/go'
GOROOT='$USER/local/go'
GOTELEMETRY='local'
GOTELEMETRYDIR='$USER/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='$USER/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.24.3'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

none

What did you see happen?

none

What did you expect to see?

When opening a large Go project in VS Code, gopls analyzes the project and builds index caches, which consumes a lot of CPU and memory. This often makes the computer very slow or even unusable for other tasks.

I’m wondering if it’s possible to expose this indexing process as a separate command in the gopls CLI, so that I could run it manually during idle times or set it up as a scheduled task. like:

gopls index $WORKSPACE

Editor and settings

No response

Logs

No response

[gabyhelp]

Related Issues

• x/tools/gopls: Is there a way to reduce the memory usage? #61352 (closed)
• x/tools/gopls: long initial workspace load durations for workspace with many packages #69523
• x/tools/gopls: high memory consumption on 19MLoC input #73709
• x/tools/gopls: add ability to avoid scanning an entire workspace #53278 (closed)
• x/tools/gopls: gopls causes bad vscode performance on large projects due to excessive LSP traffic #74876
• x/tools/gopls: gopls compiles after every newstart on large projects #53332 (closed)
• x/tools/gopls: high cpu usage all the time #70172
• x/tools/gopls: option to enable analysis across entire workspace #45600 (closed)
• x/tools/gopls: expensive go list run on startup #35818 (closed)
• x/tools/gopls: vscode take minutes to load packages before completion becomes available #69743 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[adonovan]

For now you can use the command gopls check cmd/main/main.go, applied to your application main package, to process and index all of its dependencies.

But in general gopls should not be making your machine slow even for large projects. We may need to change the degree of concurrency it uses. Would you mind sharing the output of gopls stats in your workspace? I'm curious just how large the project is.

[zhdaniel]

@adonovan Thanks for your solution. And the output of gopls stats in my workspace:

Initializing workspace...     2025/10/16 10:36:02 modcacheState.refreshIndex: bad semantic version v1.21.1-f!p!i!c
done (2m24.256087416s)
Gathering bug reports...      done (4.068300833s)
Querying memstats...          done (9.449666ms)
Querying workspace stats...   done (105.625µs)
Collecting directory info...  done (127.478334ms)
{
  "BugReports": [],
  "CacheDir": "$USER/Library/Caches/gopls/a163b833",
  "DirStats": {
    "Files": 5819,
    "TestdataFiles": 0,
    "GoFiles": 2199,
    "ModFiles": 1,
    "Dirs": 912
  },
  "GOARCH": "arm64",
  "GOOS": "darwin",
  "GOPACKAGESDRIVER": "",
  "GOPLSCACHE": "",
  "GoVersion": "go1.24.3",
  "GoplsVersion": "v0.20.0",
  "InitialWorkspaceLoadDuration": "2m24.256087416s",
  "MemStats": {
    "HeapAlloc": 102716000,
    "HeapInUse": 104931328,
    "TotalAlloc": 976399120
  },
  "WorkspaceStats": {
    "Files": {
      "Total": 1,
      "Largest": 91657,
      "Errs": 0
    },
    "Views": [
      {
        "GoCommandVersion": "go1.24.3",
        "AllPackages": {
          "Packages": 0,
          "LargestPackage": 0,
          "CompiledGoFiles": 0,
          "Modules": 0
        },
        "WorkspacePackages": {
          "Packages": 0,
          "LargestPackage": 0,
          "CompiledGoFiles": 0,
          "Modules": 0
        },
        "Diagnostics": 0
      }
    ]
  }
}

[zhdaniel]

Hi @adonovan , I ran gopls check main.go in my project, but it reported an error. The project builds and runs without any issue. Could you please help me understand what might be causing this?

2025/10/16 11:42:55 Error:2025/10/16 11:42:55 go/packages.Load #1: err: exit status 1: stderr: malformed import path "{{context.Compiler}}": invalid char '{'
malformed import path "--": leading dash

	view_id="1"
	snapshot=0
	directory=$WORKSPACE
	query=[$WORKSPACE/... builtin]
	packages=0
	duration=1.788060625s
2025/10/16 11:42:55 Error:2025/10/16 11:42:55 initial workspace load failed: packages.Load error: err: exit status 1: stderr: malformed import path "{{context.Compiler}}": invalid char '{'
malformed import path "--": leading dash

[adonovan]

How strange. Can you try go install golang.org/x/tools/go/packages/gopackages@master followed by gopackages main.go? That might help to narrow it down. Also, are you use a go/packages driver (GOPACKAGESDRIVER)? This environment variable is not one that the go command is concerned with so it wouldn't appear in the go env dump above (though perhaps it should).

[zhdaniel]

Emmm, it seems the gopackages main.go output looks fine:

$ gopackages main.go
Go command "command-line-arguments":
	package main
	has no exported type info
	file $WORKSPACE/main.go
	import "PRIVATE_REPO/af/common"
	import "PRIVATE_REPO/af/plugin/metrics"
	import "PRIVATE_REPO/af/plugin/notice"
	import "PRIVATE_REPO/af/plugin/pack_resp"
	import "PRIVATE_REPO/af/plugin/web_app"
	import "PRIVATE_REPO/clients/client_metrics"
	import "PRIVATE_REPO/WORKSPACE/biz/dal/cache"
	import "PRIVATE_REPO/WORKSPACE/biz/utils/abutil"
	import "PRIVATE_REPO/WORKSPACE/biz/views"
	import "PRIVATE_REPO/WORKSPACE/client"
	import "PRIVATE_REPO/WORKSPACE/setup"
	import "PRIVATE_REPO/gdp/af"
	import "PRIVATE_REPO/gdp/af/adapter"
	import "PRIVATE_REPO/gdp/gdp"
	import "PRIVATE_REPO/gopkg/env"
	import "PRIVATE_REPO/mock/mock_sdk"
	import "PRIVATE_REPO/passport/session_lib"

and I rerun go env output:

AR='ar'
CC='clang'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='clang++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='$USER/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='$USER/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/00/51v9d1hx0cz8r36v_s077_xw0000gn/T/go-build1572225358=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='$USER/go/src/WORKSPACE/go.mod'
GOMODCACHE='$USER/go/pkg/mod'
GONOPROXY='$SOME_CONFIG'
GONOSUMDB='$SOME_CONFIG'
GOOS='darwin'
GOPATH='$USER/go'
GOPRIVATE='$SOME_CONFIG'
GOPROXY='$PROXY_CONFIG'
GOROOT='$USER/local/go'
GOSUMDB='sum.golang.google.cn'
GOTELEMETRY='local'
GOTELEMETRYDIR='$USER/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='$USER/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.24.3'
GOWORK=''
PKG_CONFIG='pkg-config'

I also checked my all environment variables, no variable like GOPACKAGESDRIVER

[hyangah]

It looks like we have a clear explanation for the technical error,
but we want to think carefully about the design of the feature request itself.

1. The gopackages error seems tobe a red herring

The error malformed import path "{{context.Compiler}}": invalid char '{'
can be a symptom of a custom go wrapper script or alias
that incorrectly splits arguments on spaces (e.g., using go $* instead of go "$@").

When gopls (or go/packages under the hood) queries compiler sizes, it runs:
go list -f "{{context.GOARCH}} {{context.Compiler}}" -- unsafe

If a wrapper splits that format string, go list receives {{context.Compiler}}
as a separate positional argument and tries to parse it as an import path, failing on the {.

By default gopackages runs in imports mode,
which doesn't load types of sizes and doesn't run the same query.

Does gopackages -mode=types main.go succeed?

2. Feature request "Offline Indexing" via CLI

Before we consider exposing a dedicated indexing command in the CLI,
we want to make sure we are solving the root problem in the best way.
Forcing users to manually run commands or manage cron jobs
to keep their editor fast is generally a UX pattern we want to avoid.

• Is a CLI command the right solution? If gopls is consuming so much
  CPU/memory on startup that it makes your computer unusable, the root issue
  might be that gopls is not throttling itself properly during the initial workspace load.
  As you edit files or make major editing or git branch switching, the problem
  will occur again. How did you envision using a gopls index command?

• Alternative: Would it be better to focus on identifying the root cause
  of high resource consumption and improving gopls's automatic
  background resource management? For example, limiting CPU thread usage
  or running the initial load with lower process priority (like nice/ionice equivalents)
  so that it quietly warms the cache in the background without lagging your machine.

We will keep this open for discussion on the best way to improve the large-workspace startup experience.

[adonovan]

Thanks for investigating @hyangah. We recently received a stack of four commits as a gift from Cockroach Labs each showing a different optimization that together improved gopls indexing times significantly. We landed the first of these (https://go.dev/cl/793381) and it will be in the upcoming gopls release this month; the others will have to wait for the next release.