Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: gopls walks entire filesystem of a go module #52993

Open
slon opened this issue May 19, 2022 · 5 comments
Open

x/tools/gopls: gopls walks entire filesystem of a go module #52993

slon opened this issue May 19, 2022 · 5 comments
Assignees
Labels
gopls Tools
Milestone

Comments

@slon
Copy link
Contributor

@slon slon commented May 19, 2022

gopls version

Build info
----------
golang.org/x/tools/gopls v0.8.3
    golang.org/x/tools/gopls@(devel)
    github.com/BurntSushi/toml@v1.0.0 h1:dtDWrepsVPfW9H/4y7dDgFc2MBUSeJhlaDtK13CxFlU=
    github.com/google/go-cmp@v0.5.7 h1:81/ik6ipDQS2aGcBfIN5dHDB36BwrStyeAQquSYCV4o=
    github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
    golang.org/x/exp/typeparams@v0.0.0-20220218215828-6cf2b201936e h1:qyrTQ++p1afMkO4DPEeLGq/3oTsdlvdH4vqZUBWzUKM=
    golang.org/x/mod@v0.6.0-dev.0.20220106191415-9b9b3d81d5e3 h1:kQgndtyPBW/JIYERgdxfwMYh3AVStj88WQTlNDi2a+o=
    golang.org/x/sync@v0.0.0-20210220032951-036812b2e83c h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ=
    golang.org/x/sys@v0.0.0-20220209214540-3681064d5158 h1:rm+CHSpPEEW2IsXUib1ThaHIjuBVZjxNgSKmBLFfD4c=
    golang.org/x/text@v0.3.7 h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
    golang.org/x/tools@v0.1.11-0.20220407163324-91bcfb1bdf9c h1:rYiuK/jn6TCxjAq1hQ6MR6pgOeP3A5gIHDxgrQ8Vbys=
    golang.org/x/vuln@v0.0.0-20220324005316-18fd808f5c7f h1:9dMzk88fnONra7zrEalqkRMGa9jMGf9B5mdzhYVyI28=
    golang.org/x/xerrors@v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
    honnef.co/go/tools@v0.3.0 h1:2LdYUZ7CIxnYgskbUZfY7FPggmqnh6shBqfWa8Tn3XU=
    mvdan.cc/gofumpt@v0.3.0 h1:kTojdZo9AcEYbQYhGuLf/zszYthRdhDNDUi2JKTxas4=
    mvdan.cc/xurls/v2@v2.4.0 h1:tzxjVAj+wSBmDcF6zBB7/myTy3gX9xvi8Tyr28AuQgc=
go: go1.18

go env

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/prime/.cache/go-build"
GOENV="/home/prime/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/prime/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/prime/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/prime/.ya/tools/v4/3102076311"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/prime/.ya/tools/v4/3102076311/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/prime/arc/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/var/tmp/go-build3354655002=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Our internal version control system is working over fuse. Big mono-repository is mounted as a virtual file system and loaded lazily on demand. Whole repository is a single go module with single go.mod file at the root.

When I open subset of big mono-repository in vscode, gopls starts walking all directories of the repository, breaking lazy load.

What did you expect to see?

gopls should interact only with directories added to workspace.

What did you see instead?

gopls scans all directories of a go module.

This issue originates imports cache. https://github.com/golang/tools/blob/cf66aec62b42a726b36e083c001d3b3231cba2b3/internal/imports/fix.go#L675-L689

It loads all directories from disk without any way of setting up excludes.

I tried adding exclude filter to gopls configuration. This patch fixed issue for me: slon/tools@bfcfbb1

Editor and settings

{
	"folders": [
		{
			"path": "arc/yt"
		},
		{
			"path": "arc/mapreduce"
		},
		{
			"path": "arc/library/cpp"
		},
		{
			"path": "arc/junk/prime"
		},
		{
			"path": "arc/yp"
		},
		{
			"path": "arc/devtools/yamaker"
		},
		{
			"path": "arc/devtools/ya"
		},
		{
			"path": "arc/web/app_host/api/service"
		},
		{
			"path": "arc/library"
		},
		{
			"path": "arc/contrib/libs/tcmalloc"
		}
	],
	"settings": {
        "go.toolsEnvVars": {
            "CGO_ENABLED": "0"
        }, 
        "go.logging.level": "verbose", 
        "go.goroot": "/home/prime/.ya/tools/v4/1272507071", 
        "gopls": {
            "verboseOutput": true, 
            "formatting.local": "a.yandex-team.ru", 
            "build.env": {
                "CGO_ENABLED": "0"
            }, 
            "build.expandWorkspaceToModule": false,
        }, 
        "files.watcherExclude": {
            "**": true
        }
    }
}

Logs

@gopherbot gopherbot added Tools gopls labels May 19, 2022
@gopherbot gopherbot added this to the Unreleased milestone May 19, 2022
@findleyr
Copy link
Contributor

@findleyr findleyr commented May 19, 2022

Thanks for the issue, and for the patch. We should reuse the directoryFilters setting, or go.work files, rather than add a new setting. But otherwise what you did looks good.

Rationalizing these settings with go.work is in the plan for gopls@v0.9.0, so I'm including this in that milestone (our next release).

@findleyr findleyr removed this from the Unreleased milestone May 19, 2022
@findleyr findleyr added this to the gopls/v0.9.0 milestone May 19, 2022
@slon
Copy link
Contributor Author

@slon slon commented May 20, 2022

directoryFilters setting is broken in my case.

My workspace consists of multiple folders. directoryFilter is evaluated relative to each directory. But import cache filter should be evaluated relative to module root.

Because of this issue, I added separate setting in my proof-of-concept patch.

It is possible to reuse directoryFilter, but this would require changing semantics in case where workspace directory is not module root.

@findleyr
Copy link
Contributor

@findleyr findleyr commented May 20, 2022

I see! Thank you for clarifying. We will fix this.

@findleyr
Copy link
Contributor

@findleyr findleyr commented Jun 24, 2022

Our plan to fix this is to extend directoryFilters to support wildcards, and pass this to goimports. @dle8 is working on the former feature.

https://go.dev/issue/52784 also fixed a bug in gopls that caused scanning of GOPATH when using go.work.

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 27, 2022

Change https://go.dev/cl/414454 mentions this issue: internal/lsp: Add exclude filters to gopls ModuleResolver.scan

@findleyr findleyr removed this from the gopls/v0.9.0 milestone Jun 28, 2022
@findleyr findleyr added this to the gopls/v0.9.1 milestone Jun 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gopls Tools
Projects
None yet
Development

No branches or pull requests

4 participants