x/tools/gopls: imports.Resolver is *imports.gopathResolver, not *imports.ModuleResolver

[gwillem]

What did you do?

After vscode (1.41.1-1576681836 @ Ubuntu 16.04) upgraded gopls from v0.2.2 to v0.3.1, it crashes on start.

I could not isolate it to specific code. Gopls only crashes inside vscode. My project (some 66 .go files, using go-yara C interface) does not produce a crash when I run

find . -name '*.go' | grep -v vendor | xargs gopls -rpc.trace -v check

I have downgraded to v0.2.2 for now.

What did you expect to see?

No crash

What did you see instead?

vscode reports gopls output:

[Info  - 3:58:01 PM] 2020/02/07 15:58:01 Build info
----------
golang.org/x/tools/gopls v0.3.1
    golang.org/x/tools/gopls@v0.3.1 h1:yNTWrf4gc4Or0UecjOas5pzOa3BL0WDDyKDV4Wz5VaM=
    github.com/BurntSushi/toml@v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/go-diff@v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
    golang.org/x/mod@v0.1.1-0.20191105210325-c90efee705ee h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
    golang.org/x/sync@v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
    golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7 h1:hWZVyLW37WdETuLIGQMvQIhMfXXAOANmAIEAluZMy3c=
    golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
    honnef.co/go/tools@v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM=
    mvdan.cc/xurls/v2@v2.1.0 h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=

Go info
-------
go version go1.13.7 linux/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/willem/.cache/go-build"
GOENV="/home/willem/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/willem/code/golang"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/willem/Sync/golang/src/github.com/gwillem/project/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-L/opt/yara-3.8.1/libyara/.libs -lyara -lm"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build847115564=/tmp/go-build -gno-record-gcc-switches"

[Info  - 3:58:07 PM] 2020/02/07 15:58:07 go/packages.Load
	snapshot = 0
	query = [./... builtin]
	packages = 42
panic: interface conversion: imports.Resolver is *imports.gopathResolver, not *imports.ModuleResolver

goroutine 9061 [running]:
golang.org/x/tools/internal/lsp/cache.(*view).RunProcessEnvFunc(0xc000372000, 0xe55740, 0xc008bd57a0, 0xc0011ef3b0, 0x0, 0x0)
	/home/willem/code/golang/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/cache/view.go:318 +0x5e7
golang.org/x/tools/internal/lsp/source.AllImportsFixes(0xe55740, 0xc008bd57a0, 0xe656e0, 0xc015350060, 0xe537c0, 0xc015350000, 0x0, 0x13661f0, 0x1, 0x1, ...)
	/home/willem/code/golang/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/source/format.go:90 +0x684
golang.org/x/tools/internal/lsp.(*Server).codeAction(0xc000222fc0, 0xe55740, 0xc00446a2a0, 0xc003abf700, 0xc003abf700, 0x0, 0x0, 0x0, 0xc000120580)
	/home/willem/code/golang/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/code_action.go:74 +0x95a
golang.org/x/tools/internal/lsp.(*Server).CodeAction(0xc000222fc0, 0xe55740, 0xc00446a2a0, 0xc003abf700, 0xc003abf700, 0x0, 0x0, 0x40e26b, 0x40e438)
	/home/willem/code/golang/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/server_gen.go:12 +0x4d
golang.org/x/tools/internal/lsp/protocol.serverHandler.Deliver(0xe73460, 0xc000222fc0, 0xe55740, 0xc00446a2a0, 0xc001d50340, 0xc00446a200, 0xc0012da960)
	/home/willem/code/golang/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/protocol/tsserver.go:433 +0x25ff
golang.org/x/tools/internal/jsonrpc2.(*Conn).Run.func1(0xc00a984360, 0xc001d50340, 0xc000284480, 0xe55740, 0xc00446a2a0, 0x0, 0x0, 0xc001c41cd0)
	/home/willem/code/golang/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/jsonrpc2/jsonrpc2.go:370 +0x170
created by golang.org/x/tools/internal/jsonrpc2.(*Conn).Run
	/home/willem/code/golang/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/jsonrpc2/jsonrpc2.go:354 +0x877
[Error - 4:01:04 PM] Request textDocument/codeAction failed.
  Message: method "textDocument/codeAction" did not reply
  Code: -32603 
[Info  - 4:01:04 PM] Connection to server got closed. Server will restart.
[Error - 4:01:04 PM] Request textDocument/documentSymbol failed.

Build info

golang.org/x/tools/gopls v0.3.1
    golang.org/x/tools/gopls@v0.3.1 h1:yNTWrf4gc4Or0UecjOas5pzOa3BL0WDDyKDV4Wz5VaM=
    github.com/BurntSushi/toml@v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/go-diff@v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
    golang.org/x/mod@v0.1.1-0.20191105210325-c90efee705ee h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
    golang.org/x/sync@v0.0.0-20190423024810-112230192c58 h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
    golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7 h1:hWZVyLW37WdETuLIGQMvQIhMfXXAOANmAIEAluZMy3c=
    golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
    honnef.co/go/tools@v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM=
    mvdan.cc/xurls/v2@v2.1.0 h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=

Go info

go version go1.13.7 linux/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/willem/.cache/go-build"
GOENV="/home/willem/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/willem/code/golang"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/willem/go/project/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build407652727=/tmp/go-build -gno-record-gcc-switches"

[gopherbot]

Thank you for filing a gopls issue! Please take a look at the Troubleshooting guide, and make sure that you have provided all of the relevant information here.

[heschi]

Very strange. I could fix the crash very easily, but you'd be left with broken autocomplete. The problem is that the goimports code used for autocomplete thinks you're in GOPATH mode. I'm not sure why.

A couple questions:
Can you show us your settings? I'm particularly interested in any changes to the environment, working dir, etc.
If you set "gopls": {"verboseOutput":true} in your settings, the first time you type in a Go file there will be a couple log lines like:

[Info  - 5:29:33 PM] 2020/02/07 17:29:33 11.878056ms for GOROOT=... GOPATH=... GO111MODULE= GOPROXY=https://proxy.golang.org,direct PWD=... go [go env GOMOD]

Can you show us those too?
Which directory do you start VS Code in?

[gwillem]

Thanks for quick followup!

[Info  - 11:19:34 AM] 2020/02/08 11:19:34 1.024911674s for GOROOT=/opt/go GOPATH=/home/willem/code/golang GO111MODULE= PWD=/home/willem/go/project go "list" "-ldflags" "-w -s -extldflag='-static'" "-tags" "netgo yara_static" "-e" "-json" "-compiled=true" "-test=true" "-export=false" "-deps=true" "-find=false" "-ldflags" "-w -s -extldflag='-static'" "-tags" "netgo yara_static" "--" "./..." "builtin", stderr: <<

GOPATH is set in my .bashrc, remnant from non-module projects. When I unset GOPATH before launching code (from my project dir), gopls still crashes but only after a number of files seem to have been inspected successfully:

[Info  - 11:31:44 AM] 2020/02/08 11:31:44 7.241208ms for GOROOT=/opt/go GOPATH=/home/willem/go GO111MODULE= PWD=/home/willem/go/project go "env" "-json" "GOMOD" "GOPATH", stderr: <<>> stdout: <<{
	"GOMOD": "/home/willem/go/project/go.mod",
	"GOPATH": "/home/willem/go"
}
[...]
[Info  - 11:31:44 AM] 2020/02/08 11:31:44 go/packages.Load
	snapshot = 0
	package = github.com/gwillem/project/log
	files = [/home/willem/go/project/log/log.go]
[Info  - 11:31:44 AM] 2020/02/08 11:31:44 go/packages.Load
	snapshot = 0
	package = github.com/gwillem/project/webclient
	files = [/home/willem/go/project/webclient/client.go]
[...]
[Error - 11:31:46 AM] Request textDocument/codeAction failed.
  Message: method "textDocument/codeAction" did not reply
  Code: -32603 
panic: interface conversion: imports.Resolver is *imports.gopathResolver, not *imports.ModuleResolver

goroutine 6 [running]:
golang.org/x/tools/internal/lsp/cache.(*view).RunProcessEnvFunc(0xc0001b4a00, 0xe559e0, 0xc00871b4a0, 0xc00e495a40, 0x0, 0x0)
	/home/willem/go/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/cache/view.go:318 +0x5e7
golang.org/x/tools/internal/lsp/source.AllImportsFixes(0xe559e0, 0xc00871b4a0, 0xe65980, 0xc0009c0240, 0xe53a60, 0xc0001a01e0, 0x0, 0x13661f0, 0x1, 0x1, ...)
	/home/willem/go/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/source/format.go:90 +0x684
golang.org/x/tools/internal/lsp.(*Server).codeAction(0xc000274a80, 0xe559e0, 0xc0002f7740, 0xc000cdec80, 0xc000cdec80, 0x0, 0x0, 0x0, 0xc0001c9ad0)
	/home/willem/go/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/code_action.go:74 +0x95a
golang.org/x/tools/internal/lsp.(*Server).CodeAction(0xc000274a80, 0xe559e0, 0xc0002f7740, 0xc000cdec80, 0xc000cdec80, 0x0, 0x0, 0x0, 0x0)
	/home/willem/go/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/server_gen.go:12 +0x4d
golang.org/x/tools/internal/lsp/protocol.serverHandler.Deliver(0xe73700, 0xc000274a80, 0xe559e0, 0xc0002f7740, 0xc000084ec0, 0xc0002f7700, 0xc00031a140)
	/home/willem/go/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/lsp/protocol/tsserver.go:433 +0x25ff
golang.org/x/tools/internal/jsonrpc2.(*Conn).Run.func1(0xc000042240, 0xc000084ec0, 0xc0002b62a0, 0xe559e0, 0xc0002f7740, 0x0, 0x0, 0xc0000733b0)
	/home/willem/go/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/jsonrpc2/jsonrpc2.go:370 +0x170
created by golang.org/x/tools/internal/jsonrpc2.(*Conn).Run
	/home/willem/go/pkg/mod/golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7/internal/jsonrpc2/jsonrpc2.go:354 +0x877
[Error - 11:31:46 AM] Connection to server got closed. Server will not be restarted.

[heschi]

Still very strange.

There should be more than one call to go env. The one that matters is this one: https://github.com/golang/tools/blob/61798d64f0258df3b7b6e3667997eaebec9c0d99/internal/imports/fix.go#L793 which only reads GOMOD and doesn't pass -json. Something strange is happening with it that's not happening with the others.

Now that I'm looking at it, swallowing the error is a bad idea. I'll look at fixing that next week. If you'd like, figuring out what's going wrong with it will get this resolved more quickly.

[gwillem]

I see no non-json go env calls in the log.

Am happy to dig further. Any ideas how I should replicate this panic outside of vscode?
Are you sure it's with the imports functionality? This doesn't produce any panics:

find . -name '*.go' | while read f; do echo $f; gopls -v imports -d $f; done 

[gwillem]

Got it. I replaced gopls with a wrapper to debug:

strace -v -f -o /tmp/gopls.log -s 1024 -e trace=write,execve,chdir gopls.bin $*

Turns out, gopls first runs go env -json GOMOD GOPATH (correct output) and then go env GOMOD with GOFLAGS="-ldflags -w -s -extldflag='-static' -tags netgo yara_static":

go env -w: arguments must be KEY=VALUE: invalid argument: GOMOD

It fails because it is not correctly quoted. For reference, this works fine:

GOFLAGS="-ldflags='-w -s -extldflag=\"-static\"' -tags='netgo yara_static'" go env GOMOD

In my vscode settings, I have:

    "gopls": {
        "verboseOutput": true,
        "buildFlags": [
            "-ldflags", "-w -s -extldflag='-static'",
            "-tags", "netgo yara_static",
        ],
        "env": {
            "PKG_CONFIG_PATH": "/opt/yara-3.8.1/libyara/yara.pc",
            "CGO_LDFLAGS": "-L/opt/yara-3.8.1/libyara/.libs -lyara -lm",
        }
    },

I don't know why it worked with v0.2.2, perhaps previously GOFLAGS wasn't passed to go env ?

Observations:

• vscode did not quote build flags from the settings as I expected
• gopls did not croak on an unexpected go env GOMOD result
• gopls verbose output did not report the actual value of the GOFLAGS env var (see my initial report above).

My problem is now solved, because I have eliminated the GOFLAGS entirely from my vscode setup.

Thanks @heschik for your help in resolving this!

[heschi]

Thanks, that makes sense. Glad it's working for you for the moment, I'll look into where the quoting problem is and fix it when I have a chance.

[gopherbot]

Change https://golang.org/cl/218857 mentions this issue: internal/imports: change processEnv to use buildflags