x/tools/cmd/gopls: nil pointer dereference when creating new file

[myitcv]

What version of Go are you using (go version)?

$ go version
go version devel +c3c53661ba Tue Sep 17 04:37:46 2019 +0000 linux/amd64
$ go list -m golang.org/x/tools
golang.org/x/tools v0.0.0-20190917215024-905c8ffbfa41
$ go list -m golang.org/x/tools/gopls
golang.org/x/tools/gopls v0.0.0-20190917215024-905c8ffbfa41

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/myitcv/.cache/go-build"
GOENV="/home/myitcv/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/myitcv/gostuff"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/myitcv/gos"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/myitcv/gos/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/myitcv/gostuff/src/github.com/myitcv/govim/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build291981782=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Discovered by govim cron tests against gopls and x/tools master.

Creating a new file in a package appears to result in one of two panics due to nil pointer dereferences:

Stack trace 1

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xc57dc5]

goroutine 72 [running]:
golang.org/x/tools/internal/lsp/cache.(*importer).cachePerFile(0xc0004341e0, 0xf547e0, 0xc0004221e0, 0xc0000ef290, 0xf54a20, 0xc000419fe0, 0xc00041c440, 0x0, 0x0)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/cache/check.go:354 +0x205
golang.org/x/tools/internal/lsp/cache.(*importer).cachePackage(0xc0004341e0, 0xf547e0, 0xc0004221e0, 0xc00041c440, 0xc00041c440, 0x0)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/cache/check.go:333 +0x1d6
golang.org/x/tools/internal/lsp/cache.(*view).loadParseTypecheck(0xc0002cc000, 0xf547e0, 0xc0004221e0, 0xc0000ef290, 0xf549e0, 0xc0003b9360, 0x0, 0x0)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/cache/load.go:42 +0x620
golang.org/x/tools/internal/lsp/cache.(*goFile).CheckPackageHandles(0xc0000ef290, 0xf547e0, 0xc0004220f0, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/cache/gofile.go:45 +0x665
golang.org/x/tools/internal/lsp/source.Diagnostics(0xf547e0, 0xc0004220f0, 0xf60000, 0xc0002cc000, 0xf57160, 0xc0000ef290, 0x0, 0x0, 0x0, 0x0, ...)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/source/diagnostics.go:71 +0x201
golang.org/x/tools/internal/lsp.(*Server).diagnostics(0xc0001f1aa0, 0xf60000, 0xc0002cc000, 0xc00026be50, 0x4a, 0x0, 0x0)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/diagnostics.go:36 +0x345
created by golang.org/x/tools/internal/lsp.(*Server).didOpen
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/text_synchronization.go:34 +0x28d

Stack trace 2

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x7a2e2f]

goroutine 30 [running]:
go/ast.(*File).Pos(...)
	/home/myitcv/gos/src/go/ast/ast.go:990
golang.org/x/tools/internal/lsp/source.IsGenerated(0xbf0ca0, 0xc0003781c0, 0xbfbce0, 0xc0002d6000, 0xc00042a1e0, 0x45, 0x2)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/source/util.go:99 +0x1ef
golang.org/x/tools/internal/lsp.(*Server).didChange(0xc000230120, 0xbf0ca0, 0xc0003781c0, 0xc000210750, 0x0, 0xc00029a160)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/text_synchronization.go:74 +0x2f7
golang.org/x/tools/internal/lsp.(*Server).DidChange(0xc000230120, 0xbf0ca0, 0xc0003781c0, 0xc000210750, 0xc000210750, 0x0)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/server.go:138 +0x49
golang.org/x/tools/internal/lsp/protocol.serverHandler.Deliver(0xc02a80, 0xc000230120, 0xbf0ca0, 0xc0003781c0, 0xc000378280, 0xc0004ce700, 0xc000300001)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/lsp/protocol/tsserver.go:123 +0x2780
golang.org/x/tools/internal/jsonrpc2.(*Conn).Run.func1(0xc0001a8660, 0xc000378280, 0xc000230180, 0xbf0ca0, 0xc0003781c0, 0x0, 0x0, 0xc00006e1a0)
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/jsonrpc2/jsonrpc2.go:370 +0x170
created by golang.org/x/tools/internal/jsonrpc2.(*Conn).Run
	/home/myitcv/gostuff/pkg/mod/golang.org/x/tools@v0.0.0-20190917215024-905c8ffbfa41/internal/jsonrpc2/jsonrpc2.go:354 +0x867

What did you expect to see?

No panic.

What did you see instead?

Per above.

---

cc @stamblerre @ianthehat @muirrn

[myitcv]

In the govim tests, we only see stack trace 1 when running with -race (for both the tests and gopls). We have not yet seen stack trace 1 in the govim tests.

Stack trace 2 can be triggered by hand when creating a new file in a package and then editing the file. On the first change to the new file the panic is triggered.

[gopherbot]

Change https://golang.org/cl/196142 mentions this issue: internal/lsp: ensure that an AST cannot be nil without an error