x/tools/gopls: broken build with x/vuln@latest

[bcmills]

What version of Go are you using (go version)?

~/src/golang.org/x/tools/gopls$ go1.20.3 version
go version go1.20.3 linux/amd64

Does this issue reproduce with the latest release?

Yes; it also reproduces with CL 471595 patched in.

What operating system and processor architecture are you using (go env)?

go env Output

$ go1.20.3 env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/usr/local/google/home/bcmills/.cache/go-build"
GOENV="/usr/local/google/home/bcmills/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/tmp/tmp.zk603vMqhA"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/usr/local/google/home/bcmills"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/google/home/bcmills/sdk/go1.20.3"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/google/home/bcmills/sdk/go1.20.3/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.3"
GCCGO="/usr/bin/gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="c++"
CGO_ENABLED="1"
GOMOD="/usr/local/google/home/bcmills/src/golang.org/x/tools/gopls/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build577603139=/tmp/go-build -gno-record-gcc-switches"

What did you do?

~/src/golang.org/x/tools/gopls$ go1.20.3 get -u -t ./...

What did you expect to see?

Dependencies of gopls successfully updated to their latest versions.

What did you see instead?

~/src/golang.org/x/tools/gopls$ go1.20.3 get -u -t ./...
golang.org/x/tools/gopls/internal/govulncheck imports
        golang.org/x/vuln/client: cannot find module providing package golang.org/x/vuln/client
golang.org/x/tools/gopls/internal/govulncheck imports
        golang.org/x/vuln/exp/govulncheck: cannot find module providing package golang.org/x/vuln/exp/govulncheck
golang.org/x/tools/gopls/internal/govulncheck imports
        golang.org/x/vuln/osv: cannot find module providing package golang.org/x/vuln/osv
golang.org/x/tools/gopls/internal/vulncheck imports
        golang.org/x/vuln/vulncheck: cannot find module providing package golang.org/x/vuln/vulncheck

The missing packages were moved in CL 475015.

(attn @hyangah @julieqiu; CC @findleyr @adonovan)

[hyangah]

@bcmills can you tell us more why is labelled "soon"?

The x/vuln API is not stable at all and our intention was not to update x/vuln dependency until a new API becomes available and we migrate to the new one.

[findleyr]

If this is WAI, I propose that we close this issue.

[bcmills]

The x/vuln API is not stable at all and our intention was not to update x/vuln dependency until a new API becomes available and we migrate to the new one.

Even unstable dependencies have points of stability. x/vuln has exactly one tagged version (v0.1.0), and gopls can't build against it.

That means that if critical bugs or vulnerabilities are found and patched in x/vuln, gopls will presumably be unable to upgrade (or will need its own backports), and that if any of the other dependencies of gopls happens to upgrade to the tagged x/vuln those dependencies will become stuck too.

If x/vuln is not usable at its only tagged version, perhaps we should back out the integration and remove the dependency until the API is more stable.

[findleyr]

If x/vuln is not usable at its only tagged version, perhaps we should back out the integration and remove the dependency until the API is more stable.

I really don't understand this perspective, given that we own both projects. They are versioned together, and any critical bugfix would need to be fixed in both x/vuln and gopls. We have this responsibility. Bryan, is your point that they have diverged too much for this to be feasible in a timely manner? In that case, I would defer to @hyangah and @julieqiu to decide.

[bcmills]

Bryan, is your point that they have diverged too much for this to be feasible in a timely manner?

Yes — that, and that the divergence adds friction for testing and maintenance in other ways.

For example:

• https://pkg.go.dev/golang.org/x/vuln only lists the tagged version of x/vuln, which does not include the documentation for the packages imported by gopls.
• Being unable to upgrade x/vuln to its latest tagged release means that we can't use go get -u to upgrade gopls dependencies in general, which is the current workaround for x/build: update x/ dependencies in untagged x/ repos when applying tags #56530.
  • That makes it more likely that the other dependencies of gopls will become stale, potentially masking regressions and compatibility issues.
  • And x/vuln would have to become a special case if x/build: update x/ dependencies in untagged x/ repos when applying tags #56530 is addressed.

[hyangah]

x/vuln is currently actively working on API finalization. The API is significantly different and the current mode is an alternative to copying or vendoring the whole repo. This will be addressed in a couple of months, so let's not worry about this.

[hyangah]

This is the milestone for x/vuln API work https://github.com/golang/go/milestone/308