cmd/go: go mod tidy randomly removes indirect dependency

[tobiaszheller]

What version of Go are you using (go version)?

$ go version
go version go1.19.5 linux/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_arm64"
GOVCS=""
GOVERSION="go1.19.5"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/go/src/teleport/teleport/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2007327935=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Running go mod tidy on gravitational/teleport@ce86ad5 (not only this commit, other on master as well)

What did you expect to see?

No changes after running go mod tidy.

What did you see instead?

$ go mod tidy -v
unused github.com/coreos/pkg

It happens on my machine both on mac and inside docker, it also happens on dependabot running on GHA. However for at least 3 other people running the same version of Go (they checked on darwin/amd64), go mod tidy returns no changes.

Running go mod graph returns following modules for given indirect package:

$ go mod graph | grep coreos/pkg
github.com/spf13/viper@v1.4.0 github.com/coreos/pkg@v0.0.0-20180928190104-399ea9e2e55f
go.etcd.io/etcd@v0.0.0-20191023171146-3cf2f69b5738 github.com/coreos/pkg@v0.0.0-20160727233714-3ac0863d7acf

[seankhliao]

Do the people with no change add it back if it's removed?

[tobiaszheller]

yes, running go mod tidy on "valid" machines bring it back.

[seankhliao]

This looks like module graph pruning https://go.dev/ref/mod#graph-pruning, the github.com/coreos/pkg looks to only be required for tests far out in the dependency graph.

Can you find a way to reliably reproduce keeping the entry?
You say running in docker will remove it.

cc @bcmills @matloob

[tobiaszheller]

@seankhliao I have just pushed new branch (gravitational/teleport@630226e) where I use newest package of our fork go-oidc. https://github.com/gravitational/go-oidc/blob/master/go.mod as you can see here github.com/coreos/pkg is linked there are direct dependency.

And it is still removed on my machine/docker.

[bcmills]

In a fresh checkout of gravitational/teleport@630226e, go mod why -m github.com/coreos/pkg reports that the dependency is unused.

For the folks who are not seeing the dependency removed, what do the following commands report?

• go mod why -m github.com/coreos/pkg
• go env GOMOD GOWORK GOVERSION
• git status --short --untracked=all | grep '\.go'

[tobiaszheller]

It turned out that given module was used in enterprise package which is checkout as git submodule. We were able to solve the issue, thanks for help.