cmd/go: running go1.23.0 mod vendor on go1.22.6 source unexpectedly vendors golang.org/x/crypto/sha3

[dagood]

Go version

go version go1.23.0 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/dagood/.cache/go-build'
GOENV='/home/dagood/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/dagood/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/dagood/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/dagood/sdk/go1.23.0'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/home/dagood/sdk/go1.23.0/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.23.0'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/dagood/.config/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/dagood/git/golang_go/src/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build23445954=/tmp/go-build -gno-record-gcc-switches'

What did you do?

After checking out go1.22.6, in src:

$ go1.22.6 mod vendor
$ git diff
$ go1.23.0 mod vendor
$ git diff

(Used https://pkg.go.dev/golang.org/dl to download both Go versions.)

What did you see happen?

The first git diff has no output. This means go1.22.6's vendor directory is reproducible when I run Go 1.22.6.

The second git diff shows:

diff --git a/src/vendor/modules.txt b/src/vendor/modules.txt
index 9a234e59b1..ce8a7b8441 100644
--- a/src/vendor/modules.txt
+++ b/src/vendor/modules.txt
@@ -7,6 +7,7 @@ golang.org/x/crypto/cryptobyte/asn1
 golang.org/x/crypto/hkdf
 golang.org/x/crypto/internal/alias
 golang.org/x/crypto/internal/poly1305
+golang.org/x/crypto/sha3
 # golang.org/x/net v0.19.1-0.20240412193750-db050b07227e
 ## explicit; go 1.18
 golang.org/x/net/dns/dnsmessage

(The golang.org/x/crypto/sha3 source is also added into vendor/.)

What did you expect to see?

I expected reproducible results both times.

I filed this based on https://groups.google.com/g/golang-nuts/c/BEsmO16g37I, where I mentioned why this is what I expected:

I've been working under the assumption that backward compatibility promises would extend to "go mod vendor". This way, developers using 1.23.x and 1.22.x won't have thrashing changes when working on the same module.

[gabyhelp]

Related Issues and Documentation

• cmd/go: bad error message when using `vendor` directory while specifying too old Go version #67587 (closed)
• Vendor directory indirectly included in module ZIP due to including `vendor/modules.txt` #64440 (closed)
• Module Info (runtime.modinfo) prevents binary comparison #49573 (closed)
• go version -m vs go mod vendor #47935 (closed)
• all.bash build from source failed to import new vendors #56076 (closed)
• cmd/go: 'inconsistent vendoring' error when a user module is located within GOROOT/src #34657 (closed)
• cmd/go: record -mod=vendor in buildinfo #57782
• `go mod verify` should consider the vendor directory instead of only the modules cache #48420 (closed)
• cmd/go: build with vendor fails when go.mod doesn't record dependencies #47823 (closed)
• cmd/go: 'go build' in module mode rebuilds vendored dependencies in GOROOT #27285 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ianlancetaylor]

CC @matloob @samthanawalla

[dagood]

GOROOT='/home/dagood/sdk/go1.23.0'

I tried changing GOROOT to point at the repository directory, and that seems to make go1.23.0 mod vendor reproducible. The golang.org/x/crypto/sha3 seems to be bleeding through from 1.23.0's GOROOT.

I suppose this might be reasonable: go mod treats the imports in the std module as imports of the go1.23.0 standard library because that's what GOROOT is pointing at, so go1.23.0 mod vendor finds those dependencies in that source and vendors them. If this is true, then maybe this is special with std and doesn't affect ordinary Go development.

I'm not sure how this is intended to work, but maybe it's fine if documented in README.vendor?