cmd/go: executables built with `-trimpath` are not reproducible with development Go toolchains unless `-ldflags` removes `-buildid=`

[bcmills]

What version of Go are you using (go version)?

$ go version
go version devel go1.21-63a08e61bd Mon Apr 10 17:13:41 2023 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bryan/.cache/go-build"
GOENV="/home/bryan/.config/go/env"
GOEXE=".exe"
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/bryan/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="windows"
GOPATH="/home/bryan"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/bryan/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/bryan/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.21-63a08e61bd Mon Apr 10 17:13:41 2023 +0000"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="c++"
CGO_ENABLED="0"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-m64 -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build840843456=/tmp/go-build -gno-record-gcc-switches"

What did you do?

GOOS=windows CGO_ENABLED=0 go install -trimpath cmd/test2json using a Windows build of the Go toolchain and a Linux build of the Go toolchain checked out at the same commit.

What did you expect to see?

Since the two Go toolchains are built from the same source code, and they are both targeting the same platform, and neither should be using an external compiler or linker, the resulting binaries should be identical.

What did you see instead?

The binaries are only identical if I also pass -ldflags=all=-buildid=.

Adding the -x flag to the go install command confirms that cmd/go is passing different -buildid flags in the two configurations.

[rsc]

The build IDs in unversioned copies of Go include a hash of the compiler binary to incorporate the semantics of the compiler somehow (the version string is assumed good enough in a versioned copy, although that's only true if you don't modify the files in your versioned checkout), so this is working as expected.

[jrick]

We saw this behavior with 1.20.3 as well, not running tip.

[jrick]

Er, would that version string include the host operating system too?

[bcmills]

Shouldn't, no. But maybe on 1.20.3 the build-ID depends on the GOAMD setting (which is the thing already fixed for 1.21). Maybe try checking out the development tree at HEAD, give it a VERSION file like go1.21.999999 (to avoid polluting the build cache if you forget to clean it), and see if that's enough?

[jrick]

Sure, let me find a working commit first :)

>$ (cd src && bash all.bash)
Building Go cmd/dist using /usr/lib64/go/1.20. (go1.20.3 linux/amd64)
Building Go toolchain1 using /usr/lib64/go/1.20.
# bootstrap/internal/platform
/home/jrick/src/go/src/internal/platform/zcgo.go:8: undefined: osArchSupportsCgo
/home/jrick/src/go/src/internal/platform/zcgo.go:12: undefined: osArchSupportsCgo
go tool dist: FAILED: /usr/lib64/go/1.20/bin/go install -tags=math_big_pure_go compiler_bootstrap purego bootstrap/cmd/...: exit status 1

[rsc]

For Go 1.20 you have the default GOARM discrepancy on Linux vs Windows, so I'm not sure how you'd have gotten far enough that -ldflags=-buildid= would help.

[bcmills]

@jrick, you'll want to git clean away the files from my pending CL that you patched in to test earlier. 😅