cmd/go: value of -buildmode in buildinfo can differ from what is used in build ID

[capnspacehook]

What version of Go are you using (go version)?

$ go version
go version go1.21.3 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/capnspacehook/.cache/go-build'
GOENV='/home/capnspacehook/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/capnspacehook/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/capnspacehook/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.1'
GCCGO='gccgo'
GOAMD64='v3'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD=''
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3246179384=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Built two binaries from identical source code, passing -buildmode=default to the first and -buildmode=exe to the second.

What did you expect to see?

Both binaries should be identical and have the same hash.

What did you see instead?

The binaries hashes differed, though using xbindiff showed that the build ID was the only difference between them. It seems that the buildmode that is actually used is included in binaries build information (exe in both cases), but the exact value of -buildmode as passed to go build is used when creating the build ID. To make reproducing binaries easier I think the same buildmode value that is used in the embedded build information should be used when creating the build ID.

Currently it can be very difficult to tell what value of -buildmode was passed at build time when you only have a Go binary and no information on its build process. This change would make it easier to reproduce Go binaries using the embedded build information as a guide.

[bcmills]

The line that puts this data in the cache ID is here:
https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/work/exec.go;l=1348;drc=ee8b8e988c47a400d035891539a045031854b391

And the logic for the embedded buildinfo is here:
https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/load/pkg.go;l=2371-2379;drc=5873bd1d7e87e0686390d71a840a44e2e2c3e11d

That said, the embedded logic seems off too: on some platforms, default is pie rather than exe:
https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/work/init.go;l=231-238;drc=5873bd1d7e87e0686390d71a840a44e2e2c3e11d

And the logic for whether to set codegenArg to -fPIE or -shared or leave it unset is different between the pie and default cases — that's probably a bug?

Either way, this needs a careful test for platforms that default to PIE executables.

[quantonganh]

It seems that the buildmode that is actually used is included in binaries build information (exe in both cases)

If a binary is built using -buildmode=default:

$ go build -buildmode=default -o main.default

Should we record the -buildmode in the build info as it is instead of exe?

$ go version -m main.default
main.default: devel go1.22-777540cd23 Tue Nov 14 08:20:14 2023 +0700
        path    m
        mod     m       (devel)
        build   -buildmode=exe
        build   -compiler=gc
        build   CGO_ENABLED=1
        build   CGO_CFLAGS=
        build   CGO_CPPFLAGS=
        build   CGO_CXXFLAGS=
        build   CGO_LDFLAGS=
        build   GOARCH=arm64
        build   GOOS=darwin

[quantonganh]

When -buildmode=default is used on platforms that default to PIE, codegenArg will be set to -shared:

go/src/cmd/go/internal/work/init.go

Line 236 in c7ea996

codegenArg = "-shared"

This change causes forcedGcflags to be altered:

go/src/cmd/go/internal/work/init.go

Line 201 in 5993251

forcedGcflags = append([]string{codegenArg}, forcedGcflags...)

subsequently affecting the build id:

go/src/cmd/go/internal/work/exec.go

Line 212 in 5993251

fmt.Fprintf(h, "compile %s %q %q\n", b.toolID("compile"), forcedGcflags, p.Internal.Gcflags)

$ go tool buildid main.default
sXYlCmiozkAhsK7gKdc1/gNrES6_ME4j4MT5LqwVJ/TJYC4wzic3MbCcIJp9vw/OXE-kHIiAcvetgY7Ag7H

$ go tool buildid main.exe
i2tRUqE42w9e3W5Tm5Oi/oLoaliHU9uVRiFRcb4kx/rcVyJqY-QPyHlugRPly_/Q1mjqnn3Ybz1xs_Efesg

I'm wondering if this behavior aligns with the expected functionality.