cmd/dist: no way to disable buildvcs stamping during bootstrap

[williamh]

Hi,

I am the packager for Go on Gentoo linux.

We have an on-going issue with the VCS stamping which has appeared in another way.

Since go 1.17 and older are not in our tree any longer, we are able to add "-buildvcs=false" to our default build flags for go-based packages to prevent vcs stamping from breaking our package builds.

However, if go is being bootstrapped and the bootstrap binaries are newer than version 1.18, it is possible for vcs stamping to crash the build as shown below.

What version of Go are you using (go version)?

$ go version
go version go1.18.5 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/william/.cache/go-build"
GOENV="/home/william/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/william/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/william/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.5"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="x86_64-pc-linux-gnu-gcc"
CXX="x86_64-pc-linux-gnu-g++"
CGO_ENABLED="1"
GOMOD="/home/william/build/go/src/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build162734369=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ cd
$ mkdir build
$ cd build
$ git init
$ chmod 000 .git
$ tar -xf /path/to/go1.18.5.tar.gz
$ cd go/src
$ export GOROOT_BOOTSTRAP=/usr/lib/go
$ ./all.bash

What did you expect to see?

Successful build.

What did you see instead?

Building Go cmd/dist using /usr/lib/go. (go1.18.5 linux/amd64)
Building Go toolchain1 using /usr/lib/go.
error obtaining VCS status: exit status 128
Use -buildvcs=false to disable VCS stamping.
error obtaining VCS status: exit status 128
Use -buildvcs=false to disable VCS stamping.
error obtaining VCS status: exit status 128
Use -buildvcs=false to disable VCS stamping.
error obtaining VCS status: exit status 128
Use -buildvcs=false to disable VCS stamping.
go tool dist: FAILED: /usr/lib/go/bin/go install -gcflags=-l -tags=math_big_pure_go compiler_bootstrap bootstrap/cmd/...: exit status 1

[williamh]

This was reported by a user here, but I wanted to reproduce it outside of our package manager before I reported it to you.

[seankhliao]

Note that we currently have 1.17 as our minimum bootstrap version.

If you only want to support 1.18+ as bootstrap you could add -buildvcs=false to the flags passed in src/cmd/dist/buildtool.go

[bcmills]

Hmm. As of CL 398855 (backported to Go 1.18.2), the default behavior is to stamp VCS information when the VCS tool is present, and to omit it when the VCS tool is not found.

Moreover, we intentionally avoid stamping VCS information for programs found in the standard library:
https://cs.opensource.google/go/go/+/master:src/cmd/go/internal/load/pkg.go;l=2431;drc=e64c87157d1e8fbc512a670b8c0af8abc3afa7c1

So one option might be to change that condition from !p.Standard to !p.Goroot, but then that would also omit VCS information from binaries in other GOROOT modules (such as GOROOT/misc).

[bcmills]

That said, looking at https://bugs.gentoo.org/836261 I don't understand why this would affect Go bootstrapping.

The issue reported there is that the VCS stamping is somehow coming up with /gentoo/fossil_repo/var.fossil as the VCS metadata to stamp into the binaries, but that should not be possible: cmd/dist/buildtool.go places the bootstrap module in $GOROOT/pkg/bootstrap/src/bootstrap, in which case the presence of the $GOROOT/.git directory should cause that (not /gentoo/fossil_repo/var.fossil) to be treated as the repo root.

So I guess a contributing factor is that $GOROOT/.git is stripped from the repo during Gentoo builds?

[seankhliao]

I believe Gentoo builds using a source tarball rather than a vcs checkout

[williamh]

That's correct, we build using a source tarball.