Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: builds not reproducible #36230

Closed
jakecraige opened this issue Dec 20, 2019 · 2 comments
Closed

cmd/go: builds not reproducible #36230

jakecraige opened this issue Dec 20, 2019 · 2 comments

Comments

@jakecraige
Copy link

@jakecraige jakecraige commented Dec 20, 2019

What version of Go are you using (go version)?

$ go version
go version go1.13.5 darwin/amd64
$ docker run --rm golang:1.13.5 go version
go version go1.13.5 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/username/Library/Caches/go-build"
GOENV="/Users/username/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/username/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.5/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.5/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/nz/pfhjh4nj1zx5czg6r89fxpyh0000gp/T/go-build775226103=/tmp/go-build -gno-record-gcc-switches -fno-common"
docker run --rm golang:1.13.5 go env Output
$ docker run --rm golang:1.13.5 go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build001072247=/tmp/go-build -gno-record-gcc-switches"

What did you do?

/*
* reproducible.go
*
* This file shows some unexpected differences in go binaries when we expect to have byte-for-byte
* deterministic builds. This seems to surface when the same binary is built on two different
* machines. We simulate that in this example by using docker for the second.
*
* 1. Build the first binary normally, explicitly setting options for build reproducibility:
*    `GOOS=darwin GOARCH=amd64 go build -trimpath -ldflags="-buildid=" -o one reproducible.go`
* 2. Build the second in a docker container
*    `docker run --rm -v $(pwd):/app -w /app -e GOOS=darwin -e GOARCH=amd64 golang:1.13.5 go build -trimpath -ldflags="-buildid=" -o two reproducible.go`
* 3. Shasum files for comparison: `shasum one two` to see they don't match. Analyze deper with tools like `vbindiff one two`
 */
package main

import "fmt"

func main() {
	fmt.Println("Hello world")
}

What did you expect to see?

The shasum of binaries one and two should be equal.

What did you see instead?

The shasums do not match

eb264107f7305f70fa4fb22be0c7b7db18fa3e69  one
0b5fd4561dd4a21aad2d91ef28a73cc42feff5c9  two

I know that building on a Mac vs Linux may have inconsistencies due to the environment and if I had an easy way to simulate that I would have for this issue. This is the reduction of seeing the same issue on a more complex project where we did see that the binaries were different in a similar way when one Mac built it in docker vs another Mac that built it in docker. I would expect that if a reviewer of this issue does the docker build they get the same output as me, but when another on my team did (not for this exact example unfortunately) they did not.

@jakecraige

This comment has been minimized.

Copy link
Author

@jakecraige jakecraige commented Dec 20, 2019

I'm going to need to investigate further before opening an issue. Upon further testing the Mac build vs Linux is definitely working differently so this example isn't a good representation of what I have encountered.

@jakecraige jakecraige closed this Dec 20, 2019
@jayconrod

This comment has been minimized.

Copy link
Contributor

@jayconrod jayconrod commented Dec 20, 2019

It looks like CGO_ENABLED isn't explicitly set for either build. It's on by default for darwin/amd64, but it's disabled when cross-compiling on any platform (which would be the case within a Docker container, which is linux/amd64). So that would explain the difference.

I'm not sure if we're 100% reproducible with CGO_ENABLED=0. If not we're close. Please file issues if you find problems.

There are definitely some issues (like #36072) with reproducibility when CGO_ENABLED=1. I hope to track those down next cycle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.