Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: working directory affects binaries even with -trimpath #33772

Closed
dsnet opened this issue Aug 21, 2019 · 9 comments

Comments

@dsnet
Copy link
Member

commented Aug 21, 2019

Using go1.13rc1

After #16860 was closed, I was hoping that I could build byte-for-byte identical builds regardless of what machine was used to build the binaries. At the present moment, building binaries are nearly identical regardless of the machine used, except for the following bytes:

$ diff <(hexdump -Cv linux1/protoc-gen-go.v1.19.0-devel.linux.amd64) <(hexdump -Cv linux2/protoc-gen-go.v1.19.0-devel.linux.amd64)
251,254c251,254
< 00000fa0  53 00 00 00 04 00 00 00  47 6f 00 00 76 72 6e 46  |S.......Go..vrnF|
< 00000fb0  43 55 48 32 42 74 57 4e  53 4c 6a 62 54 78 47 35  |CUH2BtWNSLjbTxG5|
< 00000fc0  2f 54 49 30 2d 6d 6e 55  4f 45 49 52 5f 41 48 6c  |/TI0-mnUOEIR_AHl|
< 00000fd0  36 57 39 66 65 2f 6b 62  30 43 44 67 66 72 38 64  |6W9fe/kb0CDgfr8d|
---
> 00000fa0  53 00 00 00 04 00 00 00  47 6f 00 00 61 69 6a 4b  |S.......Go..aijK|
> 00000fb0  48 6b 71 33 2d 41 36 37  64 6a 63 77 50 4b 48 79  |Hkq3-A67djcwPKHy|
> 00000fc0  2f 65 66 2d 78 37 5f 34  72 4c 4d 5f 73 68 36 73  |/ef-x7_4rLM_sh6s|
> 00000fd0  79 56 75 47 31 2f 6b 62  30 43 44 67 66 72 38 64  |yVuG1/kb0CDgfr8d|

There seems to be some type of build ID encoded into the binary.

Steps to reproduce:

$ git clone https://go.googlesource.com/protobuf
$ cd protobuf
$ git checkout 08ff73004835d46c133844f593fe74596d3e8194
$ go test -mod=vendor -timeout=60m -count=1 integration_test.go -buildRelease
$ md5sum bin/protoc-gen-go.v1.19.0-devel.linux.amd64

Unfortunately, that command will take a while to run since it needs to build the C++ protoc binary as a dependency.

This need not block go1.13 release.

@dsnet dsnet changed the title cmd/compile: reproducible builds cmd/compile: reproducible builds from different host machines Aug 21, 2019

@dsnet

This comment has been minimized.

Copy link
Member Author

commented Aug 22, 2019

Here's a shorter repro without needing to build protoc:

$ go get golang.org/dl/go1.13rc1
$ go1.13rc1 download
$ git clone https://go.googlesource.com/protobuf
$ cd protobuf
$ git checkout 08ff73004835d46c133844f593fe74596d3e8194
$ GOOS=linux GOARCH=amd64 go1.13rc1 build -trimpath -ldflags "-s -w" -o protoc-gen-go ./cmd/protoc-gen-go

Running that above on different machines produces different binaries, even if both machines are the same host OS.

@dsnet

This comment has been minimized.

Copy link
Member Author

commented Aug 22, 2019

It seems that the current working directory has some affect on the build ID embedded in the binary:

/tmp/protobuf $ GOOS=linux GOARCH=amd64 go1.13rc1 build -trimpath -ldflags "-s -w" -o protoc-gen-go ./cmd/protoc-gen-go && md5sum protoc-gen-go && rm protoc-gen-go
04b0ff070cc56ec8ed899cf986a97f50  protoc-gen-go
/tmp/protobuf $ GOOS=linux GOARCH=amd64 go1.13rc1 build -trimpath -ldflags "-s -w" -o protoc-gen-go ./cmd/protoc-gen-go && md5sum protoc-gen-go && rm protoc-gen-go
04b0ff070cc56ec8ed899cf986a97f50  protoc-gen-go
/tmp/protobuf $ cd .. && mv protobuf protobuf2 && cd protobuf2
/tmp/protobuf2 $ GOOS=linux GOARCH=amd64 go1.13rc1 build -trimpath -ldflags "-s -w" -o protoc-gen-go ./cmd/protoc-gen-go && md5sum protoc-gen-go && rm protoc-gen-go
47091a2082a8ef696c0b76c1ab983c62  protoc-gen-go
/tmp/protobuf2 $ GOOS=linux GOARCH=amd64 go1.13rc1 build -trimpath -ldflags "-s -w" -o protoc-gen-go ./cmd/protoc-gen-go && md5sum protoc-gen-go && rm protoc-gen-go
47091a2082a8ef696c0b76c1ab983c62  protoc-gen-go

@bcmills bcmills added this to the Go1.14 milestone Aug 22, 2019

@bcmills bcmills changed the title cmd/compile: reproducible builds from different host machines cmd/go: reproducible builds from different host machines Aug 22, 2019

@bcmills bcmills changed the title cmd/go: reproducible builds from different host machines cmd/go: current working directory affects build Aug 22, 2019

@bcmills bcmills changed the title cmd/go: current working directory affects build cmd/go: working directory affects build ID even with -trimpath Aug 22, 2019

@bcmills bcmills changed the title cmd/go: working directory affects build ID even with -trimpath cmd/go: working directory affects binaries even with -trimpath Aug 22, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented Aug 22, 2019

@jrick jrick referenced this issue Sep 5, 2019
@jrick

This comment has been minimized.

Copy link

commented Sep 5, 2019

This difference is due to the .note.go.buildid section added by the linker. It can be set to something static e.g. -ldflags=-buildid= (empty string) to gain reproducibility.

Thanks to @neutralinsomniac for working with me to find this.

gopherbot pushed a commit to protocolbuffers/protobuf-go that referenced this issue Sep 5, 2019
release.bash: support reproducible builds
Manually specifying the "-buildid" removes the last source of variability
when the release binaries is built on one system or another.

See golang/go#33772 (comment)

Change-Id: Ibc9b8e99b5b76719735942a510570bcbf96dfea6
Reviewed-on: https://go-review.googlesource.com/c/protobuf/+/193498
Reviewed-by: Damien Neil <dneil@google.com>
@hugelgupf

This comment has been minimized.

Copy link
Contributor

commented Sep 10, 2019

This difference is due to the .note.go.buildid section added by the linker. It can be set to something static e.g. -ldflags=-buildid= (empty string) to gain reproducibility.

Thanks to @neutralinsomniac for working with me to find this.

I just hit the same issue -- is there somewhere this can be documented better, and/or maybe given a -reproducible flag that == "-trimpath -ldflags=-buildid="?

@hugelgupf

This comment has been minimized.

Copy link
Contributor

commented Sep 10, 2019

cc @rsc who fixed #16860

@gopherbot

This comment has been minimized.

Copy link

commented Sep 13, 2019

Change https://golang.org/cl/195318 mentions this issue: cmd/go: don't include package dir in cache key when -trimpath is set

@gopherbot gopherbot closed this in aa680c0 Sep 16, 2019

@FiloSottile

This comment has been minimized.

Copy link
Member

commented Sep 16, 2019

I'd like this to be considered for backport to Go 1.13 (@gopherbot, please do the honors).

There is no real workaround to get the functionality of -trimpath without this fix, which AFAICT was added specifically for reproducible builds.

@gopherbot

This comment has been minimized.

Copy link

commented Sep 16, 2019

Backport issue(s) opened: #34326 (for 1.13).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.