zero out the build id for reproducible builds

[dims]

We have been having issues with making builds reproducible, especially
with the .note.go.buildid ELF section. One tip from a golang issue was
to set -ldflags=-buildid= which seems to work well. You can confirm
that the buildid is set to empty by inspecting the binaries with the go
command example go tool buildid _output/local/go/bin/kubectl

Signed-off-by: Davanum Srinivas davanum@gmail.com

What type of PR is this?
/kind bug

What this PR does / why we need it:
Tip is from here : golang/go#34186 (comment)
Struggles with reproducible builds are documented here : #70131
Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

NONE

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dims

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• hack/OWNERS [dims]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dims]

/priority important-soon
/sig release

cc @bmwiedemann Bernhard, can you please try this?

[dims]

/assign @cblecker @BenTheElder

[cblecker]

Seems reasonable to me.

/cc @justaugustus

[BenTheElder]

is it reproducible after this?
is there a pointer to how this value is normally used?

[BenTheElder]

starting points:

• https://golang.org/cmd/buildid/
• https://blog.filippo.io/reproducing-go-binaries-byte-by-byte/

The Build ID is a hash of the filenames of the compiled files, plus the version of the compiler (and other things in zversion.go, like the default GOROOT). See pkg.go.

Here we can see more about how it is used, in short it appears to be part of how go identifies if a binary is up to date for caching purposes.
https://github.com/golang/go/blob/26154f31ad6c801d8bad5ef58df1e9263c6beec7/src/cmd/go/internal/work/buildid.go#L41

It seems like we may only want to set this to zero when we produce release binaries, otherwise we may be unnecessarily defeating the cache. Some of the binaries we build probably don't need to be reproducible anyhow (code generators? linters?).

[dims]

@BenTheElder if you see the issue, we are chipping away at the reproducibility bit by bit. please see latest runs from @bmwiedemann here:
https://rb.zq1.de/compare.factory-20200303/kubernetes-compare.out

There's a few more things to take care of after that:

$ rg 'differs in' kubernetes-compare.out | grep -v buildid
/usr/bin/kube-apiserver differs in ELF section .typelink
/usr/bin/kube-controller-manager differs in ELF section .typelink
/usr/src/kubernetes/hack/../platforms/linux/amd64/e2e.test differs in ELF section .gopclntab
/usr/src/kubernetes/platforms/linux/amd64/e2e.test differs in ELF section .gopclntab
/usr/bin/kube-apiserver differs in ELF section .typelink
/usr/bin/kube-controller-manager differs in ELF section .typelink

Also note that -s -w strips out debugging information which we added a few releases ago!

[dims]

/retest

[BenTheElder]

cc @Katharine FYI WRT caching
I'm OK to do this, but I suspect stripping the cache key may bite us later.

I would suggest that reproducible builds be performed in a container (e.g. like our official releases...) which should ensure fixed paths. KIND has had reproducible release artifacts for some time by simply doing this...

[dims]

@Katharine will you have some time to dig into this? i'd like to get this in early to see if this breaks something ..

[BenTheElder]

/lgtm