Release-pipeline: reproducible builds

[ligi]

We should reduce our trust into travis

Ideally we should build on travis and another platform (can also be a local dev machine) and then compare the binaries. Only if the binaries match -> do/sign the release

Reproducible builds might get possible/easier with Go 1.13 - golang/go#16860

[fjl]

To be clear, reproducible builds have been possible for a long time. The only requirement is that the source be checked out to the same directory. The Go issue you linked is discussing an idea to remove even that restriction.

[ligi]

yea - but on CIs you often do not have full control over the paths - hence I was writing possible/easier

[ligi]

might in the process dogfood Cryptoeconomic Release & Version Control

[attila-lendvai]

i have tried to reproduce the released geth binary, but it seems to be a far fetch, at least with a simple invocation of make... first of all, the official release has stuff like BuildID:

$ file geth-linux-amd64-1.10.8-26675454/geth 
geth-linux-amd64-1.10.8-26675454/geth: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, Go BuildID=_g-OFkERgsL-bejNpcxe/QxhuoRg8rKVNlpakG0wi/e55w0R2l99Pnyp9VFmxf/dqi19jYNsojwTnpalGE5, BuildID[sha1]=e2a7372d4afe8ac055a9201233476e79ab64eaee, not stripped

is there any update/summary of the deterministic build situation?

context: i'm working on the Guix packaging of geth, and i was hoping that i could set up something that downloads and authenticates the official release binary, then builds geth locally, and compare the two.

is that feasible at this point?

[attila-lendvai]

i have opened a PR to set the buildid to a fixed (empty) string: #23527.

with that, the resulting binary is reproducible.

EDIT: this^ is wrong, or half-true. follow through to the PR if interested.

[ligi]

Thanks for caring for reproducible builds!

[attila-lendvai]

a proposal for terminology; this may be of interest here:

Reproducibility vs. Replicability: A Brief History of a Confused Terminology
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5778115/

TL;DR:

Repeatability (Same team, same experimental setup): The measurement can be obtained with stated precision by the same team using the same measurement procedure, the same measuring system, under the same operating conditions, in the same location on multiple trials. For computational experiments, this means that a researcher can reliably repeat her own computation.

Replicability (Different team, same experimental setup): The measurement can be obtained with stated precision by a different team using the same measurement procedure, the same measuring system, under the same operating conditions, in the same or a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using the author's own artifacts.

Reproducibility (Different team, different experimental setup): The measurement can be obtained with stated precision by a different team, a different measuring system, in a different location on multiple trials. For computational experiments, this means that an independent group can obtain the same result using artifacts which they develop completely independently.

[attila-lendvai]

on a related note: i have installed Guix as my daily driver, and it is an excellent tool to host replicable builds.

the distro's repo is a git repo, and it's possible to ask for a shell environment by pointing to a specific commit of the Guix git repo. building on top of Guix, a project can have everything specified in its build scripts for replicability, even several years later.

and the icing on the cake is that the binary seed for bootstrapping Guix itself (which is of course reproducible build) is now very tiny (IIUC a couple hundred bytes called HEX0, which fits into a boot sector).

[attila-lendvai]

for inspiration: Feather wallet uses Guix for reproducible build.

[karalabe]

Closing this issue. Unsure where we are on this, but it's not something we're actively trying to address so cleanup time.