Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

R4R: Infrastructure for reproducible builds #4262

merged 73 commits into from May 13, 2019


Copy link

@alessio alessio commented May 2, 2019

This change set introduces support for building gaia with gitian
on the following GOOS/GOARCH pairs:

  • darwin/386
  • darwin/amd64
  • linux/386
  • linux/amd64
  • linux/arm
  • linux/arm64
  • windows/386
  • windows/amd64

cmd/gaia/contrib/gitian-descriptors/ contains gitian descriptor files.

cmd/gaia/contrib/gitian-keys/ contains:

  • a keys.txt file that is meant to list core developers and gitian builders PGP keys.
  • to provide instructions on how to import the keys into one's
    personal GPG keyring.

The gosum utility is removed, so is the go.sum hashsum bit from gaiacli/gaiad
version string. It was meant to be a provisional mitigation to the lack of a
reproducible build process.

GOBIN is removed from all Makefiles. When GOBIN is set, go refuses to cross-compiles
binaries for foreign architectures. export GOBIN=$GOPATH/bin is unnecessary anyway
as by default go install places built binaries in $GOPATH/bin. Developers are required
to update their enviornment files and replace $GOBIN with $GOPATH/bin in PATH.

circleci configuration file is amended accordingly.

Instructions at

Closes: #4027
Closes: #4280

For Admin Use:

  • Added appropriate labels to PR (ex. wip, ready-for-review, docs)
  • Reviewers Assigned
  • Squashed all commits, uses message "Merge pull request #XYZ: [title]" (coding standards)
Copy link

@codecov codecov bot commented May 3, 2019

Codecov Report

Merging #4262 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #4262   +/-   ##
  Coverage    59.1%   59.1%           
  Files         217     217           
  Lines       14595   14595           
  Hits         8627    8627           
  Misses       5330    5330           
  Partials      638     638

alessio added 13 commits May 3, 2019
Instructions follow:

Make sure ruby is installed in your system and lives in /usr/bin/ruby:

 [ `which ruby` = '/usr/bin/ruby' ] && echo yes || echo no

Clone gitian:

 git clone

Prepare a docker image for the build:

 cd gitian-build
 bin/make-base-vm --docker
 bin/make-base-vm --docker --arch amd64
 export USE_DOCKER=1

Download go:

  mkdir inputs
  gopkg=go1.12.4.linux-amd64.tar.gz ; \
  curl -L$gopkg > inputs/$gopkg


  bin/gbuild $SDK-PATH/cmd/gaia/contrib/gitian-linux.yml \
    --commit cosmos-sdk=v0.34.3
Update version package accordingly.
@alessio alessio force-pushed the alessio/reproducible-builds branch from 2022bb9 to 9f28890 May 3, 2019
@alessio alessio changed the title Add Linux descriptor for Gitian builds Start building the infrastructure for reproducible builds May 3, 2019
@alessio alessio marked this pull request as ready for review May 3, 2019
@alessio alessio requested review from mircea-c and zmanian May 3, 2019
cmd/gaia/Makefile Show resolved Hide resolved
Copy link
Member Author

@alessio alessio commented May 9, 2019

Bnaries are now built with CGO enabled. I've successfully tested ledger support on Linux amd64.

Copy link

@liamsi liamsi left a comment

If I run ./cmd/gaia/contrib/ on macosx, I get a ./cmd/gaia/contrib/ line 196: 1: unbound variable. Update: OK, I see this needs a param passed in, e.g. ./cmd/gaia/contrib/ linux pwd.


Start the build
bin/gbuild:252:in ``': No such file or directory - sha256sum (Errno::ENOENT)
        from bin/gbuild:252:in `<main>'

OK, looks like coreutils installation failed. NVM.

Copy link

@liamsi liamsi commented May 10, 2019

The instructions should probably contain a sentence on how to verify that the build I've created matches the official build / I successfully reproduced it.

Copy link
Member Author

@alessio alessio commented May 10, 2019

@liamsi dixit:

The instructions should probably contain a sentence on how to verify that the build I've created matches the official build / I successfully reproduced it.

I will certainly do! And I will include instructions on how to retrieve and import builder keys into your GPG keyring, etc.

Copy link

@liamsi liamsi commented May 10, 2019

Update: coreutils wasn't in PATH ... NVM

I ran git pull to get the latest changes (and deleted the gaia.sigs and the gitian-build-linux dirs, and restarted the build (via ./cmd/gaia/contrib/ -s linux `pwd`) and got:

Updating apt-get repository (log in var/install.log)
Installing additional packages (log in var/install.log)
Upgrading system, may take a while (log in var/install.log)
Creating package manifest
Creating build script (var/build-script)
date: illegal time format
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ... 
            [-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
bin/gbuild:147:in `block (2 levels) in build_one_configuration': error looking up author date in cosmos-sdk (RuntimeError)
        from bin/gbuild:143:in `each'
        from bin/gbuild:143:in `block in build_one_configuration'
        from bin/gbuild:127:in `open'
        from bin/gbuild:127:in `build_one_configuration'
        from bin/gbuild:309:in `block (2 levels) in <main>'
        from bin/gbuild:304:in `each'
        from bin/gbuild:304:in `block in <main>'
        from bin/gbuild:302:in `each'
        from bin/gbuild:302:in `<main>'

Copy link

@liamsi liamsi commented May 10, 2019

@alessio alessio requested a review from liamsi May 13, 2019
Copy link

@jackzampolin jackzampolin left a comment

Builds the darwin for me!!

@alessio alessio requested a review from cwgoes May 13, 2019
@alessio alessio merged commit c0486aa into master May 13, 2019
15 checks passed
@alessio alessio deleted the alessio/reproducible-builds branch May 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet
Linked issues

Successfully merging this pull request may close these issues.

6 participants