CI feature: Reproducible build #8701

eksperimental · 2019-01-25T04:47:39Z

/cc @bmwiedemann

Main issue: #8689
Related PR: #8694

eksperimental · 2019-01-25T04:50:39Z

I'm uploading it to see if it works with Travis.

It did locally with this Bash script
https://gist.github.com/eksperimental/d44698b69066958d7bd3554e60566342

UPDATE: it worked in the first go!

Looking forward to hearing your comments

josevalim · 2019-01-25T08:00:51Z

@eksperimental I like this a lot. I would just propose to run those changes at the end of the suite, this way we don't need to remove the trailing ebin_reproducible directories and we don't block the test results on this. However, in order to do that, we would need to run make compile after we remove .git the first time around too (which I also think is a good thing to test).

fertapric · 2019-01-25T08:20:50Z

Would it make sense to move this into a script so you can run it locally?

josevalim · 2019-01-25T08:26:44Z

@fertapric I want to do that but currently there is no good placement for it as travis does not allow us to move the .travis.yml around. They do have an open issue and my plan is to move such scripts as well as the .yml files to a CI directory. We just need to wait a big longer.

bmwiedemann · 2019-01-25T08:54:33Z

The travis.yml is already running shell commands like make compile, so you can add + call a path/to/new/script

josevalim · 2019-01-25T08:56:39Z

We wanted to avoid adding assorted scripts to the repository but I believe the answer is that we can simply add this to the makefile for now. :D

josevalim · 2019-01-25T09:18:44Z

@eksperimental another suggestion, that came up on #8702, is for us to have a specific entry in the test matrix that checks for reproducible long-term builds. In this matrix entry we would:

Do the diff check you perform in this commit right now
Use libfaketime to change the time and run the tests "in the future"

This way we keep those concerns isolated in the CI instead of running them on every OTP version. We can use the latest OTP version on this build.

eksperimental · 2019-01-25T18:29:52Z

Do we want to run tests in the future, or to compile as well?

eksperimental · 2019-01-25T20:29:06Z

I ran faketime ... make test and also compiled it with faketime and withouth faketime, and I get like "9 doctests, 520 tests, 71 failures" when run locally.

It doesn't work. So I would skip it for now.
You can see it the Travis result.
https://travis-ci.org/elixir-lang/elixir/jobs/484525540

eksperimental · 2019-01-25T21:41:54Z

@josevalim can you have a look at this and tell me what would be the cause of the error
https://travis-ci.org/elixir-lang/elixir/jobs/484534189#L755
could it be because we are not running make clear ?

See elixir-lang#8689 for more information

eksperimental · 2019-01-26T07:00:20Z

Alright, I 'm done with all the requests. Please have a look.

The new command is make check_reproducible
I wasn't really happy with the idea that we had to pass SOURCE_DATE_EPOCH so I came up with a solution.
We could even run make test check_reproducible and it will work.

.gitignore

josevalim · 2019-01-26T07:57:05Z

Makefile

+	$(Q) mv lib/iex/ebin/* lib/iex/ebin_reproducible/
+	$(Q) mv lib/logger/ebin/* lib/logger/ebin_reproducible/
+	$(Q) mv lib/mix/ebin/* lib/mix/ebin_reproducible/
+	SOURCE_DATE_EPOCH="$(call SOURCE_DATE_EPOCH)" $(MAKE) compile


do you want to do a call SOURCE_DATE_EPOCH? If you export above, it is enough, no?

I cannot make it work on Travis.
I had pushed a commit right before you mention this, exporting it.
if it works, i will see If I can avoid passing the variable explicitely

Yeah, i saw the new commit, I think you have to remove the call here for the export to work. I just think though... :D not sure!

.travis.yml

josevalim · 2019-01-26T08:13:19Z

@josevalim can you have a look at this and tell me what would be the cause of the error
https://travis-ci.org/elixir-lang/elixir/jobs/484534189#L755
could it be because we are not running make clear ?

I am not sure why. If I had to guess, it changed the time for the VM but not for the filesystem operations.

Makefile

josevalim · 2019-01-26T11:03:56Z

@eksperimental I pushed a commit where we can compute the epoch correctly but make is failing for some reason.

eksperimental · 2019-01-26T19:12:29Z

@josevalim I'm running ubuntu, but a later version than the one in CI, and this is working differently. exporting variables work locally, maybe it is the GNUMake version that Travis is running.

The problem with your solution is that we need to determine SOURCE_DATE_EPOCH before moving the files from ebin to ebin_reproducible, and then use this value again when compiling the last time.

As I'm writing this the new implementation seems to have worked. So, have a look please

eksperimental · 2019-01-26T19:38:08Z

This is ready to be merged!

Makefile

josevalim · 2019-01-26T20:06:46Z

Two minor comments and we should be good to go!

eksperimental · 2019-01-28T04:53:06Z

Two minor comments and we should be good to go!

done!

josevalim · 2019-01-28T09:24:26Z

Amazing, thank you! ❤️ 💚 💙 💛 💜

eksperimental changed the title ~~Reproducible build~~ CI feature: Reproducible build Jan 25, 2019

bmwiedemann mentioned this pull request Jan 25, 2019

datetime_test fails in 2019-11 #8702

Closed

eksperimental force-pushed the reproducible_build branch 4 times, most recently from 4a4b35f to 7881cfa Compare January 25, 2019 18:05

eksperimental force-pushed the reproducible_build branch from e4820bd to b256dc9 Compare January 25, 2019 20:11

eksperimental force-pushed the reproducible_build branch from 3d4a069 to ece2b72 Compare January 25, 2019 20:34

eksperimental added 8 commits January 26, 2019 13:34

Add Reproducible Build feature to CI

ce1bab4

See elixir-lang#8689 for more information

Move commands after tests

0083ab6

Run "make compile" after without .git dir

67c4d42

do not delete files in ebin_reproducible

3674ce7

Only execute reproducible build in lastest OTP release

f3fb8a3

run tests in the future

7cc1620

Revert changes related to Run tests in the Future

2769755

Run "make clean" before second "make compile"

acd1d70

eksperimental force-pushed the reproducible_build branch from bbac840 to e1cf6eb Compare January 26, 2019 06:44

Move to Makefile

db810db

eksperimental force-pushed the reproducible_build branch from e1cf6eb to db810db Compare January 26, 2019 07:18

fix

da635ac

josevalim reviewed Jan 26, 2019

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

export SOURCE_DATE_EPOCH in Makefile to see if it works in Travis

86c19d0

josevalim reviewed Jan 26, 2019

View reviewed changes

.travis.yml Outdated Show resolved Hide resolved

fix over fix

170f76c

implement Jose's suggestions

ac632a3

josevalim reviewed Jan 26, 2019

View reviewed changes

Makefile Outdated Show resolved Hide resolved

josevalim reviewed Jan 26, 2019

View reviewed changes

Makefile Outdated Show resolved Hide resolved

don't clean files at the very end

63d8bb5

fertapric reviewed Jan 26, 2019

View reviewed changes

Makefile Outdated Show resolved Hide resolved

josevalim and others added 3 commits January 27, 2019 00:50

Update Makefile

9b43b0e

Update Makefile

19d75df

Makefile: implement (read/write)_SOURCE_DATE_EPOCH

d29239f

eksperimental force-pushed the reproducible_build branch from ca2976c to d29239f Compare January 26, 2019 19:05

Add back all OTP releases to .travis.yml

befe9cc

josevalim reviewed Jan 26, 2019

View reviewed changes

Makefile Outdated Show resolved Hide resolved

josevalim reviewed Jan 26, 2019

View reviewed changes

Makefile Outdated Show resolved Hide resolved

Use uppercase in Mafile function names

8361c43

eksperimental force-pushed the reproducible_build branch from b913411 to 8361c43 Compare January 28, 2019 04:36

josevalim merged commit 3d5b341 into elixir-lang:master Jan 28, 2019

CI feature: Reproducible build #8701

CI feature: Reproducible build #8701

Uh oh!

Conversation

eksperimental commented Jan 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eksperimental commented Jan 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

josevalim commented Jan 25, 2019

Uh oh!

fertapric commented Jan 25, 2019

Uh oh!

josevalim commented Jan 25, 2019

Uh oh!

bmwiedemann commented Jan 25, 2019

Uh oh!

josevalim commented Jan 25, 2019

Uh oh!

josevalim commented Jan 25, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eksperimental commented Jan 25, 2019

Uh oh!

eksperimental commented Jan 25, 2019

Uh oh!

eksperimental commented Jan 25, 2019

Uh oh!

eksperimental commented Jan 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

josevalim Jan 26, 2019

Choose a reason for hiding this comment

Uh oh!

eksperimental Jan 26, 2019

Choose a reason for hiding this comment

Uh oh!

josevalim Jan 26, 2019

Choose a reason for hiding this comment

Uh oh!

Uh oh!

josevalim commented Jan 26, 2019

Uh oh!

Uh oh!

Uh oh!

Uh oh!

josevalim commented Jan 26, 2019

Uh oh!

eksperimental commented Jan 26, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eksperimental commented Jan 26, 2019

Uh oh!

Uh oh!

Uh oh!

josevalim commented Jan 26, 2019

Uh oh!

eksperimental commented Jan 28, 2019

Uh oh!

josevalim commented Jan 28, 2019

Uh oh!

Uh oh!

eksperimental commented Jan 25, 2019 •

edited

Loading

eksperimental commented Jan 25, 2019 •

edited

Loading

josevalim commented Jan 25, 2019 •

edited

Loading

eksperimental commented Jan 26, 2019 •

edited

Loading

eksperimental commented Jan 26, 2019 •

edited

Loading