Elixir.Macro.Env.beam file differs between builds

[bmwiedemann]

Environment

• Elixir & Erlang/OTP versions (elixir --version): elixir-1.10.3 ; erlang-22.3
• Operating system: openSUSE-Tumbleweed 20200427

Current behavior

Builds of elixir packages differ between 1-core-VM and 4-core-VM in Elixir.Macro.Env.beam
The start of the main diff has elixir-1.10.3/lib/elixir/lib/macro/env.exj Dbgi

Two builds on 4-core-VMs produce proper identical results.

Expected behavior

It should be possible to produce identical build results anywhere, anytime.
See https://reproducible-builds.org/ for why this matters.

This is sort of a regression on what was all fixed back in #8689
My last reproducible build happened on 2020-04-01 with elixir-1.9.4

How to reproduce:

osc co openSUSE:Factory/elixir && cd $_
for N in 1 4 ; do
  osc build --no-service --vm-type=kvm \
    --keep-pkg=RPMS.$N -j$N
  unrpm RPMS.$N/elixir-1.10.3-0.noarch.rpm
  hexdump -C usr/lib/elixir/lib/elixir/ebin/Elixir.Macro.Env.beam > $N.hex
done
diff -u {1,4}.hex

[josevalim]

Hi @bmwiedemann! We do have tests for reproducible builds since then but, since compilation happens in parallel, we may have race conditions. Can you send us a copy of both beam files? That will help us iron out the issues. Thanks!

[bmwiedemann]

Here are the two differing files: https://rb.zq1.de/other/elixir/

[bmwiedemann]

Tested master now and it looks fine again. 👍

If you want to improve your reproducible builds tests to catch more of this kind in the future, you can run one of the builds under taskset 1. I found that is often sufficient to cause a process scheduling similar to a 1-core-VM.
In my setup, total build duration only increases from 400 to 619 s between 4 and 1 core.

[josevalim]

@bmwiedemann oh, this is a very neat idea. I wrote a PR for it on #10005. Let's see how it goes in our CIs. Btw, it is hard to know if #10002 solved the issue for good, so if you see any other issues please let us know!