find-debuginfo.sh: sort output of find

[bmwiedemann]

sort output of find in find-debuginfo.sh
to make build results more reproducible
in spite of indeterministic filesystem readdir order.

Even where these lists do not become part of an rpm,
it makes it easier to debug unrelated issues
by having less overall diff from diffing build filesystem trees (one of the standard reproducibility-debugging techniques I use).

For openSUSE, this helped to make squid, openssh, postfix and shadow
packages build reproducibly.

See https://reproducible-builds.org/ for why this is good.

Note: elfbins.list has remaining indeterminism from the run_job function running in parallel unless using -j1

Note: only tested in a slightly different variant on openSUSE's rpm-4.14.1

[rpm-maint]

Hi Bernhard,

On Mon, 2018-07-30 at 17:19 +0000, Bernhard M. Wiedemann wrote: sort output of find in find-debuginfo.sh to make build results more reproducible in spite of indeterministic filesystem readdir order. Even where these lists do not become part of an rpm, it makes it easier to debug unrelated issues by having less overall diff from diffing build filesystem trees (one of the standard reproducibility-debugging techniques I use). For openSUSE, this helped to make squid, openssh, postfix and shadow packages build reproducibly. See https://reproducible-builds.org/ for why this is good.

I can see why this is good. But would like to know which ones are necessary to produce reproducible builds (I assume the order in which dwz handles its arguments in one) and which are just there make diffing the build filesystem tree easier.

Note: elfbins.list has remaining indeterminism from the `run_job` function running in parallel unless using `-j1`

So, this isn't a problem for creating reproducible builds? Also you could easily fix that by adding a sort of the files after all jobs are done. e.g. cat "$temp"/elfbins.* | sort >"$ELFBINSFILE" BTW. Does sort depend on the locale set? Do we make sure it is set to some known locale? Cheers, Mark

[bmwiedemann]

We could add on top export LC_ALL=C to make sure locales do not influence the result (and drop the other 2 inline instances of that).
I did not yet bisect which of the changes makes what difference because test-cycles are slow.

Note: elfbins.list has remaining indeterminism from the run_job
function running in parallel unless using -j1

So, this isn't a problem for creating reproducible builds?

No, that seems to not make a difference on rpms - only makes the build filesystem diff a bit messier.

[bmwiedemann]

I experimented some more with it and found that only the sort in dwz_files
makes all the difference to the shadow binary rpm.
Do you prefer a minimal patch?

[bmwiedemann]

updated to be minimal

[pmatilai]

Merged, thanks for the patch.