[Improvement] Reproducible builds

[ilmanzo]

Feature Request

while packaging the software, we noticed that two subsequent builds sometimes produce a slightly different binary. This is probably related to the seccompiler that reads BPF filters definition from resources/seccomp/*.json file and uses an HashMap to store BPF data into a seccomp_filter.bpf blob that gets included in the target.

Describe the desired solution

We suggest using a data structure that preserves the order of insertion, so in the final binary we always get map keys in the same order, enabling to get identical binaries in the build.

[ilmanzo]

please find here a reproducer which highlights the problem. Ideally we should get only one line with the same MD5 repeated 100 times.

for i in $(seq 100) ; do seccompiler-bin --input-file unimplemented.json --target-arch x86_64 --output-file x > /dev/null ; md5sum x ; done | sort | uniq -c
     12 2f65b94907a365132025b35d467459c6  x
     15 5b500e29af8a3dc7ab0b7e1f99368a72  x
     17 a6e0a9f792210dfd122df4ea890e985b  x
     19 b39a8a3497d08814f2db7a18ff91140c  x
     17 ccf7c06d325fee897449f7a3a698a243  x
     20 e4b6bbb1f32214e8f11f4bbdee69fb40  x

[roypat]

Hi Andrea,

thanks for opening this issue! Having the build scripts produce reproducible output seems reasonable to me (and might actually prevent cargo from thinking it has to recompile things all the time).

Would you be interested in working on this issue? I suppose the first step would be to replace the HashMap with its deterministic sibling, BTreeMap, and check if that makes the builds reproducible.

Best,
Patrick

[ilmanzo]

Sure I can work on this. I was thinking about using an IndexMap if rust release permits , but I will do some tests and see.

[alindima]

(and might actually prevent cargo from thinking it has to recompile things all the time

This doesn't help with preventing cargo from recompiling the filters. That is already done by the build script, which watches and only reruns seccompiler if the JSON files have changed.

Also, is this the only thing that prevents Firecracker from having reproducible builds? I tend to doubt it.

Firecracker will move in version 2.0 to using the seccompiler from rust-vmm: https://github.com/rust-vmm/seccompiler, and there's an old branch for it with the work (needs rebasing): https://github.com/firecracker-microvm/firecracker/tree/feature/rust_vmm_adoption

Striving for reproducible builds seems reasonable to me too, but if you make those changes now in Firecracker, beware that they'll get overwritten sometime in the future.

[bmwiedemann]

Also, is this the only thing that prevents Firecracker from having reproducible builds? I tend to doubt it.

I tested it with #3445 and it produces bit-reproducible binaries in openSUSE. With the 2.0 version, we will just need to re-test and maybe re-fix.

[alindima]

I tested it with #3445 and it produces bit-reproducible binaries in openSUSE.

Great!

With the 2.0 version, we will just need to re-test and maybe re-fix.

You'd need to re-fix in a different place, in the seccompiler repo