Mismatch between hashes of local build and Circle public build of bin/gpg-agent

[morgulnachos]

Context of the Build

1. What board are you trying to build?
EOL_t480-hotp-maximized

2. What repository:branch are you using to build from?

• Heads:Master
• Other (please specify)

3. What version of Heads/coreboot are you trying to build?
aedae4f

4. In building the rom where did you get the blobs?

• No blobs required
• Provided by the company that installed Heads on the device
• Extracted from a backup rom taken from this device
• Extracted from another backup rom taken from another device (please identify the board model)
• Extracted from the online bios using the automated tools provided in Heads
• I don't know

5. If using the automated tools to get the blobs did you run the relevant scripts in the blobs directory

• Yes
• No

6. What operating system are you using

Ubuntu 24.04

Please describe the problem

Describe the bug
I can't reproduce the public build for EOL_t480-hotp-maximized on commit aedae4f
I reproduced the built on two independent machines. Both are debian based (ubuntu and linux mint).
I get the same hash for my local builds but it doesn't fit the public hash.
I flashed my local build and it works. So it's not a functionality problem.
I'm simply concerned whether I could've been subjected to a supply chain attack.

To Reproduce
Steps to reproduce the behavior:

1. Clone the Heads repository
2. Run ./docker_repro.sh make BOARD=EOL_t480-hotp-maximized
3. Compare hashes at build/x86/EOL_t480-hotp-maximized/hashes.txt with https://output.circle-artifacts.com/output/job/d6237762-42d9-49f1-98f5-0bb91420e5fd/artifacts/0/build/x86/EOL_t480-hotp-maximized/hashes.txt
4. See mismatch

./compare_hashes_of_heads_builds.py Downloads/heads_build_downloads/hashes.txt Documents/heads_local_build_output/EOL_t480-hotp-maximized/hashes.txt

DIFFERENT heads-EOL_t480-hotp-maximized-202607202001-v0.2.1-3094-gaedae4f.rom
left: 752a2106ce935b77ded1bd239eb5d74391e4c3840f8e1c5a04e31a263c741667
right: 4a025043dd24ac1c7b5fc0ea4d89607f24c31529574861369bc7e14e5098285e

DIFFERENT initrd.cpio.xz
left: 7ebabea7a7c067bec90511b3a64459433abb1fbad2be17cd6407512b25b32c94
right: 1a39b4c3cd22c14a3b28d5d34848f66794e00739e80905a25d852a9d9b7f9e2e

DIFFERENT tools.cpio
left: 7ca38b29495f511642cf0741433b3c47ad8be9aaa4e3d13ab039d1de9616f02f
right: 7270cccca414fe56205284b0aeb9c6a5a6210d143286bbd12fb3ca8777db89a0

DIFFERENT tools.cpio::bin/gpg-agent
left: a90d2bd42ded848af85c59a7cd62e746d652520f713a27b79a99cdc1f4af25e6
right: 5d683d1583a161fb36ca101488fe048a053a4ed7307ef4c85e724963e3a9e5e2

Entries in left: 434
Entries in right: 434
Matching: 430
Different: 4
Only left: 0
Only right: 0

Expected behavior
All hashes should match

Screenshots

Additional context

[tlaurion]

Reusing hashes.txt checks

wget https://output.circle-artifacts.com/output/job/d6237762-42d9-49f1-98f5-0bb91420e5fd/artifacts/0/build/x86/EOL_t480-hotp-maximized/hashes.txt -q -O /tmp/hashes.txt
HASHES1=/tmp/hashes.txt
HASHES2=build/x86/EOL_t480-hotp-maximized/hashes.txt
egrep '^[0-9a-f]{64}' $HASHES1 | while read line; do HASH_REF=$(echo $line|awk -F " " {'print $1'}); FILE_REF=$(echo $line|awk -F "/" {'print $NF'}); if ! grep -q "$HASH_REF" $HASHES2; then echo "$FILE_REF doesn't match";fi; done

Reproduced. Theory is that the new caching system caches artifacts that should be built clean, not reused from cached layers under circleci. checking

[tlaurion]

Actually, it seems that gnupg was trespassing docker's boundaries, probing host's /proc and kernel. Attempt to fix under #2172

[tlaurion]

Well, it seems that bumping binutils under musl-cross-make is making some output ureproducible.... Still digging.
This is why bumping versions is really time consuming. Could also just pin to older musl-cross-make's binutil if this gets too complicated.
#2173