Skip to content

fix: broken reproducibility#2014

Merged
gilcu3 merged 6 commits intomainfrom
1932-investigate-broken-reproducibility
Feb 10, 2026
Merged

fix: broken reproducibility#2014
gilcu3 merged 6 commits intomainfrom
1932-investigate-broken-reproducibility

Conversation

@gilcu3
Copy link
Contributor

@gilcu3 gilcu3 commented Feb 9, 2026

Closes #1932

The cluprit seems to have been a mix of jemalloc and the built crate. The jemalloc part was fixed in #1935. Would be nice to add it back

I got repro with this branch, locally:

commit hash: d56d8f112796d582aaf39923bce55f184b0cca70
SOURCE_DATE_EPOCH used: 0
node binary hash: 7bcec996bac31018387d166397815a364d07416a1dfbe703e39a667b6d2adc56
node docker image hash: sha256:24f18b462ac770a3701f84a34eff8fc4470abf15c4bb2291ed1537d9e470fd5e

and in CI https://github.com/near/mpc/actions/runs/21820809884/job/62953357578

added also a nit, as we are using/testing claude locally we should ignore the local config folder

This is the TL;DR of the fix:

The built crate generates ~20 constants via include!(), and we only use 4 of them. The unused ones still end up in the binary (they were pub const in a pub mod), and some would differ between local and CI (why this was not happening before is a mystery, but hopefully now we are more robust):

  • GIT_HEAD_REF: Locally Some("refs/heads/main"), in CI (detached HEAD from actions/checkout) it's None. Different string lengths → different .rodata layout.

These unused but still-present constants apparently shifted the positions of stuff in .rodata., obtained with:

readelf --wide --decompress --hex-dump=.rodata target/reproducible/mpc-node

@gilcu3 gilcu3 linked an issue Feb 9, 2026 that may be closed by this pull request
@gilcu3 gilcu3 marked this pull request as ready for review February 9, 2026 10:19
@gilcu3 gilcu3 enabled auto-merge February 9, 2026 12:22
Copy link
Contributor

@barakeinav1 barakeinav1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

@gilcu3 gilcu3 added this pull request to the merge queue Feb 10, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 10, 2026
@gilcu3 gilcu3 added this pull request to the merge queue Feb 10, 2026
Merged via the queue into main with commit 78a3d62 Feb 10, 2026
15 checks passed
@gilcu3 gilcu3 deleted the 1932-investigate-broken-reproducibility branch February 10, 2026 08:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate broken reproducibility

3 participants