Improved reproducible builds #806
+144
−33
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| name: reproducible-build-test | ||
|
|
||
| on: | ||
| workflow_dispatch: {} | ||
| schedule: | ||
| - cron: "0 1 */2 * *" | ||
|
|
||
| jobs: | ||
| build: | ||
| name: build reproducible binaries | ||
| runs-on: ${{ matrix.runner }} | ||
| strategy: | ||
| matrix: | ||
| include: | ||
| - runner: warp-ubuntu-2404-x64-32x | ||
| machine: machine-1 | ||
| - runner: warp-ubuntu-2204-x64-32x | ||
| machine: machine-2 | ||
| steps: | ||
| - uses: actions/checkout@v5 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Install rust | ||
| run: | | ||
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y | ||
| - name: Build reproducible binary with Docker | ||
| run: | | ||
| RUST_TOOLCHAIN=$(rustc --version | cut -d' ' -f2) | ||
| docker build \ | ||
| --build-arg "RUST_TOOLCHAIN=${RUST_TOOLCHAIN}" \ | ||
| -f docker/Dockerfile.reproducible -t rbuilder:release \ | ||
| --output type=local,dest=./target . | ||
| - name: Calculate SHA256 | ||
| id: sha256 | ||
| run: | | ||
| sha256sum target/reproducible/{rbuilder-operator,rbuilder-rebalancer} > checksums.sha256 | ||
| echo "Binaries SHA256 on ${{ matrix.machine }}: $(cat checksums.sha256)" | ||
| - name: Upload the hash | ||
| uses: actions/upload-artifact@v4 | ||
| with: | ||
| name: checksums-${{ matrix.machine }} | ||
| path: | | ||
| checksums.sha256 | ||
| retention-days: 1 | ||
|
|
||
| compare: | ||
| name: compare reproducible binaries | ||
| needs: build | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Download artifacts from machine-1 | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: checksums-machine-1 | ||
| path: machine-1/ | ||
| - name: Download artifacts from machine-2 | ||
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: checksums-machine-2 | ||
| path: machine-2/ | ||
| - name: Compare SHA256 hashes | ||
| run: | | ||
| echo "=== SHA256 Comparison ===" | ||
| echo "Machine 1 hashes:" | ||
| cat machine-1/checksums.sha256 | ||
| echo "Machine 2 hashes:" | ||
| cat machine-2/checksums.sha256 | ||
| if cmp -s machine-1/checksums.sha256 machine-2/checksums.sha256; then | ||
| echo "✅ SUCCESS: Binaries are identical (reproducible build verified)" | ||
| else | ||
| echo "❌ FAILURE: Binaries differ (reproducible build failed)" | ||
| exit 1 | ||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| ARG RUST_TOOLCHAIN=1.89.0 | ||
| FROM docker.io/rust:$RUST_TOOLCHAIN-trixie AS builder | ||
|
|
||
| ARG FEATURES VERSION | ||
| # Switch to snapshot repository | ||
| RUN sed -i '/^# http/{N;s|^# \(http[^ ]*\)\nURIs: .*|# \1\nURIs: \1|}' /etc/apt/sources.list.d/debian.sources | ||
| RUN apt-get -o Acquire::Check-Valid-Until=false update && \ | ||
| apt-get install -y \ | ||
| libjemalloc-dev \ | ||
| libclang-dev \ | ||
| protobuf-compiler \ | ||
| cmake | ||
| WORKDIR /build | ||
bakhtin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| COPY . . | ||
| RUN SOURCE_DATE=1730000000 make build && make build-deb | ||
bakhtin marked this conversation as resolved.
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment. Maybe use last commit timestamp as SOURCE_DATE?
Contributor
Author
There was a problem hiding this comment. It may not even be a git tree (e.g., if downloaded as a tar archive from GH) so I don't want to rely on it |
||
|
|
||
| FROM scratch AS artifacts | ||
| COPY --from=builder /build/target/x86_64-unknown-linux-gnu/ / | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,3 @@ | ||
| [toolchain] | ||
| channel = "stable" | ||
|
Contributor
Author
There was a problem hiding this comment.
|
||
| components = ["rustfmt", "clippy"] | ||
bakhtin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If rust is installed only for latest version, would it be better to just pin the version here instead of installing toolchain?
Same for release.yaml.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I install rust toolchain just to parse rust-toolchain.yml file and pull the correct Docker/Rust image. rust-toolchain may contain something like
stableor a numerical version of Rust. So it is hard to directly parse the file without (some) rust toolchain installed