Skip to content

Conversation

@bakhtin
Copy link
Contributor

@bakhtin bakhtin commented Nov 7, 2025

📝 Summary

  • Switch release pipeline to building inside the Docker container for better reproducibility. Previous implementation produced reproducible builds only on the same host. Building inside the Docker container allows producing the same binary on different hosts and even different OSes.
  • Set Docket image Rust version for building the binary by parsing a version configured for the project. That is to avoid manually updating Docker images when the Rust version bumps.
  • Add reproducible testing CI job that runs once every 2 days building on two different hosts and comparing the hashsums of produced binaries.
  • Switch to using jemalloc-sys from Debian repos instead of building it from source. A Debian version is reproducible which is hard to achieve if you build it from source. It does not require a runtime dependency as the library links statically.

💡 Motivation and Context


✅ I have completed the following steps:

  • Run make lint
  • Run make test
  • Added tests (if applicable)

Copilot AI review requested due to automatic review settings November 7, 2025 10:59
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements reproducible builds for the rbuilder project by standardizing the build environment and removing version-specific constraints. The changes introduce a Docker-based reproducible build system and add CI workflows to verify build reproducibility across different machines.

Key Changes:

  • Removed hardcoded Rust version constraint to allow more flexible toolchain management
  • Added Docker-based reproducible build infrastructure with controlled environment variables
  • Implemented CI workflow to test and verify build reproducibility across different Ubuntu versions

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
rust-toolchain.toml Removed hardcoded Rust version constraint
docker/Dockerfile.reproducible New Dockerfile for reproducible builds with snapshot repositories and controlled build environment
crates/rbuilder/Cargo.toml Added jemalloc-unprefixed feature flag for memory allocator configuration
Makefile Added RBUILDER_OPERATOR_FEATURES variable, JEMALLOC_OVERRIDE environment variable, and fixed .PHONE typo to .PHONY
.github/workflows/reprotest.yml New workflow to test build reproducibility by comparing binaries built on different machines
.github/workflows/release.yaml Updated release workflow to use Docker-based reproducible builds instead of direct compilation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings November 7, 2025 11:01
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings November 7, 2025 13:26
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings November 7, 2025 13:50
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings November 7, 2025 14:51
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated no new comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings November 7, 2025 15:24
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

echo "Machine 2 hashes:"
cat machine-2/checksums.sha256
if [ cmp -s machine-1/checksums.sha256 machine-2/checksums.sha256 ]; then
Copy link

Copilot AI Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The conditional statement syntax is incorrect. The cmp -s command should not be inside [ ] brackets. Use if cmp -s directly or use diff -q with proper conditional syntax.

Suggested change
if [ cmp -s machine-1/checksums.sha256 machine-2/checksums.sha256 ]; then
if cmp -s machine-1/checksums.sha256 machine-2/checksums.sha256; then

Copilot uses AI. Check for mistakes.
Copilot AI review requested due to automatic review settings November 7, 2025 16:14

This comment was marked as spam.

Signed-off-by: bakhtin <a@bakhtin.net>
@bakhtin bakhtin changed the title reprotest Improved reproducible builds Nov 7, 2025
@@ -1,4 +1,3 @@
[toolchain]
channel = "stable"
version = "1.88.0"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

version is not recognized as the valid option and ignored: https://rust-lang.github.io/rustup/overrides.html#the-toolchain-file

Comment on lines +25 to +31
- name: Install rust
run: |
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
- name: Build reproducible binary with Docker
run: |
RUST_TOOLCHAIN=$(rustc --version | cut -d' ' -f2)
Copy link
Contributor

@ilyaluk ilyaluk Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If rust is installed only for latest version, would it be better to just pin the version here instead of installing toolchain?

Same for release.yaml.

Copy link
Contributor Author

@bakhtin bakhtin Nov 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I install rust toolchain just to parse rust-toolchain.yml file and pull the correct Docker/Rust image. rust-toolchain may contain something like stable or a numerical version of Rust. So it is hard to directly parse the file without (some) rust toolchain installed

cmake
WORKDIR /build
COPY . .
RUN SOURCE_DATE=1730000000 make build && make build-deb
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use last commit timestamp as SOURCE_DATE?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may not even be a git tree (e.g., if downloaded as a tar archive from GH) so I don't want to rely on it

@bakhtin bakhtin merged commit 1c49435 into develop Nov 7, 2025
3 checks passed
@bakhtin bakhtin deleted the ab-reprotest branch November 7, 2025 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants