Skip to content

Conversation

jclapis
Copy link
Collaborator

@jclapis jclapis commented May 13, 2025

This PR refactors the way Linux binaries are built to use a new Docker builder (via buildx) that can cross-compile arm64 binaries on x64 platforms and vice-versa. The current setup uses Docker's QEMU-based setup to do cross-compilation which is, while effective, incredibly slow. Using Rust's native cross-compilation support is considerably faster, which is what this PR does. This takes the end-to-end time from 55 minutes down to 6 minutes on my local machine.

The refactor breaks the builder portion of the PBS and Signer Dockerfiles, which was previously duplicated between them, into a new build.Dockerfile that can be used to create the binaries and store them on the local filesystem as output artifacts. These binaries can then be used directly as release artifacts (which is also done, discussed later), and imported in the new pbs.Dockerfile and signer.Dockerfile images directly. That way building the CLI, PBS, and Signer binaries only needs to be done once instead of 3 separate times which dramatically reduces overhead.

Simultaneously, it introduces cross-compilation support to build.Dockerfile which the previous build process didn't include. The implementation of this is fairly down-in-the-weeds, setting up environment variables and multiarch packages.

As part of that effort, this PR refactors the GitHub release workflow with the new cross-compiler setup. Linux binary builds and Darwin binary builds are now separate jobs since they use different techniques.

The Linux build step uses the cross-compiler binary to generate 6 new binaries at the same time: the CLI, PBS, and Signer - all in both x64 and arm64 variants. The PBS and Signer Docker image steps copy these binaries directly when generating their images respectively.

The Darwin builder only produces the typical CLI, PBS, and Signer binaries for arm64 via direct compilation on Docker's macos-latest runner (no Docker shenanigans involved). While GitHub provides an x64 Darwin runner (macos-latest-large), it's not available on the free tier. Creating x64 binaries for Darwin would have to be done by cross-compiling on the arm64 runner which is not included in this PR.

@jclapis jclapis self-assigned this May 13, 2025
@jclapis jclapis added the core Core part of the repo (signer, modules interface) label May 13, 2025
@jclapis jclapis requested review from ManuelBilbao and ltitanb May 13, 2025 22:01
@jclapis
Copy link
Collaborator Author

jclapis commented May 29, 2025

@aimxhaisse I've addressed all of your comments now I believe, so when you get a chance please review them again. Thank you!

@ltitanb ltitanb merged commit a251554 into main Jun 2, 2025
2 checks passed
@ltitanb ltitanb deleted the cross-compile branch June 2, 2025 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core part of the repo (signer, modules interface)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants