-
Notifications
You must be signed in to change notification settings - Fork 58
Add cross-compilation support for Linux and optimize the release workflow #302
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
aimxhaisse
reviewed
May 28, 2025
@aimxhaisse I've addressed all of your comments now I believe, so when you get a chance please review them again. Thank you! |
aimxhaisse
approved these changes
Jun 2, 2025
ltitanb
approved these changes
Jun 2, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR refactors the way Linux binaries are built to use a new Docker builder (via
buildx
) that can cross-compilearm64
binaries onx64
platforms and vice-versa. The current setup uses Docker's QEMU-based setup to do cross-compilation which is, while effective, incredibly slow. Using Rust's native cross-compilation support is considerably faster, which is what this PR does. This takes the end-to-end time from 55 minutes down to 6 minutes on my local machine.The refactor breaks the builder portion of the PBS and Signer Dockerfiles, which was previously duplicated between them, into a new
build.Dockerfile
that can be used to create the binaries and store them on the local filesystem as output artifacts. These binaries can then be used directly as release artifacts (which is also done, discussed later), and imported in the newpbs.Dockerfile
andsigner.Dockerfile
images directly. That way building the CLI, PBS, and Signer binaries only needs to be done once instead of 3 separate times which dramatically reduces overhead.Simultaneously, it introduces cross-compilation support to
build.Dockerfile
which the previous build process didn't include. The implementation of this is fairly down-in-the-weeds, setting up environment variables and multiarch packages.As part of that effort, this PR refactors the GitHub release workflow with the new cross-compiler setup. Linux binary builds and Darwin binary builds are now separate jobs since they use different techniques.
The Linux build step uses the cross-compiler binary to generate 6 new binaries at the same time: the CLI, PBS, and Signer - all in both
x64
andarm64
variants. The PBS and Signer Docker image steps copy these binaries directly when generating their images respectively.The Darwin builder only produces the typical CLI, PBS, and Signer binaries for
arm64
via direct compilation on Docker'smacos-latest
runner (no Docker shenanigans involved). While GitHub provides anx64
Darwin runner (macos-latest-large
), it's not available on the free tier. Creatingx64
binaries for Darwin would have to be done by cross-compiling on thearm64
runner which is not included in this PR.