fix(build): cross-compile Docker images instead of emulating with QEMU - #2339
Merged
Conversation
The multi-arch release build ran the full Rust release build, the cgo-linked Go build, and go mod download under QEMU for linux/arm64, making the docker publish job take hours on an amd64 runner. Builder stages are now pinned to the build platform and cross-compile to TARGETARCH: diffgen via rustup target aarch64-unknown-linux-gnu and the Go binary via GOARCH/CC with the aarch64 cross toolchain. Only the small final runtime stage runs under emulation. Also fetch cargo dependencies from the manifests alone so the layer survives diffgen source changes, and build both release platforms in the PR workflow so cross-compilation breakage surfaces before release. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016NaE7LcgYZXbx5igVaN2Rg
|
|
Contributor
WalkthroughThe container build now uses Docker Buildx for Linux amd64 and arm64. The Dockerfile cross-compiles the Rust library and Go binary with target-specific toolchains, compilers, and architecture settings. ChangesMulti-platform container build
Sequence Diagram(s)sequenceDiagram
participant BuildWorkflow
participant DockerBuildx
participant RustBuild
participant GoBuild
participant ContainerImage
BuildWorkflow->>DockerBuildx: build linux/amd64 and linux/arm64
DockerBuildx->>RustBuild: compile target-specific diffgen library
RustBuild->>GoBuild: copy Rust static library
GoBuild->>ContainerImage: build target-specific production binary
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
BenchstatBase: 📊 2 minor regression(s) (all within 5% threshold)
Full benchstat output |
Member
Author
|
CI note: Generated by Claude Code |
Gavel resultsGavel exited with code . |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Since
a33bb6caddedplatforms: linux/amd64,linux/arm64, the release docker publish job builds the arm64 image under QEMU emulation on a single amd64ubuntu-latestrunner. The Dockerfile is close to a worst case for emulation — a full Rust release build (cargo build --releasefor diffgen), a cgo-linked Go build (-tags rustdiffgen), andgo mod downloadall run emulated, which is typically 5–20× slower than native. The amd64-only baseline for "Build and push image" was ~20 minutes (run 1476); the first multi-arch run (30620320760) burned ~70 minutes on attempt 1 and was still building deep into attempt 2.Fix
Cross-compile instead of emulating. All builder stages are pinned to
--platform=$BUILDPLATFORMso they always run natively on the build host, and target$TARGETARCH:rustup target add aarch64-unknown-linux-gnu+gcc-aarch64-linux-gnuas the linker; the staticlib is copied totarget/release/libdiffgen.a, the fixed pathrustdiffgen.go's cgo directive links against.CGO_ENABLED=1 GOARCH=$TARGETARCH CC=aarch64-linux-gnu-gcc make build-prod.BUILDARCH == TARGETARCH, e.g. localmake docker) skip the cross toolchain entirely and behave as before.Only the small final runtime stage still runs under emulation (
useradd+config-db go-offline, which is network-bound).Also included:
cargo fetchnow depends only onCargo.toml/Cargo.lock(with a stublib.rs), so the dependency layer survives diffgen source changes.target/releasefrom the rust stage instead of the entire multi-GBtargetdirectory.docker/setup-buildx-actionSHA asflanksource/action-workflows.Validation
diffgentoaarch64-unknown-linux-gnuwith the exact toolchain packages from the Dockerfile; verified the produced staticlib/cdylib areELF 64-bit ARM aarch64.CGO_ENABLED=1 GOARCH=arm64 CC=aarch64-linux-gnu-gcc make build-prodlinking the cross-builtlibdiffgen.a— build succeeded and the output isELF 64-bit LSB executable, ARM aarch64, dynamically linked, interpreter /lib/ld-linux-aarch64.so.1.flanksource/base-image:0.7.1digest is a multi-arch index containing amd64 and arm64.case/ifshell logic for amd64-native, arm64-cross, and unsupported-arch paths.🤖 Generated with Claude Code
https://claude.ai/code/session_016NaE7LcgYZXbx5igVaN2Rg