Skip to content

fix: split docker-build.yml multi-arch build to fix arm64 QEMU timeout - #1232

Merged
Wikid82 merged 4 commits into
mainfrom
fix/docker-build-arm64-timeout
Aug 10, 2026
Merged

fix: split docker-build.yml multi-arch build to fix arm64 QEMU timeout#1232
Wikid82 merged 4 commits into
mainfrom
fix/docker-build-arm64-timeout

Conversation

@Wikid82

@Wikid82 Wikid82 commented Aug 10, 2026

Copy link
Copy Markdown
Owner

Summary

Splits the monolithic build-and-push job in docker-build.yml into a parallel multi-arch pipeline:

  • setup — shared prep (tags, metadata) extracted once, feeding both arch legs
  • build-amd64 / build-arm64 — run concurrently on native/QEMU runners, each with its own bounded timeout (5m / 15m / 25m respectively)
  • merge-and-publish — merges the per-arch digests into a single multi-platform manifest, publishes to GHCR/Docker Hub, runs SBOM/attestation, Cosign signing, and Trivy scanning
  • scan-pr-image — PR-only image scan consuming merge-and-publish's digest output

Why

The previous single-job build ran amd64 and arm64 sequentially inside one job, and the arm64 leg (running under QEMU emulation) frequently exceeded the job's overall timeout, causing flaky CI failures (see PR #1230 arm64 timeout flake). Splitting into parallel jobs with per-arch timeout budgets fixes the flake and cuts wall-clock time for the common case.

Full design/rationale: docs/plans/current_spec.md (not included in this PR — tracked separately, out of scope here).

Test plan

  • actionlint clean
  • lefthook run pre-commit clean
  • Manual job-graph trace (needs/outputs wiring) reviewed
  • Live validation: trigger this workflow on this PR and confirm setup/build-amd64/build-arm64 all succeed within their timeout budgets, run concurrently, merge-and-publish produces a genuine multi-platform manifest (verified via docker buildx imagetools inspect), and scan-pr-image succeeds against the merged digest

Hoists the skip-check, Alpine digest resolution, branch-tag computation,
PR image-ref contract resolution, and docker/metadata-action call into a
standalone `setup` job with a full outputs: contract. `build-and-push`
now runs `needs: setup` and reads all shared values via
`needs.setup.outputs.*` instead of local `steps.*` refs.

No behavior change: build-and-push still performs the combined
linux/amd64,linux/arm64 build exactly as before. This is groundwork for
splitting the build into parallel per-platform jobs (build-amd64,
build-arm64) to fix the arm64 QEMU cross-compile job-timeout flake, per
docs/plans/current_spec.md.
The combined `linux/amd64,linux/arm64` build in a single job intermittently
exceeded its 20-minute job timeout when the QEMU-emulated arm64 cross-compile
shared budget with the fast native amd64 build (observed: PR #1230, run
31387413978, cancelled at 19m39s mid arm64 backend-builder compile).

Splits build-and-push into build-amd64 (timeout 15m) and build-arm64
(timeout 25m, matching the previous per-attempt retry budget), each pushing
its single-platform image by digest to a throwaway per-run tag, then adds
merge-and-publish (timeout 10m) which composes both digests into one
multi-platform manifest list via `docker buildx imagetools create` against
GHCR and Docker Hub independently, verifies GHCR/Docker Hub index digest
parity, and absorbs every post-build step from the old combined job (PR
artifact save, Caddy/CrowdSec CVE-2025-68156 verification, Trivy scan, SBOM
generation+attestation, Cosign signing of the merged digest) now pointed at
the merged manifest-list digest instead of a single-platform one.

Also re-enables per-platform GHA layer caching (scope=docker-build-amd64 /
-arm64, with no-cache-filter for caddy-builder/crowdsec-builder preserved)
now that the Dockerfile's builder stages use BuildKit --mount=type=cache
(commit f6361dc), which structurally supersedes the January-2026
Trivy false-positive issue that --no-cache originally worked around.

Collapses the Caddy/CrowdSec CVE verification steps' duplicated IMAGE_REF
resolution if/else (appeared 3x) into a single "Resolve merged image
reference" step per CLAUDE.md's DRY guideline, since restructuring that
exact logic to point at the merged digest was required anyway.

scan-pr-image is rewired to needs: merge-and-publish (renamed from
build-and-push) with internal logic otherwise unchanged.

See docs/plans/current_spec.md for full design rationale (Supervisor
APPROVED, 2 review rounds) and a note on where this implementation departs
from the plan's literal retry-wrapper text (nick-fields/retry cannot nest a
`uses:` action step) — resolved via the same --iidfile-based raw buildx CLI
mechanism the plan's §3.2 already specifies for digest capture.
Updates the "Docker Image Build" example in ARCHITECTURE.md to reflect
docker-build.yml's new build-amd64/build-arm64/merge-and-publish job
graph instead of the old single combined
`buildx build --platform amd64,arm64` invocation.
@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-advanced-security

Copy link
Copy Markdown
Contributor

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

✅ Supply Chain Verification Results

PASSED

📦 SBOM Summary

  • Components: 1494

🔍 Vulnerability Scan

Severity Count
🔴 Critical 0
🟠 High 0
🟡 Medium 5
🟢 Low 2
Total 14

📎 Artifacts

  • SBOM (CycloneDX JSON) and Grype results available in workflow artifacts

Generated by Supply Chain Verification workflow • View Details

nanoid <3.3.17 (pinned transitively via postcss) has a high-severity
DoS in custom generators when size=0. Bumps the lockfile pin and
drops the now-stale brace-expansion allowlist entry from audit-ci.json
that audit-ci itself flagged as no longer matching any finding.
@Wikid82
Wikid82 merged commit 3bdd618 into main Aug 10, 2026
47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants