Skip to content

ci: parallel arm64/amd64 builds with manifest merge#47

Merged
PenguinzTech merged 7 commits intov2.1.xfrom
feature/arm64-parallel-builds
Apr 22, 2026
Merged

ci: parallel arm64/amd64 builds with manifest merge#47
PenguinzTech merged 7 commits intov2.1.xfrom
feature/arm64-parallel-builds

Conversation

@PenguinzTech
Copy link
Copy Markdown
Contributor

@PenguinzTech PenguinzTech commented Apr 21, 2026

Summary

  • Splits push.yml and release.yml Docker builds into a matrix job (amd64 + arm64 run in parallel), followed by a merge job that assembles the multi-arch manifest via docker buildx imagetools create
  • Each platform builds and pushes by digest only — no tag is applied until the merge job combines both into a single manifest list
  • Upgrades actions/upload-artifact → v7 and actions/download-artifact → v8 (both pinned to verified commit SHAs)
  • Pins previously floating actions/checkout@v4 to commit SHA in build.yml

Test plan

  • Verify push.yml triggers on push to v2.1.x and all three jobs (generate-tag, build ×2, merge) pass
  • Confirm the resulting image tag resolves as a multi-arch manifest: docker buildx imagetools inspect ghcr.io/penguintechinc/squawk:<beta-tag>
  • Verify release.yml produces correct semver tags on a published release
  • Confirm no single-arch images are briefly visible under the final tag during the build window

🤖 Generated with Claude Code

Summary by Sourcery

Make Docker image publishing workflows build architecture-specific images in parallel and assemble them into a single multi-arch manifest, while pinning GitHub Actions dependencies to specific versions.

Build:

  • Refactor push and release workflows to use matrix builds for linux/amd64 and linux/arm64 with a follow-up merge job that creates and tags a multi-arch manifest from per-arch digests.
  • Update actions/upload-artifact to v7 and actions/download-artifact to v8, and pin actions/checkout@v4 to a specific commit SHA across workflows.

…ase workflows

Build each platform independently in a matrix job, push by digest (no tag),
then assemble the multi-arch manifest in a merge job. Cuts wall-clock build
time roughly in half versus sequential QEMU emulation.

- push.yml: split into generate-tag → build (matrix) → merge jobs
- release.yml: split into build (matrix) → merge jobs
- Both workflows: upgrade to upload-artifact@v7, download-artifact@v8
- build.yml: pin previously floating actions/checkout@v4 to commit SHA

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented Apr 21, 2026

Reviewer's Guide

Refactors the push and release GitHub Actions workflows to build amd64 and arm64 Docker images in parallel as per‑platform digest pushes, then assembles a multi‑arch manifest in a dedicated merge job, while upgrading/pinning core GitHub Actions dependencies to specific SHAs.

Sequence diagram for parallel multi-arch push workflow with manifest merge

sequenceDiagram
    actor Developer
    participant GitHub_Actions
    participant Job_generate_tag
    participant Job_build_amd64
    participant Job_build_arm64
    participant GitHub_Artifacts
    participant GHCR
    participant Job_merge

    Developer->>GitHub_Actions: Push to branch v2.1.x
    GitHub_Actions->>Job_generate_tag: Start generate-tag job
    Job_generate_tag->>Job_generate_tag: Compute tag from ref
    Job_generate_tag-->>GitHub_Actions: Output tag

    GitHub_Actions->>Job_build_amd64: Start build job (linux/amd64)
    GitHub_Actions->>Job_build_arm64: Start build job (linux/arm64)

    par Parallel_platform_builds
        Job_build_amd64->>GHCR: Push image by digest only
        Job_build_amd64->>GitHub_Artifacts: Upload digest artifact (digest-linux-amd64)

        Job_build_arm64->>GHCR: Push image by digest only
        Job_build_arm64->>GitHub_Artifacts: Upload digest artifact (digest-linux-arm64)
    end

    GitHub_Actions->>Job_merge: Start merge job after generate-tag and build
    Job_merge->>GitHub_Artifacts: Download all digest-* artifacts
    Job_merge->>Job_merge: Set up buildx and login
    Job_merge->>GHCR: docker buildx imagetools create 
    Note over Job_merge,GHCR: Create multi-arch manifest tagged IMAGE:tag
Loading

File-Level Changes

Change Details Files
Introduce matrix-based per-platform Docker builds that push images by digest only, followed by a dedicated merge job that creates and tags a multi-arch manifest.
  • Split the original single build-and-push job into a tag-generation job, a matrix build job for linux/amd64 and linux/arm64, and a merge job in push.yml.
  • Configure docker/build-push-action to build per-platform images, push by digest with canonical names, and emit digests as outputs.
  • Export per-platform digests to files under /tmp/digests and upload them as short-lived artifacts keyed by platform.
  • Add merge jobs that download all digest artifacts, log in to the registry, and run docker buildx imagetools create to assemble and push a multi-arch manifest under the final tag(s).
.github/workflows/push.yml
.github/workflows/release.yml
Adjust metadata/tag handling to align with the new split build/merge flow for push and release workflows.
  • Move tag generation for push.yml into its own generate-tag job and wire its output into downstream jobs via needs outputs.
  • Use docker/metadata-action without explicit tags during per-platform builds in push.yml and re-apply tags only at merge time when creating the manifest.
  • In release.yml, keep semver and ref-based tags in the metadata step but apply them only during the manifest creation in the merge job instead of on the per-platform builds.
.github/workflows/push.yml
.github/workflows/release.yml
Upgrade artifact actions and pin checkout to specific SHAs for reproducible CI.
  • Replace actions/upload-artifact usage with v7 pinned to a verified commit SHA in push.yml and release.yml.
  • Replace actions/download-artifact usage with v8 pinned to a verified commit SHA in the merge jobs.
  • Pin actions/checkout@v4 to a specific commit SHA in push.yml, release.yml, and build.yml for all usages.
.github/workflows/push.yml
.github/workflows/release.yml
.github/workflows/build.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The matrix build for linux/arm64 relies on cross-building but only sets up Buildx; consider adding docker/setup-qemu-action to ensure arm64 emulation is available and avoid architecture-specific build failures on GitHub-hosted runners.
  • There is substantial duplication between push.yml and release.yml for the matrix, digest artifact, and merge logic; consider extracting this into a reusable workflow or composite action to keep the logic DRY and easier to maintain.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The matrix build for linux/arm64 relies on cross-building but only sets up Buildx; consider adding docker/setup-qemu-action to ensure arm64 emulation is available and avoid architecture-specific build failures on GitHub-hosted runners.
- There is substantial duplication between push.yml and release.yml for the matrix, digest artifact, and merge logic; consider extracting this into a reusable workflow or composite action to keep the logic DRY and easier to maintain.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

PenguinzTech and others added 6 commits April 21, 2026 15:39
…ackage

penguin-libs/packages/python-limiter no longer exists, causing pip install
to fail on every CI run. Switch to the published flask-limiter>=4.1.1 and
update app.py to use its API (Limiter + get_remote_address, storage_uri).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…d pattern to server-release and cron workflows

python-ldap requires libldap2-dev/libsasl2-dev on the runner host to build
its C extension. Add apt-get install step to build.yml and server-release.yml
before pip install.

Also stages hook-applied parallel build pattern updates to server-release.yml
and cron.yml (same digest-based matrix + merge approach as push/release).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
debian:bookworm-slim and python:3.13-slim-bookworm digests in
dns-server/Dockerfile, Dockerfile.dns-server, and Dockerfile.api
were pointing to a deleted manifest. Updated to current digests.

Also stages hook-applied parallel build updates to go-client-release.yml.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
debian:bookworm-slim does not include python3.13 in its default apt repos.
Using the official python:3.13-slim-bookworm image removes the need to
manually install Python and pip, consolidates the two apt layers into one,
and pins to a verified digest.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-slim-bookworm

debian:bookworm-slim does not carry python3.13 in its default apt repos; the
python:3.13-slim-bookworm image ships Python 3.13, pip, and venv pre-installed.
Consolidates the two RUN apt layers into one and removes the manual pip bootstrap.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
actions/checkout v4 → v5, actions/setup-python v5 → v6,
codecov/codecov-action v4 → v6. Node.js 20 actions are deprecated
and will be forced to Node.js 24 on June 2nd, 2026.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@PenguinzTech PenguinzTech merged commit 240e0aa into v2.1.x Apr 22, 2026
19 checks passed
@PenguinzTech PenguinzTech deleted the feature/arm64-parallel-builds branch April 22, 2026 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant