Skip to content

fix: produce real multi-arch images from ghcr and ecr workflows #128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rswanson merged 1 commit into from
May 5, 2026
+196 −76
Merged

fix: produce real multi-arch images from ghcr and ecr workflows #128

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 99 additions & 43 deletions .github/workflows/ecr-build-and-push.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
# Publishes the Docker image.
# Builds and pushes a multi-arch (linux/amd64 + linux/arm64) Docker image to
# Amazon ECR. Each arch is built natively on a matching GitHub-hosted runner
# and pushed by digest; a final job assembles the manifest list under the
# real tags (semver, branch, sha, latest).

name: Docker ECR Build and Push

Expand All @@ -12,7 +15,7 @@ on:
environment:
description: 'Environment to deploy to'
required: true
type: string
type: string
requires-private-deps:
description: 'Requires private dependencies to be fetched, sets up ssh-agent'
required: false
Expand All @@ -23,7 +26,7 @@ on:
required: false
default: 'Dockerfile'
type: string
secrets:
secrets:
aws-ecr-repository:
description: 'ECR repository to push to'
required: true
Expand All @@ -33,7 +36,7 @@ on:
SSH_PRIVATE_KEY:
description: 'SSH private key for fetching private dependencies'
required: false
SSH_PRIVATE_KEY_2:
SSH_PRIVATE_KEY_2:
description: 'SSH private key for fetching private dependencies'
required: false
SSH_PRIVATE_KEY_3:
Expand All @@ -42,29 +45,28 @@ on:

env:
CARGO_TERM_COLOR: always
ECR_REPOSITORY: ${{ secrets.AWS_ECR_REPOSITORY }}

jobs:
build:
name: build and push
runs-on:
group: init4-runners
name: build ${{ matrix.platform }}
runs-on: ${{ matrix.runner }}
environment: ${{ inputs.environment }}
permissions:
packages: write
contents: read
id-token: write
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-latest
platform: linux/amd64
platform-pair: linux-amd64
- runner: ubuntu-24.04-arm
platform: linux/arm64
platform-pair: linux-arm64
steps:
- uses: actions/checkout@v6
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
- name: Cache Docker layers
uses: actions/cache@v5
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v6
with:
Expand All @@ -74,17 +76,16 @@ jobs:
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2
- name: Setup ssh-agent
id: ssh-agent
if: ${{ inputs.requires-private-deps == true }}
uses: webfactory/ssh-agent@v0.10.0
with:
ssh-private-key: |
${{ secrets.SSH_PRIVATE_KEY }}
${{ secrets.SSH_PRIVATE_KEY }}
${{ secrets.SSH_PRIVATE_KEY_2 }}
${{ secrets.SSH_PRIVATE_KEY_3 }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Setup Docker Metadata
- name: Docker metadata
id: meta
uses: docker/metadata-action@v6
with:
Expand All @@ -94,40 +95,95 @@ jobs:
type=raw,value=latest,enable=true
type=semver,pattern={{version}}
type=sha,format=short
- name: Build and push
- name: Build and push by digest
if: ${{ inputs.requires-private-deps == false }}
id: build
uses: docker/build-push-action@v7
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/arm64
context: .
file: ${{ inputs.dockerfile-path }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max
env:
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ secrets.aws-ecr-repository }}/${{ inputs.rust-binary-name }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=gha,scope=ecr-${{ inputs.rust-binary-name }}-${{ matrix.platform-pair }}
cache-to: type=gha,mode=max,scope=ecr-${{ inputs.rust-binary-name }}-${{ matrix.platform-pair }}
env:
CARGO_NET_GIT_FETCH_WITH_CLI: true
- name: Build and push
- name: Build and push by digest (with ssh)
if: ${{ inputs.requires-private-deps == true }}
id: build-ssh
uses: docker/build-push-action@v7
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/arm64
context: .
file: ${{ inputs.dockerfile-path }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new,mode=max
ssh: |
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ secrets.aws-ecr-repository }}/${{ inputs.rust-binary-name }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=gha,scope=ecr-${{ inputs.rust-binary-name }}-${{ matrix.platform-pair }}
cache-to: type=gha,mode=max,scope=ecr-${{ inputs.rust-binary-name }}-${{ matrix.platform-pair }}
ssh: |
default=${{ env.SSH_AUTH_SOCK }}
env:
env:
CARGO_NET_GIT_FETCH_WITH_CLI: true
- # Temp fix
# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896
- name: Move cache
- name: Export digest
run: |
digest="${{ steps.build.outputs.digest || steps.build-ssh.outputs.digest }}"
if [ -z "$digest" ]; then
echo "::error::no digest produced by build step"
exit 1
fi
mkdir -p /tmp/digests
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v6
with:
name: ecr-digests-${{ matrix.platform-pair }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1

merge:
name: merge manifest
runs-on: ubuntu-latest
needs: build
environment: ${{ inputs.environment }}
permissions:
packages: write
contents: read
id-token: write
steps:
- name: Download digests
uses: actions/download-artifact@v7
with:
path: /tmp/digests
pattern: ecr-digests-*
merge-multiple: true
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v6
with:
role-to-assume: ${{ secrets.aws-ecr-deployer-role-arn }}
aws-region: us-east-1
- name: Login to Amazon ECR
uses: aws-actions/amazon-ecr-login@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Docker metadata
id: meta
uses: docker/metadata-action@v6
with:
images: ${{ secrets.aws-ecr-repository }}/${{ inputs.rust-binary-name }}
tags: |
type=ref,event=branch
type=raw,value=latest,enable=true
type=semver,pattern={{version}}
type=sha,format=short
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create \
$(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ secrets.aws-ecr-repository }}/${{ inputs.rust-binary-name }}@sha256:%s ' *)
- name: Inspect image
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
docker buildx imagetools inspect \
${{ secrets.aws-ecr-repository }}/${{ inputs.rust-binary-name }}:${{ steps.meta.outputs.version }}
130 changes: 97 additions & 33 deletions .github/workflows/ghcr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,44 +1,104 @@
#
# Builds and pushes a multi-arch (linux/amd64 + linux/arm64) Docker image to
# GHCR. Each arch is built natively on a matching GitHub-hosted runner and
# pushed by digest; a final job assembles the manifest list under the real
# tags (semver, branch, pr, sha, latest).

name: Create and publish a Docker image

# Configures this workflow to be called by other workflows using the `workflow_call` trigger.
on:
workflow_call:

# Defines two custom environment variables for the workflow. These are used for the Container registry domain, and a name for the Docker image that this workflow builds.
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

# There is a single job in this workflow. It's configured to run on the latest available version of Ubuntu.
jobs:
build-and-push-image:
runs-on:
group: init4-runners
strategy:
matrix:
platform:
- linux/amd64
- linux/arm64
# Sets the permissions granted to the `GITHUB_TOKEN` for the actions in this job.
build:
name: build ${{ matrix.platform }}
runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write
attestations: write
id-token: write
#
strategy:
fail-fast: false
matrix:
include:
- runner: ubuntu-latest
platform: linux/amd64
platform-pair: linux-amd64
- runner: ubuntu-24.04-arm
platform: linux/arm64
platform-pair: linux-arm64
steps:
- name: Checkout repository
uses: actions/checkout@v6
# Uses the `docker/login-action` action to log in to the Container registry registry using the account and password that will publish the packages. Once published, the packages are scoped to the account defined here.
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Docker metadata
id: meta
uses: docker/metadata-action@v6
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
- name: Build and push by digest
id: build
uses: docker/build-push-action@v7
with:
context: .
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=gha,scope=ghcr-${{ matrix.platform-pair }}
cache-to: type=gha,mode=max,scope=ghcr-${{ matrix.platform-pair }}
- name: Export digest
run: |
digest="${{ steps.build.outputs.digest }}"
if [ -z "$digest" ]; then
echo "::error::no digest produced by build step"
exit 1
fi
mkdir -p /tmp/digests
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v6
with:
name: ghcr-digests-${{ matrix.platform-pair }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1

merge:
name: merge manifest
runs-on: ubuntu-latest
needs: build
permissions:
contents: read
packages: write
attestations: write
id-token: write
steps:
- name: Download digests
uses: actions/download-artifact@v7
with:
path: /tmp/digests
pattern: ghcr-digests-*
merge-multiple: true
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4
- name: Log in to the Container registry
uses: docker/login-action@v4
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# This step uses [docker/metadata-action](https://github.com/docker/metadata-action#about) to extract tags and labels that will be applied to the specified image. The `id` "meta" allows the output of this step to be referenced in a subsequent step. The `images` value provides the base name for the tags and labels.
- name: Extract metadata (tags, labels) for Docker
- name: Docker metadata
id: meta
uses: docker/metadata-action@v6
with:
Expand All @@ -50,23 +110,27 @@ jobs:
type=semver,pattern={{major}}.{{minor}}
type=sha
type=raw,value=latest,enable=true
# This step uses the `docker/build-push-action` action to build the image, based on your repository's `Dockerfile`. If the build succeeds, it pushes the image to GitHub Packages.
# It uses the `context` parameter to define the build's context as the set of files located in the specified path. For more information, see [Usage](https://github.com/docker/build-push-action#usage) in the README of the `docker/build-push-action` repository.
# It uses the `tags` and `labels` parameters to tag and label the image with the output from the "meta" step.
- name: Build and push Docker image
id: push
uses: docker/build-push-action@v7
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

# This step generates an artifact attestation for the image, which is an unforgeable statement about where and how it was built. It increases supply chain security for people who consume the image. For more information, see [Using artifact attestations to establish provenance for builds](/actions/security-guides/using-artifact-attestations-to-establish-provenance-for-builds).
- name: Create manifest list and push
id: manifest
working-directory: /tmp/digests
run: |
docker buildx imagetools create \
$(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@sha256:%s ' *)
# Resolve the manifest digest of the freshly pushed multi-arch image
# (needed for build-provenance attestation).
version_tag="${{ steps.meta.outputs.version }}"
manifest_digest=$(docker buildx imagetools inspect \
"${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${version_tag}" \
--format '{{ json .Manifest }}' | jq -r '.digest')
echo "digest=${manifest_digest}" >> "$GITHUB_OUTPUT"
- name: Inspect image
run: |
docker buildx imagetools inspect \
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.meta.outputs.version }}
- name: Generate artifact attestation
uses: actions/attest-build-provenance@v4
with:
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME}}
subject-digest: ${{ steps.push.outputs.digest }}
subject-name: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
subject-digest: ${{ steps.manifest.outputs.digest }}
push-to-registry: true