feat: native multi-platform builds via runner matrix (no QEMU emulation)#435
feat: native multi-platform builds via runner matrix (no QEMU emulation)#435tgenov wants to merge 24 commits intodevcontainers:mainfrom
Conversation
Add platformTag and mergeTag inputs to support building on native ARM runners in a matrix strategy, then merging per-platform images into a multi-arch manifest via docker buildx imagetools create. This avoids slow QEMU emulation for multi-platform builds by allowing each matrix job to build natively for its own platform.
The devcontainer CLI rejects --platform without --output. For native single-platform builds (platformTag set), use type=docker to load the image into the local daemon for subsequent docker push.
The devcontainer CLI rejects --platform for docker-compose-based devcontainers. When platformTag is set, the runner is already the correct native architecture, so --platform is unnecessary.
- Mirror platformTag/mergeTag logic in azdo-task (task.json inputs, runMain/runPost in main.ts, createManifest wrapper in docker.ts) - Add unit tests for createManifest in common/__tests__/docker.test.ts - Update docs/github-action.md and docs/azure-devops-task.md input tables - Add native multi-platform builds section to docs/multi-platform-builds.md with examples for both GitHub Actions and Azure DevOps Pipelines
|
@microsoft-github-policy-service agree |
There was a problem hiding this comment.
Pull request overview
Adds a native (non-QEMU) multi-platform build strategy by splitting per-architecture builds into matrix jobs that push platform-suffixed tags, followed by a final manifest-merge job that publishes a multi-arch tag.
Changes:
- Introduces
platformTag(per-platform tagging/push) andmergeTag(manifest merge) flows for both GitHub Actions and Azure DevOps. - Adds a
createManifestimplementation usingdocker buildx imagetools create, plus unit tests. - Updates docs and action/task metadata to document and expose the new inputs.
Reviewed changes
Copilot reviewed 11 out of 15 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
action.yml |
Exposes new platformTag / mergeTag inputs for the GitHub Action. |
github-action/src/main.ts |
Implements platform-suffixed tagging, merge-only early return, and post-step manifest creation/push behavior. |
github-action/src/docker.ts |
Adds a GitHub Action wrapper for createManifest. |
github-action/dist/sourcemap-register.js |
Updated build artifact. |
github-action/dist/licenses.txt |
Updated build artifact. |
common/src/docker.ts |
Adds shared createManifest implementation via docker buildx imagetools create. |
common/__tests__/docker.test.ts |
Adds unit tests for createManifest. |
azdo-task/DevcontainersCi/task.json |
Exposes new platformTag / mergeTag inputs for the AzDO task. |
azdo-task/DevcontainersCi/src/main.ts |
Mirrors the platform-suffixed tagging and manifest merge flow in AzDO. |
azdo-task/DevcontainersCi/src/docker.ts |
Adds an AzDO wrapper for createManifest. |
docs/multi-platform-builds.md |
Documents the new native matrix strategy and examples. |
docs/github-action.md |
Documents new GitHub Action inputs. |
docs/azure-devops-task.md |
Documents new AzDO task inputs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "required": false | ||
| }, | ||
| { | ||
| "name": "mergeTag", |
There was a problem hiding this comment.
Could you clarify what naming you'd prefer? Happy to rename.
|
@tgenov Thanks for kickstarting this work, it would be very useful to have native builds. I'll be happy to re-review once the above comments are addressed. |
Move the mergeTag block after the push option filtering logic in both GitHub Action and Azure DevOps implementations. Previously, mergeTag would bypass all push gating and could publish manifests on PRs or when push was set to 'never'.
…orm.ts Extract buildImageNames and mergeMultiPlatformImages helpers to eliminate duplicated logic between GitHub Action and Azure DevOps implementations.
… platformTag - Fail early with a clear error if mergeTag is set without push: always, preventing silent no-ops when default push filtering skips the manifest. - Fail early if both mergeTag and platformTag are set on the same step. - Simplify redundant return logic in mergeTag runPost blocks. - Add push: always to manifest job examples in docs.
- Remove duplicate "Creating multi-arch manifest" log from AzDO wrapper (mergeMultiPlatformImages already logs this). - Shorten GH Action group header to avoid repeating the log message. - Update General Notes to reflect GitHub's hosted ARM runners and link to the native matrix strategy. - Add missing Docker login and buildx setup steps to the AzDO native multi-platform example.
|
Thanks for the review. All (but one) comments addressed. Beyond the requested changes, I also added validation that mergeTag and platformTag cannot be set together, and updated the docs General Notes section to reflect GitHub's hosted ARM runners. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 17 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ageName - Move isDockerBuildXInstalled() before the mergeTag early return so missing buildx fails immediately with a clear error instead of at runtime in the post step. - Validate imageName is set when mergeTag is used. - Guard buildImageNames calls so empty imageName produces [] instead of invalid image refs like ':latest'.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 17 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
…xample - Trim platformTag input and reject values containing whitespace or commas with a clear error message. - Remove platform from the native matrix example since it is ignored when platformTag is set.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 13 out of 17 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| | noCache | false | Builds the image with `--no-cache` (takes precedence over `cacheFrom`) | | ||
| | cacheTo | false | Specify the image to cache the built image to | | ||
| | platform | false | Platforms for which the image should be built. If omitted, defaults to the platform of the GitHub Actions Runner. Multiple platforms should be comma separated. | | ||
| | platform | false | Platforms for which the image should be built. If omitted, defaults to the platform of the GitHub Actions Runner. Multiple platforms should be comma separated. Ignored when `platformTag` is set (matrix mode). | |
There was a problem hiding this comment.
The platform input description says it “defaults to the platform of the GitHub Actions Runner”, but this document is for the Azure DevOps task. This should reference the Azure Pipelines agent (or simply “the build agent”) to avoid confusing users.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Problem
The current multi-platform build support (
platforminput) relies on QEMU emulation through Docker buildx. This works but has significant drawbacks:GitHub Actions and Azure DevOps both offer native ARM runners (
ubuntu-24.04-arm,ARM64pool), making emulation unnecessary if the action supports a matrix-based workflow where each platform builds natively on its own runner.Proposed Solution
Add two new inputs that enable a split build/merge workflow:
platformTag(per-platform build phase)Used in matrix jobs. Each job runs on a native runner for its target platform and sets
platformTagto a suffix (e.g.,linux-amd64,linux-arm64). The action:-{platformTag}to each image tag (e.g.,myimage:latest-linux-amd64)--platformflag needed)mergeTag(manifest merge phase)Used in a final job after all matrix builds complete. The action:
docker buildx imagetools createto combine the per-platform images into a multi-arch manifest under the original tag (e.g.,myimage:latest)Example Workflow (GitHub Actions)
Scope of Work
Per CONTRIBUTING.md, changes must maintain feature parity between GitHub Actions and Azure DevOps, include tests, and compile via
./scripts/build-local.sh.GitHub Action
platformTagandmergeTaginputs toaction.ymlrunMain()ingithub-action/src/main.ts:mergeTagis set (save state for post step)--platformand--outputflags whenplatformTagis set (native runner builds correct arch)-{platformTag}suffix to image tagsplatformTagis setrunPost()ingithub-action/src/main.ts:docker buildx imagetools createwhenmergeTagstate is presentdocker pushwith platform-suffixed tags whenplatformTagstate is presentcreateManifestwrapper togithub-action/src/docker.tsAzure DevOps Task
platformTagandmergeTaginputs toazdo-task/DevcontainersCi/task.jsonrunMain()/runPost()logic inazdo-task/DevcontainersCi/src/main.tscreateManifestwrapper toazdo-task/DevcontainersCi/src/docker.tsCommon
createManifestfunction tocommon/src/docker.tsusingdocker buildx imagetools createTests
createManifestincommon/__tests__/docker.test.tsplatformTag/mergeTaginput handlingDocumentation
docs/github-action.mdwithplatformTagandmergeTaginput descriptionsdocs/azure-devops-task.mdwith equivalent input descriptionsdocs/multi-platform-builds.mdalongside the existing QEMU approachDesign Notes
Why two inputs instead of extending
platform?The existing
platforminput (e.g.,linux/amd64,linux/arm64) triggers a single QEMU-emulated build. The new inputs are orthogonal —platformTagtags the output of a single-platform native build, andmergeTagmerges tagged outputs. This avoids breaking the existing QEMU-based workflow.Why
docker buildx imagetools createinstead ofdocker manifest?imagetools createworks with remote registry images without pulling them locally, and supports OCI image indexes natively. It is the recommended approach for combining multi-platform images that are already pushed to a registry.Runner polymorphism
This design makes the action polymorphic across runner types. Without the new flags, the action behaves as before — a single QEMU-emulated build on one runner. With
platformTagandmergeTag, the same action drives a matrix of native runners where each builds its own architecture natively, then a final job merges the results.platformonlyplatform+platformTag+mergeTagBackwards compatibility
When neither
platformTagnormergeTagis set, the action behaves exactly as before. The existingplatforminput with QEMU emulation continues to work unchanged.Open Questions
Polymorphism vs. separation of concerns
The current design overloads a single action with two distinct multi-platform strategies (QEMU vs. native runners) controlled by input flags. An alternative would be to ship the native runner workflow as a separate action (e.g.,
devcontainers/ci/native-multiplatform) with a dedicated interface, keeping the original action focused on single-runner builds.Polymorphism keeps the surface area small and avoids forcing users to switch actions when migrating from QEMU to native runners. Separation would make each action's contract simpler and avoid the conditional logic around
platformTag/mergeTagthat touches bothrunMainandrunPost. Which trade-off does the project prefer?Note
We are already using the fork internally with GitHub Actions. The DevOps task implementation follows the same patterns but has not been tested in an AzDO pipeline.