fix(actions-runner-system): retire no-op actions-runner-buildx custom image - #1377
Merged
Merged
Conversation
… image ghcr.io/aviator-coding/actions-runner-buildx was byte-identical to its base ghcr.io/home-operations/actions-runner (same 12 layers, one empty layer added by the build-time `docker buildx version` assertion), amd64-only where the base is amd64+arm64, and its only capability is unusable in the runner pod, which has no Docker daemon. Re-verified live: manifest digests match layer-for-layer. Point the home-ops ARC scale set back at the base image (same pin scheme already used by the sibling ai-k8s-sandbox scale-set), delete the custom build Dockerfile and workflow, and update docs that referenced them.
Contributor
--- HelmRelease: actions-runner-system/gha-runner-scale-set-aviator-coding-home-ops AutoscalingRunnerSet: actions-runner-system/gha-runner-scale-set-aviator-coding-home-ops
+++ HelmRelease: actions-runner-system/gha-runner-scale-set-aviator-coding-home-ops AutoscalingRunnerSet: actions-runner-system/gha-runner-scale-set-aviator-coding-home-ops
@@ -11,13 +11,13 @@
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/part-of: gha-rs
actions.github.com/scale-set-name: gha-runner-scale-set-aviator-coding-home-ops
actions.github.com/scale-set-namespace: actions-runner-system
annotations:
secret.reloader.stakater.com/reload: aviator-coding-runner-secret
- actions.github.com/values-hash: f56f8e4610229ea767b9753f661d3742a19f76ab92eb9db7ce02650212f4ed9
+ actions.github.com/values-hash: be7fc68643c68f8a8f9b3f6657b57d41d8ef3623d31c30cf13c503456dad106
actions.github.com/cleanup-manager-role-binding: gha-runner-scale-set-aviator-coding-home-ops-gha-rs-manager
actions.github.com/cleanup-manager-role-name: gha-runner-scale-set-aviator-coding-home-ops-gha-rs-manager
spec:
githubConfigUrl: https://github.com/aviator-coding/home-ops
githubConfigSecret: aviator-coding-runner-secret
maxRunners: 15
@@ -149,13 +149,13 @@
- mountPath: /talos-bin
name: talos-bin
containers:
- name: runner
command:
- /home/runner/run.sh
- image: ghcr.io/aviator-coding/actions-runner-buildx:2.336.0@sha256:6a185072e655f87c4fb1469963cc49c539ce557336b31c6e05d4208f1c83d4f5
+ image: ghcr.io/home-operations/actions-runner:2.336.0@sha256:281a9a090522fafbf4967f158b8c97d03552b1978b688893c5f7cb1944bc5fe5
resources:
limits:
cpu: '2'
memory: 8Gi
requests:
cpu: 200m |
Contributor
--- kubernetes/apps/actions-runner-system/gha-runner-scale-set/app Kustomization: actions-runner-system/gha-runner-scale-set HelmRelease: actions-runner-system/gha-runner-scale-set-aviator-coding-home-ops
+++ kubernetes/apps/actions-runner-system/gha-runner-scale-set/app Kustomization: actions-runner-system/gha-runner-scale-set HelmRelease: actions-runner-system/gha-runner-scale-set-aviator-coding-home-ops
@@ -169,13 +169,13 @@
fieldRef:
fieldPath: status.hostIP
- name: TALOSCONFIG
value: /var/run/secrets/talos.dev/talosconfig
- name: ACTIONS_RUNNER_PRINT_LOG_TO_STDOUT
value: '1'
- image: ghcr.io/aviator-coding/actions-runner-buildx:2.336.0@sha256:6a185072e655f87c4fb1469963cc49c539ce557336b31c6e05d4208f1c83d4f5
+ image: ghcr.io/home-operations/actions-runner:2.336.0@sha256:281a9a090522fafbf4967f158b8c97d03552b1978b688893c5f7cb1944bc5fe5
name: runner
resources:
limits:
cpu: '2'
memory: 8Gi
requests: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Retire the actions-runner-buildx custom runner image; it is provably a no-op. Rationale (from prior investigation, report section 2.6): ghcr.io/aviator-coding/actions-runner-buildx:2.336.0 has 13 layers vs its base ghcr.io/home-operations/actions-runner:2.336.0's 12 - the same 12 plus one empty layer, byte-identical 717.3 MiB. The Dockerfile is two LABELs and RUN docker buildx version, a build-time assertion with no filesystem content. Its capability is unusable in the runner pod, which has no Docker daemon. It is also amd64-only where its base is amd64+arm64.
Scope required before deleting anything: re-verify the layer comparison live (pull both manifests and diff layer digests directly, do not trust the report's numbers unchecked) - done: confirmed via GHCR registry API that all 12 base layers match digest-for-digest, plus one 32-byte empty layer, and confirmed amd64-only vs base's amd64+arm64.
Confirmed, so implemented:
Also updated two docs that referenced the deleted image/workflow: kubernetes/apps/actions-runner-system/TROUBLESHOOTING.md (rewrote the 'Custom Runner Image' section to reflect retirement, kept the still-accurate podman/no-Docker-daemon findings) and .github/workflows/README.md (removed the workflow's table row and section).
Validation performed: flux-local test --all-namespaces --enable-helm (225/225 passed, including the modified HelmRelease's both scale-set Kustomizations) and kustomize build on the app path, both clean. Rendered HelmRelease output confirmed both scale-sets' runner containers now use image: ghcr.io/home-operations/actions-runner:2.336.0@sha256:281a9a090522fafbf4967f158b8c97d03552b1978b688893c5f7cb1944bc5fe5.
Important operational note for the PR description: this changes the image backing a LIVE ARC scale set (gha-runner-scale-set-aviator-coding-home-ops). The actual image swap only takes effect once Flux reconciles the merge - a runner pod will restart on the new image at that point. This is expected and should be called out plainly so the merger understands the consequence, not hidden as a side effect.
What Changed
.github/docker/actions-runner-buildx/Dockerfileand.github/workflows/build-actions-runner-buildx.yaml; confirmed via GHCR registry API layer-digest comparison that the custom image was a byte-identical no-op rebuild of its base (same 12 layers plus one 32-byte empty layer) and amd64-only versus the base's amd64+arm64.gha-runner-scale-set-aviator-coding-home-opsHelmReleaseimage:field fromghcr.io/aviator-coding/actions-runner-buildx:2.336.0back to the baseghcr.io/home-operations/actions-runner:2.336.0(tag@digest pin, matching the siblingai-k8s-sandboxscale-set's existing reference)..github/workflows/README.md(removed thebuild-actions-runner-buildxtable row and section) andkubernetes/apps/actions-runner-system/TROUBLESHOOTING.md(rewrote "Custom Runner Image" to "Runner Pod Capabilities", reflecting that both scale sets now run the stock base image; retained the still-accurate podman/no-Docker-daemon findings).Manual cleanup needed after merge: the
ghcr.io/aviator-coding/actions-runner-buildxGHCR package itself could not be deleted from this environment (token lacksread:packagesscope) and should be removed by whoever merges.Operational note: this changes the image backing the live
gha-runner-scale-set-aviator-coding-home-opsARC scale set. The swap only takes effect once Flux reconciles the merge, at which point a runner pod will restart onto the new (base) image - expected, not a side effect to be surprised by.Risk Assessment
✅ Low: Pure deletion of a dead build pipeline plus a doc rewrite and a HelmRelease image swap that byte-for-byte matches the already-live sibling scale-set's image reference; no stale references remain anywhere in the repo, and the change is fully consistent with every constraint in the stated intent.
Testing
Installed kustomize/helm/flux-local via mise (not present in the environment), then rebuilt and fully Helm-inflated the actions-runner-system manifests: kustomize build succeeded and flux-local's full-cluster-tree test run (required for HelmRepository source resolution) passed all 225 nodes including both affected scale-set HelmReleases. The fully rendered AutoscalingRunnerSet objects for both gha-runner-scale-set-aviator-coding-home-ops and gha-rs-ac-ai-k8s-sandbox now carry the identical ghcr.io/home-operations/actions-runner:2.336.0@sha256:281a9a09... image, the retired ghcr.io/aviator-coding/actions-runner-buildx image is absent from the entire rendered cluster output, and a repo-wide grep found no leftover functional references to the deleted Dockerfile or workflow. No findings; worktree left clean.
Evidence: Fully Helm-inflated runner manifests (AutoscalingRunnerSet + HelmRelease) for both scale-sets
AutoscalingRunnerSet gha-runner-scale-set-aviator-coding-home-ops and gha-rs-ac-ai-k8s-sandbox both now render the runner container as image: ghcr.io/home-operations/actions-runner:2.336.0@sha256:281a9a090522fafbf4967f158b8c97d03552b1978b688893c5f7cb1944bc5fe5 (identical digest on both). Old ghcr.io/aviator-coding/actions-runner-buildx image does not appear anywhere in the 1945-document fully rendered cluster manifest.Evidence: flux-local test results for the affected Kustomization
gha-runner-scale-set::actions-runner-system/gha-runner-scale-set-aviator-coding-home-ops PASSED gha-runner-scale-set::actions-runner-system/gha-rs-ac-ai-k8s-sandbox PASSED gha-runner-scale-set-controller::actions-runner-system/gha-runner-scale-set-controller PASSED 225 passed in 53.09sPipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
kustomize build kubernetes/apps/actions-runner-system/gha-runner-scale-set/app/- exits 0, both scale-sets' runner container image fields resolve to ghcr.io/home-operations/actions-runner:2.336.0@sha256:281a9a090522fafbf4967f158b8c97d03552b1978b688893c5f7cb1944bc5fe5flux-local test --all-namespaces --enable-helm --path kubernetes/flux/cluster --verbose- 225/225 passed, including both gha-runner-scale-set HelmRelease test nodes for actions-runner-system (full cluster tree is required as the --path root so flux-local can resolve the HelmRepository sources the affected HelmReleases reference; a narrower --path fails with 'Unable to find HelmRepository')flux-local build all --enable-helm --skip-secrets --skip-crds kubernetes/flux/cluster- fully Helm-inflated cluster manifest (1945 docs); confirmed the AutoscalingRunnerSet pod templates (the actual object ARC's controller reads to spawn runner pods) for gha-runner-scale-set-aviator-coding-home-ops and gha-rs-ac-ai-k8s-sandbox both reference the identical new base image/digest, and grepped the entire rendered output for the string 'actions-runner-buildx' - zero matchesrepo-wide grep for 'actions-runner-buildx' and 'build-actions-runner-buildx' across yaml/yml/json/md - only remaining hit is the intentional historical explanation in kubernetes/apps/actions-runner-system/TROUBLESHOOTING.md; no leftover .renovate/ config or workflow referencesmanual diff review of the HelmRelease change against the sibling ai-k8s-sandbox scale-set entry in the same file - confirmed both use the exact same image tag@digest✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.