fix(ci): push tutorial images from buildx instead of re-pushing a stale local tag - #478
Merged
Merged
Conversation
…le local tag The publish path built a multi-platform image without --push, then a separate 'docker push' step shipped ':latest'. But a multi-arch buildx build can't be loaded into the local Docker image store, so no fresh local ':latest' exists; the preceding validation step's 'docker run' pulls the *existing* (stale) ':latest' from ghcr, and 'docker push' then re-pushes that same stale image. Result: ':latest' has been frozen at a Dec-2025 build, re-pushed unchanged on every run, which is why the 020_state_machine agent's mcp<2 pin never reached the published image and the scale-agentex integration test kept failing. Fix: on the publish path, pass --push to 'agentex agents build' so buildx sends the freshly-built multi-arch image straight to the registry, and drop the now-redundant separate 'docker push' step. Validation still runs and now exercises the freshly-pushed image. Also revert the CLI build cache default back to True (keep the --cache/--no-cache flag): the staleness was never a layer-cache problem, so forcing --no-cache globally only slowed builds without fixing anything. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address the validate-after-publish gap: build+push to an immutable candidate tag (the commit SHA), validate that exact pushed image, then promote :latest onto it via 'docker buildx imagetools create' (a registry-side manifest copy, no rebuild). If validation fails, :latest is left on the last known-good build and only the SHA tag is dirty -- no unvalidated image ever lands on :latest. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
erichwoo-scale
approved these changes
Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
020_state_machine:latest(and other tutorial-agent images) have been frozen at a December 2025 build and re-published unchanged on every run — which is why the mergedmcp<2pin never reached the image andscale-agentex's020-state-machineintegration test keeps failing on themcp 2.0.0McpError→MCPErrorcrash.Root cause — the publish path re-pushes a stale image, not the one it just built:
A multi-platform buildx build cannot be loaded into the local Docker image store, so the build step never creates a local
:latest. The validation step'sdocker run "$FULL_IMAGE"then pulls the existing (stale):latestfrom the registry, and the separatedocker pushre-pushes that same stale image back. The freshly-built image is silently discarded — so:latestis self-perpetuating and stuck at its December digest.This was verified end-to-end:
docker buildfrom the pinned source produces a fresh image with the pin.docker push) re-pushed the identical December digest (1f90…,Created 2025-12-30, no pin) even with--no-cacheand SDK 0.22.0.Fix
--pushtoagentex agents buildso buildx sends the freshly-built multi-arch image straight to the registry.docker pushstep. Validation still runs and now exercises the freshly-pushed image.True(keeping the--cache/--no-cacheflag). The staleness was never a layer-cache problem, so forcing--no-cacheglobally only slowed builds.Effect
The next publish (push to
mainunderexamples/tutorials/**, or arebuild_alldispatch) will push the freshly-built, pinned020_state_machine:latest, turning thescale-agentexintegration check green — and fixes the same silent-stale-publish for every tutorial agent.Note on the earlier PR
This supersedes the root-cause theory behind the earlier no-cache change (#476):
--no-cachewas confirmed to run in CI yet the image stayed stale, which is what led to finding the real build/push bug here.🤖 Generated with Claude Code
Greptile Summary
This PR fixes a long-standing silent publish bug where multi-platform buildx builds were being silently discarded and a stale
:latestre-pushed in its place. The fix changes the publish path to push directly from buildx to an immutable SHA-tagged candidate, validate that exact image, then promote it to:latestviadocker buildx imagetools create— keeping:latestclean until validation passes. Thecachedefault is also reverted toTruesince caching was never the root cause of staleness.--pushso buildx sends the fresh multi-arch manifest straight to the SHA tag; a separate "promote" step then points:latestat that validated digest withdocker buildx imagetools create, preserving the multi-arch manifest list without rebuilding.cache: bool = Truein bothagents.pyandagent_handlers.py;--no-cacheremains available but is no longer the default since layer-cache staleness was not the root cause.Confidence Score: 5/5
Safe to merge — the change correctly routes the publish path through buildx --push, validates the live SHA-tagged image, and only then promotes :latest.
The three changed files are tightly scoped: the workflow correctly pushes from buildx to an immutable SHA tag, validates that exact image, and promotes :latest via a registry-side manifest copy. The Python cache-default reversal is a one-line change with no side effects. The only finding is a dead env variable that is exported but never read, which has no functional impact.
Files Needing Attention: No files require special attention beyond a quick read of the SHOULD_PUSH vs PROMOTE_LATEST interplay in the workflow.
Important Files Changed
Reviews (2): Last reviewed commit: "validate before promoting :latest (candi..." | Re-trigger Greptile