fix(ci): retry docker compose up on transient registry pull failures#24546
Merged
spalladino merged 2 commits intoJul 6, 2026
Merged
Conversation
ecbdb98 to
112db05
Compare
spalladino
approved these changes
Jul 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
e2e_ha_full.parallel.test.ts("should produce blocks with HA coordination and attestations") failed in CI on an unrelated PR (#24492) before the test even started: Docker Hub returned a transient 502 whiledocker compose uppulledpostgres:16-alpinefor the HA compose stack.CI log: http://ci.aztec-labs.com/cac25b9251c99496
This can hit any compose-based test (
run_compose_testis shared by the e2e compose/web3signer/ha suites and docs examples) whenever a required image is not in the local Docker cache and the registry blips.Fix
Wrap the
docker compose up -d --force-recreateinci3/run_compose_testwith the existingci3/retry -p <regex>helper, retrying only failures that match transient registry/network errors (5xx, HEAD-request failures, TLS/connection timeouts, rate limiting). This reuses the same pattern-gated retry convention already used for network flakes inl1-contracts/bootstrap.sh,barretenberg/sol/bootstrap.sh, andnoir-projects/aztec-nr/bootstrap.sh, so genuine failures (e.g. an image tag that doesn't exist) still fail immediately without retrying.Retrying
up -d --force-recreateis idempotent: a retry recreates any containers from a partially failed attempt.Verified the exact error message from the CI log matches the retry pattern (retried, then succeeds), while a
manifest not foundfailure exits immediately without retries.No tracked issue exists for this flake.
Created by claudebox · group:
slackbot