Skip to content

Commit 03ba406

Browse files
akarrasclaude
andauthored
ci: stop the Actions cache quota from killing every main Docker build (#981)
Every push to main since 2026-07-24 failed the Docker workflow with `failed to solve: not_found`, always at `#40 exporting to GitHub Actions Cache`. PRs passed because they set `cache-to: ''` and only read. Root cause is the shared 10GB Actions cache quota, not the Dockerfile. rust.yml used a plain `actions/cache`, and Actions caches are scoped per-ref, so every PR wrote its own ~1.2GB copy of the same key. Six copies held 7.22GB of the 10GB budget; the repo sat pinned at 10.44GB. That left under 3GB for buildkit, so a mode=max export of the cargo-chef build evicted its own already-uploaded blobs and 404'd finalizing the manifest. The recoverable `#28 CACHED` -> `#28 ERROR: not_found` earlier in the same logs is the same eviction hitting the import side. #977's serialization addressed docker-vs-docker contention, but the competitor was rust.yml — run 30175930394 was alone in flight and still failed. The concurrency block stays; it's still worth not running three 90-minute builds at once. - rust.yml: split cache into restore/save, save only on main pushes. PRs stay warm via restore-keys against main's copy. Frees ~6GB. - docker-publish.yml: move the buildkit cache to a GHCR registry cache, which is outside the Actions quota, so the two workflows stop competing for it at all. Login is ungated so PRs can read it. - docker-publish.yml: drop setup-qemu-action. We build linux/amd64 on an amd64 runner, so it did nothing but park a 32MB binfmt cache per ref (13 of them, 419MB). Also pruned the five merged-PR cargo caches and the stale binfmt entries out of band: the repo is now at 4.0GB across 18 entries. Diff is YAML-only, so clippy can't be affected; `cargo fmt --all -- --check` passes. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 630a9e3 commit 03ba406

2 files changed

Lines changed: 37 additions & 8 deletions

File tree

.github/workflows/docker-publish.yml

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -52,14 +52,15 @@ jobs:
5252
with:
5353
submodules: recursive
5454

55-
- name: Set up QEMU
56-
uses: docker/setup-qemu-action@v3
55+
# No QEMU: we build linux/amd64 only, on an amd64 runner. It bought
56+
# us nothing and parked a 32MB binfmt cache on every ref.
5757

5858
- name: Set up Docker Buildx
5959
uses: docker/setup-buildx-action@v3
6060

61+
# PRs log in too — they don't push, but they read the registry
62+
# build cache below, which needs auth if the package is private.
6163
- name: Log in to registry
62-
if: github.event_name != 'pull_request'
6364
uses: docker/login-action@v3
6465
with:
6566
registry: ${{ env.REGISTRY }}
@@ -81,10 +82,16 @@ jobs:
8182
push: ${{ github.event_name != 'pull_request' }}
8283
tags: ${{ steps.meta.outputs.tags }}
8384
labels: ${{ steps.meta.outputs.labels }}
84-
cache-from: type=gha
85+
# Registry cache, not type=gha. A mode=max export of this
86+
# cargo-chef build is multiple GB and shares the repo's 10GB
87+
# Actions cache quota with rust.yml; once that filled up,
88+
# GitHub evicted blobs mid-export and every main build died
89+
# with `failed to solve: not_found`. GHCR storage is outside
90+
# that quota, so the two workflows stop competing.
91+
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache
8592
# PRs read main's cache but don't write — avoids thrashing
8693
# the shared cache from short-lived branches.
87-
cache-to: ${{ github.event_name != 'pull_request' && 'type=gha,mode=max' || '' }}
94+
cache-to: ${{ github.event_name != 'pull_request' && format('type=registry,ref={0}/{1}:buildcache,mode=max', env.REGISTRY, env.IMAGE_NAME) || '' }}
8895

8996
- name: Install cosign
9097
if: github.event_name != 'pull_request'
@@ -128,7 +135,7 @@ jobs:
128135
outputs: type=local,dest=./debug
129136
# Reuse the layer cache the main build already populated;
130137
# this step just emits the unstripped binary as a local file.
131-
cache-from: type=gha
138+
cache-from: type=registry,ref=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:buildcache
132139

133140
- name: Upload debug symbols to GlitchTip
134141
if: steps.glitchtip-check.outputs.configured == 'true'

.github/workflows/rust.yml

Lines changed: 24 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,8 +39,15 @@ jobs:
3939
with:
4040
toolchain: nightly-2026-01-06
4141
components: rustfmt, clippy
42-
- name: Cache cargo registry and build artifacts
43-
uses: actions/cache@v4.2.0
42+
# Restore on every run, but only *save* from main. Actions caches
43+
# are scoped per-ref, so a plain actions/cache here wrote a fresh
44+
# ~1.2GB copy of this key for every PR — six of them filled 69% of
45+
# the repo's 10GB quota, and GitHub's LRU then evicted the Docker
46+
# workflow's buildkit blobs mid-export (`failed to solve:
47+
# not_found`). PRs still get a warm cache: restore-keys falls back
48+
# to main's copy.
49+
- name: Restore cargo registry and build artifacts
50+
uses: actions/cache/restore@v4.2.0
4451
with:
4552
path: |
4653
~/.cargo/bin
@@ -79,6 +86,21 @@ jobs:
7986
# it would be good to support --all-features, but xiv-gen is a little unstable.
8087
run: cargo clippy --all-targets -- -D warnings
8188

89+
# Only main writes the cache (see the restore step above). Runs on
90+
# failure too, so a red lint run still leaves the next build warm;
91+
# skipped when the key already exists, hence continue-on-error.
92+
- name: Save cargo registry and build artifacts
93+
if: ${{ !cancelled() && github.event_name == 'push' && github.ref == 'refs/heads/main' }}
94+
continue-on-error: true
95+
uses: actions/cache/save@v4.2.0
96+
with:
97+
path: |
98+
~/.cargo/bin
99+
~/.cargo/registry
100+
~/.cargo/git
101+
target
102+
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
103+
82104
#- name: Run cargo-deny, outdated, udeps, audit, and pants
83105
# run: |
84106
# cargo deny check

0 commit comments

Comments
 (0)