ci: fix Docker layer caching gaps#208
Merged
Merged
Conversation
- deploy/Dockerfile.prod: cache the React yarn install independently of submodule changes (copy only package.json/yarn.lock first). The pipenv sync layer can't be decoupled the same way - several submodules are installed as editable path packages straight from the Pipfile, so pipenv sync requires them checked out first. - Add a Buildx + GHA cache-from/cache-to step to pre-build the test job's dev ckan image (ckan/Dockerfile), which previously had no cross-run layer caching at all and rebuilt apt/node/yarn from scratch on every run.
cooper667
force-pushed
the
ci-improve-caching
branch
from
July 14, 2026 05:32
e42b6ef to
7063252
Compare
Split the build job into build (no push, runs concurrently with test) and push (needs [test, build], only pushes once tests pass). Push reuses the build job's GHA cache, so it's a cache hit rather than a rebuild - the only new cost is uploading + pushing the already-built image. deploy now depends on push instead of build.
…her, giftless) The previous fix only pre-built and cached the ckan service image. docker-compose.yml actually builds six images, and giftless/datapusher do real pip wheel builds from scratch on every run (confirmed in run 29309402751: ~65s and ~44s respectively, dominating the 'Build images and start the dev stack' step). Pre-build all of them with the same Buildx + GHA cache pattern, distinct scope per service.
Uses GitHub Actions' new parallel: step block (June 2026) to run the independent per-service Buildx pre-build steps concurrently against the same builder instead of one after another.
cooper667
enabled auto-merge (squash)
July 14, 2026 07:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cache and parallelise CI building so it's a little quicker