Skip to content

Attempt to cache docker images in CI#13125

Merged
pchickey merged 1 commit intobytecodealliance:mainfrom
alexcrichton:docker-cache
Apr 20, 2026
Merged

Attempt to cache docker images in CI#13125
pchickey merged 1 commit intobytecodealliance:mainfrom
alexcrichton:docker-cache

Conversation

@alexcrichton
Copy link
Copy Markdown
Member

Linux package servers have been particularly flaky today so I've re-investigated how to maybe cache everything. The idea here is that we use the "officially recommended" way of caching builds of docker images on github actions, notably using the github actions cache. The way this is configured is that some extra steps happen on github which populate the github actions cache and local docker daemon such that when the actual build happens, totally independently of these github actions steps, it just so happens to get cache hits. This is done to ensure that if we make a mistake here it doesn't result in stale builds, just slower builds.

The overall hope here is that by using the github actions cache for docker images we can hit package installation less than we currently do. The hope is that all the images fit within the github actions cache. I think they do but it's pretty noisy and each build generates new entries so I'm not entirely sure what's happening. Local testing shows that reruns do indeed proceed faster and don't hit apt-get for example, though.

This doesn't entirely insulate us from issues with apt-get because runs will still use apt-get externally from docker, such as just installing a few extra packages. I don't know how to make those more robust, but hopefully we can at least reduce some flakiness by caching some things.

@alexcrichton alexcrichton requested a review from a team as a code owner April 16, 2026 19:53
@alexcrichton alexcrichton requested review from fitzgen and removed request for a team April 16, 2026 19:53
Linux package servers have been particularly flaky today so I've
re-investigated how to maybe cache everything. The idea here is that we
use the "officially recommended" way of caching builds of docker images
on github actions, notably using the github actions cache. The way this
is configured is that some extra steps happen on github which populate
the github actions cache and local docker daemon such that when the
actual build happens, totally independently of these github actions
steps, it just so happens to get cache hits. This is done to ensure that
if we make a mistake here it doesn't result in stale builds, just slower
builds.

The overall hope here is that by using the github actions cache for
docker images we can hit package installation less than we currently do.
The hope is that all the images fit within the github actions cache. I
think they do but it's pretty noisy and each build generates new entries
so I'm not entirely sure what's happening. Local testing shows that
reruns do indeed proceed faster and don't hit `apt-get` for example,
though.

This doesn't entirely insulate us from issues with `apt-get` because
runs will still use `apt-get` externally from docker, such as just
installing a few extra packages. I don't know how to make those more
robust, but hopefully we can at least reduce some flakiness by caching
some things.

prtest:full
@pchickey pchickey added this pull request to the merge queue Apr 20, 2026
Merged via the queue into bytecodealliance:main with commit e5f572a Apr 20, 2026
177 checks passed
@alexcrichton alexcrichton deleted the docker-cache branch April 20, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants