Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add restore key for cache restoring #145

Closed
wants to merge 1 commit into from
Closed

Conversation

RSWilli
Copy link

@RSWilli RSWilli commented Sep 1, 2023

Description

we have a lot of gstreamer caches in https://github.com/go-gst/go-gst/actions/caches but the caches were not used and the build times for ubuntu-latest were >25min since gstreamer was built all the time

e.g.: https://github.com/go-gst/go-gst/actions/runs/6050337478/job/16419459150#step:3:16 (16min setup gstreamer)

 Pre-built not found in cache; creating a new one. (key: "go-gst-go-gst-https://gitlab.freedesktop.org/gstreamer/gstreamer.git-1.22.0-x86_64-Ubuntu-22.04-meson-1.2.1-1")

Although that key exists a lot in the cache files.

Fix

this PR addresses this issue by adding a restore key to the cache restore call. I experimented in go-gst/go-gst#35 with my fork and now the action only takes 52sec: https://github.com/go-gst/go-gst/actions/runs/6052196157/job/16425193945?pr=35

@RSWilli RSWilli requested a review from a team as a code owner September 1, 2023 17:22
@RSWilli RSWilli requested review from btgoodwin and removed request for a team September 1, 2023 17:22
@RSWilli
Copy link
Author

RSWilli commented Sep 1, 2023

I dont know why this is necessary by the way. I assume it has something to do with caches from different branches.

@btgoodwin
Copy link
Contributor

Thank you for taking the time to look into this and the details. The proposed change would amount to calling the same key twice in a row though (restoreCache). Instead, I think what you may be running into is the design of github workflow cache itself.
It appears your various keys all of the same ID are on different pull requests but not on the main branch, and that's probably why you were getting the initial miss. Workflow Cache Restrictions:

When a cache is created by a workflow run triggered on a pull request, the cache is created for the merge ref (refs/pull/.../merge). Because of this, the cache will have a limited scope and can only be restored by re-runs of the pull request. It cannot be restored by the base branch or other pull requests targeting that base branch.

If that's not the case, I would be happy to look into this further.

@RSWilli
Copy link
Author

RSWilli commented Sep 6, 2023

yes I know this "abuses" the restoreKeys feature a bit. I didn't want to make the key shorter, because the key is essentially the version that will be installed.

We do have the caches on different branches, but the whole reason for caching the setup-gstreamer is that the installation of a specific version is faster if it has been performed before.

If you think this is too much of a breaking change comparing to the current version we could also add another parameter like cache-across-prs: true where we restore the cache with the restoreKey

@RSWilli RSWilli closed this Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants