CI: reduce cache usage in GitHub Actions #23987
Conversation
rgommers
added
the
CI
|
Might be better to push this forward once the unrelated CI failures are resolved. |
|
This looks good. In multiple PRs one of the windows test jobs is hanging consistently. Do we have any changes recently that we can attribute to ? |
Based on when it started happening, the culprit is likely to be gh-23844. @lucascolley that job was hanging in gh-23844 right before the merge commit, so you probably shouldn't have merged that one. The lock file contains changes for the |
Ah drat, I was so frustrated with the noise from gh-23958 that I thought it was one of the cancelled jobs being caused by that. Let me see if I can figure it out. EDIT: debugging at lucascolley#36 |
lucascolley
force-pushed
the
ci-reduce-gha-caches
branch
from
a85bbbf to
9801ad0
Compare
lucascolley
left a comment
lucascolley
left a comment
There was a problem hiding this comment.
diff here LGTM if CI is happy!
rgommers
force-pushed
the
ci-reduce-gha-caches
branch
from
9801ad0 to
24be7d9
Compare
|
Should turn green now; still draft until the |
Release builds are 3x-4x smaller than debug builds, hence adding `--release` shrinks the cache usage on GitHub Actions a lot where `ccache` is used. Caching the downloaded packages from PyPI isn't worth it given the 10 GB org-wide limit.
This shows up with GCC 13.3 but not 11.2, see pythran#2373
rgommers
force-pushed
the
ci-reduce-gha-caches
branch
from
93b04d7 to
69d62fc
Compare
Except I forgot we need new NumPy nightlies - trying to get those out. |
|
Green at last 😅 |
|
what a battle that was! |
3 participants
The primary change is using release builds, which are 3x-4x smaller than debug builds, hence adding
--releaseshrinks the cache usage on GitHub Actions a lot whereccacheis used.Caching the downloaded packages from PyPI isn't worth it given the 10 GB org-wide limit.
xref #21491 (comment) for context
Note: there is more to gain along these lines. Using
--releasein conda/pixi envs is complicated by the compiler activation issue (it adds-O2which clashes with--release). This is better left alone until conda-forge finally fixes this issue, since there's some recent motion there.