run_tests workflow: add sccache #3127
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
sporksmith
commented
Aug 24, 2023
•
sporksmith
commented
- Set up and use sccache in the "run_tests" workflow. This causes it to be about as fast as the "run_tests_incremental" workflow. It brings the total run time from 2.5h to about 1.5h
- Delete the "run_tests_incremental" workflow. It no longer has a speed advantage over the "run_tests" workflow, required periodically generating new pre-baked Docker images (which currently involved manually running the workflow), and is conceptually more complicated.
- Disable rust incremental builds in the extra_tests workflow. I think they may be causing the cache to be incompatible with the cache of the same system config in the "run_tests" workflow, and incremental builds are unhelpful overhead in this case anyway.
github-actions
bot
added
the
Component: Testing
|
@robgjansen you might also want to take a look to verify this makes sense for "maintenance mode" the sccaches are around 400 MB each, and I think the account limit is 10 GB, so you might occasionally need to manually delete some caches if too many new versions end up getting saved before they get GC'd |
|
Hmm it looks like we're already over our cache limit and for now Github is doing the smart thing:
So I think we're a 👍 for maintenance mode. :) |
stevenengler
approved these changes
Aug 25, 2023
This is redundant with the "run_tests" workflow, which is now just as fast since adding sccache.
* Incremental builds don't help here * sccache won't cache incrementally compiled crates * this is also set in the "run_tests" workflow, where the corresponding configuration shares cache with this job. Using different configurations may cause the caches to be incompatible (i.e. cause cache misses)
I don't think SCCACHE_RECACHE would have worked as intended in this scenario. It tells the sccache server to reset its cache, but the CI wouldn't have written back the cache anyway since the cache key didn't change. Instead we can increment SCCACHE_CACHE_VERSION to force new caches to be saved.
14 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.