-
Notifications
You must be signed in to change notification settings - Fork 11.9k
ci: remove circle ci cache fallback #17533
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Cache fallback can be useful in some cases, but in our case it is hindering the CI performance as the cache is growing everyday which results in slower CI times because of the restoring and saving phases. A fresh cache size is about 450Mb, while currently with fallbacking enabled it has grown to 3.8Gb Below are some timings taken were we see that having cache fallbacking is doesn't help CI times: | Caching | Cache Size | Restore Cache | Yarn Install | Save Cache | Total Time Elapsed | |---------------------------|------------|---------------|--------------|------------|--------------------| | Cache w/ Fallback (Miss) | 3.81Gb | 2m 50s | 30s | 7m 20s | 10m 40s | | Cache w/ Fallback (Hit) | 3.81Gb | 2m 50s | 20s | 0 | 3m 10s | | Cache wo/ Fallback (Miss) | 0 | 0 | 40s | 30s | 1m 10s | | Cache wo/ Fallback (Hit) | 458mb | 20s | 20s | 0 | 40s |
Is the assumption here is that, even if we using a smaller backup cache, it still would be slower than just ignoring it? The 3.81gb is surely bloated, but we can discard it by just changing key prefixes and thus go back to a similar cache size. If the backup cache is also ~450mbs, then it should have have a similar total time of |
Yes, because there is the overhead of restoring the fallback cache and saving the new cache.
We'd need to change the keys quite often, based on how many packages have been updated per week. Earlier this week:
Today:
Indeed the timings would be very similar, but what would be the benefit seeing that without cache fallback is as performant, and we'd need to keep re-invalidated the cache key every couple of days manually, since we don't have any other means to re-invalid the fallback cache. |
Eish, wasn't expecting our package church to be that high. I agree it's better to just skip the fallback then. |
I think you meant package cache 😛 |
This issue has been automatically locked due to inactivity. Read more about our automatic conversation locking policy. This action has been performed automatically by a bot. |
Cache fallback can be useful in some cases, but in our case it is hindering the CI performance as the cache is growing everyday which results in slower CI times because of the restoring and saving phases.
A fresh cache size is about 450Mb, while currently with fallbacking enabled it has grown to 3.8Gb
Below are some timings taken were we see that having cache fallbacking is doesn't help CI times: