chore: add caching to azure pipelines #8866
Conversation
| @@ -22,6 +22,12 @@ steps: | |||
| - script: node scripts/remove-postinstall | |||
| displayName: 'Remove postinstall script' | |||
|
|
|||
| - task: CacheBeta@0 | |||
There was a problem hiding this comment.
|
Ergh @willsmythe could you provide some info here? 🙏 |
|
I checked the telemetry and the problem doesn’t appear to be specific to your pipeline/job. Can you try re-running the pipeline and see what happens? This might have been an intermittent problem that impacted the agent this particular job got assigned. I’ll follow-up with the engineering team …
|
|
Cool, thanks! I've clicked the button. I guess a new run will also let me verify if the cache worked for the mac and windows runs, as they completed successfully 🙂 |
| - task: CacheBeta@0 | ||
| inputs: | ||
| key: yarn | $(Agent.OS) | yarn.lock | ||
| path: $(YARN_CACHE_FOLDER) |
There was a problem hiding this comment.
Btw, there is a caching feature coming in the next few weeks (tarring cache contents) that will make it easier/possible to cache node_modules. This generally results in better caching performance compared to caching the Yarn (or npm) global cache folder.
Definitely curious to see what improvements you get here. Keep in mind the first "save cache" operation within an org typically takes longer than future save operations (so don't be surprised if your first run takes longer than usual).
There was a problem hiding this comment.
Awesome, that sounds great! That's what we cache on circus as well. I'll set up a reminder for myself to check back in a couple of weeks 🙂
There was a problem hiding this comment.
Is there an issue I could track for this work? I tried searching on https://github.com/microsoft/azure-pipelines-yaml (dunno if it's even the correct repo) without finding anything
There was a problem hiding this comment.
Hey @SimenB - see microsoft/azure-pipelines-tasks#10925
Net-net: once your hosted agent's version (which is reported in the Initialize job step) shows as 2.157.0 or later, you can try caching the node_modules folder after setting the AZP_CACHING_TAR variable to true.
There was a problem hiding this comment.
Awesome, thanks! Seems we're 2.155 for now, so I'll check back in a few days 🙂
There was a problem hiding this comment.
@willsmythe I see we're getting the 2.160 agents now. Do you still recommend to change anything? The docs still state to cache the yarn cache folder: https://docs.microsoft.com/en-us/azure/devops/pipelines/caching/?view=azure-devops#nodejsyarn
There was a problem hiding this comment.
@SimenB - yah we should try caching node_modules using the released version of the Cache task (V2). I'll test out a few options.
There was a problem hiding this comment.
Here are a few tests. Note: I wouldn't say this is totally scientific because, as we have seen, there is some variability in terms of build time from one run to the next (even without any changes):
| Option | Install dependencies (Linux) | Run tests (Windows) | Total time |
|---|---|---|---|
| A. Yarn global | 1m 7s | 18m 22s (84%) | 21m 57s |
B. node_modules |
1m 52s | 18m 50s (80%) | 23m 51s |
C. node_modules and Yarn global |
37s | 18m 42s (86%) | 21m 41s |
Net-net: I don't think Jest will see huge performance improvements by switching to caching just node_modules versus Yarn's global cache directory. The added time to restore/save the extra bytes and thousands of files seems to offset any gains.
It is interesting that caching both (option C) resulted in better "install dependency" times, but just caching node_modules resulted in significantly worse times.
I experimented some with Yarn's offline cache, but didn't see any significant improvements.
The third column highlights where most of the overall time is spent (running tests). If you look at the Tests results for a run and filter to include Passing tests and order by Duration, you can see there are numerous tests (in all platforms) that take over 5s each. This obviously adds up over time.
Based on this, the best way to speed up Jest test runs is probably to split Jest test runs across more pipeline jobs,
There was a problem hiding this comment.
Ah, ok. We'll leave it alone then. Thank you so much for diving into this @willsmythe!
|
Doesn't seem like the build wants to start for whatever reason. Maybe just a queue?
|
@SimenB - do you recall which button you clicked? Was it in the GitHub Checks tab, or in Azure Pipelines? Asking because I see a new run (https://dev.azure.com/jestjs/jest/_build/results?buildId=3224&view=results), but it looks stalled. I've asked the engineering team here about this as well ... |
|
Looks stuck, yeah. I clicked the
|
|
The build never triggered... Any idea what's going on? |
|
I'll just rebase this, which should trigger a build. |
SimenB
force-pushed
the
azure-pipeline-cache
branch
from
2e127a1
to
6b48a4d
Compare
Codecov Report
@@ Coverage Diff @@
## master #8866 +/- ##
=======================================
Coverage 64.13% 64.13%
=======================================
Files 275 275
Lines 11627 11627
Branches 2845 2844 -1
=======================================
Hits 7457 7457
Misses 3544 3544
Partials 626 626Continue to review full report at Codecov.
|
|
Ish 3 min install now on all OSes - hopefully we'll see an improvement |
|
Yeah, it's about a minute faster, which is great: Caching |
|
@SimenB - good to hear. The issue with the failed job (which only happens for some orgs on the very first job using caching) is being investigated. Looks to be a race condition. I haven't heard back from the team on the second issue (rebuild run fails to start). |
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Summary
Hopefully this'll speed up CI.
Test plan
Green CI with some nice cache output?