🏗 Clean up build cache on Travis #22926
Conversation
There was a problem hiding this comment.
I believe sauce_connect is/used to be where we stored the sauce proxy binary. @rsimha for more context
.travis.yml
Outdated
| @@ -77,19 +77,19 @@ jobs: | |||
| script: | |||
| - unbuffer node build-system/pr-check/remote-tests.js | |||
| after_script: | |||
| - build-system/sauce_connect/stop_sauce_connect.sh | |||
There was a problem hiding this comment.
This was intentionally duplicated because when a test task fails, the sauce connect proxy isn't stopped by the Travis job due to execOrDie. When tests pass, the second after_script call is a no-op.
.travis.yml
Outdated
| cache: | ||
| directories: | ||
| - node_modules | ||
| - build-system/tasks/e2e/node_modules | ||
| - build-system/tasks/visual-diff/node_modules | ||
| - sauce_connect |
There was a problem hiding this comment.
This is the directory to which build-system/sauce_connect/start_sauce_connect.sh downloads the proxy. Caching it avoids a fresh download for each build.
| - validator/node_modules | ||
| - validator/nodejs/node_modules | ||
| - validator/webui/node_modules | ||
| - $HOME/.cache/pip |
There was a problem hiding this comment.
Travis' native pip caching only works for python projects. $HOME/.cache/pip was cached based on these results.
From the logs, this line suggests that a cached install location is being used (even though, as you pointed out, the cache is updated at the end of the build). Curious: does the install time stay the same when caching is disabled?
There was a problem hiding this comment.
Interesting repo! By removing it from the cache, pip install increases from 3s to 5s but store build cache decreases from 85s to 10s.
There was a problem hiding this comment.
Yeah, good catch on your part in noticing the much slower store step!
There was a problem hiding this comment.
LGTM after reverting the two sauce_connect changes.
Fixing pip caching, or alternatively, replacing pip with something else (context: #22528 (comment)) can be done separately.
* Remove pip and sauce_connect from Travis cache, remove duplicate stop_sauce_connect script * Add debugging for remote and e2e jobs * readd sauce
Removes
$HOME/.cache/pipfrom the cache because it changes every build, which means uploading new files for every Travis job.pipshould already be supported on Xenial VMs but if not, we should install it in abefore_installscript instead of keeping in the cache.Removes
stop_sauce_connect.shbecause it's a duplicate.Also removes
sauce_connectfrom the cache - unclear what that is?Adds
ps -efasafter_scripts on the remote test and e2e jobs, because they have a tendency to hang and I'd like to see what processes are still running after the tests are complete.