Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upEnable sccache for Servo CI #17106
Enable sccache for Servo CI #17106
Comments
|
Using environment variables is easier than updating the .servobuild file for CI. Will installing sccache and setting |
|
That will make it work for caching rust compiles, but not C++ compiles. |
|
OMG this is amazing - thank you so much for doing this, including the testing, @luser! One question: do you know how well this handles invalidation when rustc or cargo are updated? I was chatting with @edunham and we were thinking about doing a "merge commit by merge commit" test of @metajack We're gonna HAVE to try rolling this out to see if we can shave some time off of macOS builds :-) |
…larsbergstrom Support env vars in buildbot steps This makes it possible to specify environment variables for CI builds directly in `buildbot_steps.yml`, whether permanently, such as configuring LLVM assertions in servo/servo#15564, or temporarily, e.g. testing out `SCCACHE` for servo/servo#17106 before putting it into saltfs or trying out Android env vars. This is also helpful for windows, which AFAIK doesn't have an equivalent to the `env` command. r? @larsbergstrom @edunham <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/687) <!-- Reviewable:end -->
…larsbergstrom Support env vars in buildbot steps This makes it possible to specify environment variables for CI builds directly in `buildbot_steps.yml`, whether permanently, such as configuring LLVM assertions in servo/servo#15564, or temporarily, e.g. testing out `SCCACHE` for servo/servo#17106 before putting it into saltfs or trying out Android env vars. This is also helpful for windows, which AFAIK doesn't have an equivalent to the `env` command. r? @larsbergstrom @edunham <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/687) <!-- Reviewable:end -->
…larsbergstrom Support env vars in buildbot steps This makes it possible to specify environment variables for CI builds directly in `buildbot_steps.yml`, whether permanently, such as configuring LLVM assertions in servo/servo#15564, or temporarily, e.g. testing out `SCCACHE` for servo/servo#17106 before putting it into saltfs or trying out Android env vars. This is also helpful for windows, which AFAIK doesn't have an equivalent to the `env` command. r? @larsbergstrom @edunham <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/687) <!-- Reviewable:end -->
|
Are there pre-built releases of sccache available? That would be the easiest way to deploy onto our build machines. |
|
Also, it looks like the |
sccache hashes the contents of all the sharedlibs under $sysroot/lib and uses that as an input to the hash used for the cache key, so it shouldn't be a problem. |
@alexcrichton has binary builds for Rust CI, so you ought to be able to use those until I get actual binary releases setup in sccache CI. |
|
FWIW, I've been testing enabling sccache for Rust compilation in Firefox builds, and for our stylo builds (which build a big chunk of Servo code), it looks to save about 5 minutes per build. |
Install sccache Part of servo/servo#17106. <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/688) <!-- Reviewable:end -->
Install sccache Part of servo/servo#17106. <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/688) <!-- Reviewable:end -->
Enable sccache <!-- Please describe your changes on the following line: --> Update buildbot_steps lint to handle env variables. Enable sccache for all builds. --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #17106 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [x] These changes do not require tests because they update the test commands <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/17171) <!-- Reviewable:end -->
Enable sccache <!-- Please describe your changes on the following line: --> Update buildbot_steps lint to handle env variables. Enable sccache for all builds. --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #17106 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [x] These changes do not require tests because they update the test commands <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/17171) <!-- Reviewable:end -->
|
This has been enabled. |
|
According to jdm who is in person with me right now, this is not actually fixed yet. |
Enable sccache <!-- Please describe your changes on the following line: --> Update buildbot_steps lint to handle env variables. Enable sccache for all builds. --- <!-- Thank you for contributing to Servo! Please replace each `[ ]` by `[X]` when the step is complete, and replace `__` with appropriate data: --> - [ ] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #17106 (github issue number if applicable). <!-- Either: --> - [ ] There are tests for these changes OR - [x] These changes do not require tests because they update the test commands <!-- Also, please make sure that "Allow edits from maintainers" checkbox is checked, so that we can help you if you get stuck somewhere along the way.--> <!-- Pull requests that do not address these steps are welcome, but they will require additional verification as part of the review process. --> <!-- Reviewable:start --> --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/17171) <!-- Reviewable:end -->
|
FYI: I fixed my AppVeyor config, and the 0.2.6 release of sccache includes a Windows binary package: https://github.com/mozilla/sccache/releases/tag/0.2.6 |
|
This was fixed a while ago. |
I've been testing sccache with Servo lately, and it's working pretty well for caching crate compilation. Details are available on this users.rlo thread. I needed to make one change to an upstream crate, and I finally submitted a PR to update that dependency which has now been merged.
Unless large chunks of the dependency graph change frequently I would expect a pretty good cache hit rate, which should save a fair amount of time for CI builds. Additionally, once we get S3 buckets configured we can set
[build] ccache = 'sccache'and use sccache to cache the C++ compilation from the build, which may produce better results than using ccache, since multiple builders can share the same cache.cc/ @alexcrichton @edunham @larsbergstrom