build: disable incremental build in CI #7838
Conversation
github-actions
bot
added
Invalid PR Title
type/build
Codecov Report
@@ Coverage Diff @@
## main #7838 +/- ##
==========================================
- Coverage 71.73% 71.73% -0.01%
==========================================
Files 1109 1109
Lines 176825 176825
==========================================
- Hits 126850 126839 -11
- Misses 49975 49986 +11
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
opt-level=2 opt-level=3 (Why it's slower?? 😅) |
|
Why Is it because |
xxchan
changed the title
|
sccache shows Non-cacheable calls 215 -> 188 |
| @@ -36,3 +36,4 @@ RUN cargo install cargo-llvm-cov cargo-nextest cargo-udeps cargo-hakari cargo-so | |||
|
|
|||
| ENV RUSTC_WRAPPER=sccache | |||
| ENV SCCACHE_BUCKET=ci-sccache-bucket | |||
| ENV CARGO_INCREMENTAL=0 | |||
There was a problem hiding this comment.
Does this override incremental build for non sim build? Did we show a speedup or lack of regression there?
There was a problem hiding this comment.
We already have that set for ci-dev profile previously.
Lines 95 to 97 in 90833f3
I think generally it makes not sense to have incremental build in CI (besides affecting sccache, IIRC it also has a little bit overheads), so I added the envvar as an one-for-all approach.
There was a problem hiding this comment.
Disable incremental compilation. CI builds often are closer to from-scratch builds, as changes are typically much bigger than from a local edit-compile cycle. For from-scratch builds, incremental adds an extra dependency-tracking overhead. It also significantly increases the amount of IO and the size of ./target, which make caching less effective.
There was a problem hiding this comment.
Great. I wonder why we did not make the deterministic build non-incremental in the first place then :)
Anw, if it is faster then we should definitely do it. Btw, do we test the CI on a release build?
There was a problem hiding this comment.
opt-level=2
Perhaps we should be testing at the same opt-level as our release build?
| @@ -36,3 +36,4 @@ RUN cargo install cargo-llvm-cov cargo-nextest cargo-udeps cargo-hakari cargo-so | |||
|
|
|||
| ENV RUSTC_WRAPPER=sccache | |||
| ENV SCCACHE_BUCKET=ci-sccache-bucket | |||
| ENV CARGO_INCREMENTAL=0 | |||
There was a problem hiding this comment.
Why do we need this extra env? 🤔
There was a problem hiding this comment.
Well as @xxchan mentioned we are building the docker image from scratch
There was a problem hiding this comment.
Although, I'm not sure if we use this dockerfile to build release binaries/performance pipeline. If we do it is even better since the docker build is always from scratch.
There was a problem hiding this comment.
Why do we need this extra env? 🤔
So that we don’t need to set every ci-* profile…🤪 I just didn’t delete the profile settings
There was a problem hiding this comment.
Although, I'm not sure if we use this dockerfile to build release binaries/performance pipeline. If we do it is even better since the docker build is always from scratch.
Didn’t thought about that, but yes it can be enabled. It’s a free optimization 🤔
There was a problem hiding this comment.
Yes, I'm saying it's in fact this env var which would set the incremental build to true for those, as the benchmarking build relies on the docker image built from this file.
|
Hi, is there any comparison before/after using sccache? I want to shout it out (as a sccache contrituor)~ |
You can look at the buildkite pipeline page 🤔 |
I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
significantly reduced build time