Don't enqueue onto a disabled dep_graph. #36973

nnethercote · 2016-10-05T05:00:32Z

This commit guards all calls to DepGraphThreadData::enqueue with a
check to make sure it is enabled. This avoids some useless allocation
and vector manipulations when it is disabled (i.e. when incremental
compilation is off) which improves speed by 1--2% on most of the
rustc-benchmarks.

This commit has an observable functional change: when the dep_graph is
disabled its shadow_graph will no longer receive messages. This should
be ok because these message are only used when debug assertions are
enabled.

r? @nikomatsakis

nikomatsakis · 2016-10-05T12:17:51Z

This should be ok because these message are only used when debug assertions are
enabled.

I used to have this setup, but I changed it to the current one specifically so that the shadow graph was active even if incremental is disabled. This is because it detects common errors that are otherwise overlooked (at least until people are using incremental by default).

Perhaps we could do this setup, but only if debug-assertions are not enabled?

michaelwoerister · 2016-10-05T16:45:18Z

FYI: As far as I know, debug assertions are always enabled on the build bots.

TimNN · 2016-10-05T18:35:40Z

@michaelwoerister: I think it's the exact opposite (at least on the stable/beta/nightly build bots): debug_assertions are never enabled (which is why DEBUG logging does not work in these builds)

michaelwoerister · 2016-10-05T18:38:28Z

@TimNN I might not be using the right terminology. What I meant was that the tests that gate whether a PR gets merged are run using a compiler where debug assertions are enabled.

TimNN · 2016-10-05T18:47:26Z

Ah well, I suppose those are build bots as well. (Not sure if there is an "official" terminology).

(And I just checked, debug_assertions are indeed enabled on the test bots (or whatever you want to call them): https://github.com/rust-lang/rust-buildbot/blob/master/master/master.cfg#L1775)

michaelwoerister · 2016-10-05T19:53:01Z

@TimNN Thanks for making facts out of my speculations :)

nnethercote · 2016-10-05T22:41:00Z

Ok, I will update the code to still push the messages if debug assertions are enabled.

nnethercote · 2016-10-06T02:57:41Z

r? @nikomatsakis

nikomatsakis · 2016-10-11T14:07:47Z

src/librustc/dep_graph/raii.rs

@@ -19,15 +19,21 @@ pub struct DepTask<'graph> {

 impl<'graph> DepTask<'graph> {
    pub fn new(data: &'graph DepGraphThreadData, key: DepNode<DefId>)
-               -> DepTask<'graph> {
-        data.enqueue(DepMessage::PushTask(key.clone()));


Have you measured the impact of this on the perf also when construction is enabled? When I first wrote the code, I went to some lengths to make it straightline and cheap when the graph was enabled, and the overhead I measured was quite low. The idea was that copying into the buffer without a branch could actually be cheaper than not copying at all. I did measure and I recall finding this to be the case (I think I tested libsyntax), but a lot has changed since then, and for all I know my measurements at the time were flawed. =) We have a better setup now.

(After all, we do expect "incremental enabled" to be the common case, right? I would think that incr will always be on, unless you are building a final release artifact, in which case you are already asking for a lot of time to be spent on optimization.)

nnethercote · 2016-10-11T20:03:20Z

The only measurement of incremental compilation I did is of syntex-incr in rustc-benchmarks. Its speed was unaffected by the patch. Perfectly predictable branches tend to be cheap :)

nikomatsakis · 2016-10-11T21:06:51Z

Seems good.

@bors r+

bors · 2016-10-11T21:06:53Z

📌 Commit d54b044 has been approved by nikomatsakis

Rollup of 17 pull requests - Successful merges: #36762, #36831, #36973, #36991, #36995, #37023, #37049, #37050, #37056, #37064, #37066, #37067, #37084, #37089, #37091, #37092, #37110 - Failed merges:

alexcrichton · 2016-10-13T00:51:00Z

@bors: r-

Unfortunately I think this caused this failure, specifically:

Building stage1 std artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
    Finished release [optimized] target(s) in 656.99 secs
   Compiling build_helper v0.1.0 (file:///C:/bot/slave/auto-win-msvc-64-cargotest/build/src/build_helper)
   Compiling core v0.0.0 (file:///C:/bot/slave/auto-win-msvc-64-cargotest/build/src/libcore)
   Compiling gcc v0.3.35
   Compiling unwind v0.0.0 (file:///C:/bot/slave/auto-win-msvc-64-cargotest/build/src/libunwind)
   Compiling libc v0.0.0 (file:///C:/bot/slave/auto-win-msvc-64-cargotest/build/src/rustc/libc_shim)
Build failed, waiting for other jobs to finish...
error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

thread 'rustc' panicked at 'should never swap if not fully enabled', C:\bot\slave\auto-win-msvc-64-cargotest\build\src\librustc\dep_graph\thread.rs:108
note: Run with `RUST_BACKTRACE=1` for a backtrace.

This happened on MSVC for me but not on Linux, and I had to take it out of a rollup as a result.

This commit guards all calls to `DepGraphThreadData::enqueue` with a check to make sure it is enabled. This requires distinguishing between a "fully enabled" and an "enqueue-enabled" graph. This change avoids some useless allocation and vector manipulations when the graph is disabled (i.e. when incremental compilation is off) which improves speed by ~1% on some of the rustc-benchmarks.

nnethercote · 2016-10-18T04:27:24Z

I've fixed the problem and pushed an updated commit. The problem was that I mistakenly removed the the enabled check in enqueue (right at the end of the diff). It's back now.

r? @nikomatsakis

nikomatsakis · 2016-10-19T20:21:37Z

@bors r+

bors · 2016-10-19T20:21:38Z

📌 Commit cde42cd has been approved by nikomatsakis

bors · 2016-10-20T05:05:13Z

⌛ Testing commit cde42cd with merge eb38d42...

@nikomatsakis

Don't enqueue onto a disabled dep_graph. This commit guards all calls to `DepGraphThreadData::enqueue` with a check to make sure it is enabled. This avoids some useless allocation and vector manipulations when it is disabled (i.e. when incremental compilation is off) which improves speed by 1--2% on most of the rustc-benchmarks. This commit has an observable functional change: when the dep_graph is disabled its `shadow_graph` will no longer receive messages. This should be ok because these message are only used when debug assertions are enabled. r? @nikomatsakis

bors · 2016-10-20T08:09:17Z

rust-highfive assigned nikomatsakis Oct 5, 2016

nnethercote force-pushed the dep_graph branch from b4e841c to d54b044 Compare October 6, 2016 02:57

nikomatsakis reviewed Oct 11, 2016

View reviewed changes

alexcrichton mentioned this pull request Oct 12, 2016

Rollup of 17 pull requests #37118

Merged

nnethercote force-pushed the dep_graph branch from d54b044 to cde42cd Compare October 18, 2016 04:25

bors merged commit cde42cd into rust-lang:master Oct 20, 2016

nnethercote deleted the dep_graph branch October 21, 2016 03:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Don't enqueue onto a disabled dep_graph. #36973

Don't enqueue onto a disabled dep_graph. #36973

nnethercote commented Oct 5, 2016

nikomatsakis commented Oct 5, 2016

michaelwoerister commented Oct 5, 2016

TimNN commented Oct 5, 2016

michaelwoerister commented Oct 5, 2016

TimNN commented Oct 5, 2016

michaelwoerister commented Oct 5, 2016

nnethercote commented Oct 5, 2016

nnethercote commented Oct 6, 2016

nikomatsakis Oct 11, 2016 •

edited

Loading

nnethercote commented Oct 11, 2016

nikomatsakis commented Oct 11, 2016

bors commented Oct 11, 2016

alexcrichton commented Oct 13, 2016

nnethercote commented Oct 18, 2016

nikomatsakis commented Oct 19, 2016

bors commented Oct 19, 2016

bors commented Oct 20, 2016

bors commented Oct 20, 2016

Don't enqueue onto a disabled dep_graph. #36973

Don't enqueue onto a disabled dep_graph. #36973

Conversation

nnethercote commented Oct 5, 2016

nikomatsakis commented Oct 5, 2016

michaelwoerister commented Oct 5, 2016

TimNN commented Oct 5, 2016

michaelwoerister commented Oct 5, 2016

TimNN commented Oct 5, 2016

michaelwoerister commented Oct 5, 2016

nnethercote commented Oct 5, 2016

nnethercote commented Oct 6, 2016

nikomatsakis Oct 11, 2016 • edited Loading

Choose a reason for hiding this comment

nnethercote commented Oct 11, 2016

nikomatsakis commented Oct 11, 2016

bors commented Oct 11, 2016

alexcrichton commented Oct 13, 2016

nnethercote commented Oct 18, 2016

nikomatsakis commented Oct 19, 2016

bors commented Oct 19, 2016

bors commented Oct 20, 2016

bors commented Oct 20, 2016

nikomatsakis Oct 11, 2016 •

edited

Loading