use precalculated dominators in explain_borrow #65172

tanriol · 2019-10-06T21:34:34Z

This looks like the only place calculating dominators from the MIR body every time instead of using the ones stored on the MirBorrowckCtxt. For example, in #65131 a big generated function with a number of borrowck errors takes a few hours(!) recalculating the dominators while explaining the errors.

I don't know enough about this part of rustc codebase to know for sure that this change is correct, but no tests seem to fail as a result of this change in local testing.

This looks like the only place calculating dominators from the MIR body every time instead of using the ones stored on the `MirBorrowckCtxt`. For example, in rust-lang#65131 a big generated function with a number of borrowck errors takes a few hours(!) recalculating the dominators while explaining the errors. I don't know enough about this part of rustc codebase to know for sure that this change is correct, but no tests seem to fail as a result of this change in local testing.

rust-highfive · 2019-10-06T21:34:37Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @eddyb (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

nagisa · 2019-10-07T02:01:48Z

@bors r+

The only case where this change can be wrong is in case we have wrong code elsewhere that does not invalidate the cache in some way after doing some modification. Since borrowck does not mutate anything, this change seems perfectly fine to me.

bors · 2019-10-07T02:01:49Z

📌 Commit 4cc707c has been approved by nagisa

nagisa · 2019-10-07T02:04:14Z

@bors r- delegate+

@tanriol would be great if you also added a regression test case. A good test could be just something that runs for a dozen minutes without this fix and completes in under a second with the fix. If that is not feasible, then something that checks whether the compiler terminates with the minimized reproducer from #65131 would also do.

bors · 2019-10-07T02:04:16Z

✌️ @tanriol can now approve this pull request

bors · 2019-10-07T02:06:51Z

✌️ @tanriol can now approve this pull request

nagisa · 2019-10-07T02:07:40Z

I have also added Fixes #65131 to the issue description body for you.

kpp · 2019-10-07T07:03:39Z

On this case: #65131 (comment)

19.8s (nightly 1.40) vs 0.3s (this branch)

<3

tanriol · 2019-10-07T09:05:03Z

@nagisa What's the proper way of testing this? As written currently this is a performance-only change, and compiletest does not seem to have any test timeouts. Sure, I can scale up this test to take an hour and hope there are CI timeouts that'll fail, but this does not seem to be a good idea.

The options I'm seeing are:

work on adding timeout support to compiletest,
(probably easier) add to compiletest support for success time limit without terminating rustc,
make a run-make test and run rustc with a timeout from the Makefile (probably UNIX-only),
(alternatively) add some checks to rustc to ensure that body.dominators() is not called "too many" times,
any other ideas?

UPD: sorry, this reply was written based on the email notification without noticing the edits.

glandium · 2019-10-07T11:17:16Z

How about adding failure tests on https://perf.rust-lang.org/ ?

nagisa · 2019-10-07T11:37:58Z

The easiest approach is probably a run-pass test similar to this one which spawns itself as another process with an extra argument and then calculates the time necessary for the child to complete from the parent process.

tanriol · 2019-10-07T11:40:59Z

The slowdown is not to runtime, but to compilation process, so the alternative would be to... spawn rustc, I guess?

bjorn3 · 2019-10-07T11:46:08Z

Useful trick: When you use - as source file, rustc will read from stdin. Just make sure you close stdin, as rustc will otherwise keep waiting for more input.

nagisa · 2019-10-07T11:54:18Z

The slowdown is not to runtime, but to compilation process

Ah yeah, sorry, I didn’t wake up entirely before writing that down. In that case this one is an example of a test that invokes rustc within some arbitrary Rust executable.

JohnCSimon · 2019-10-12T04:13:38Z

Hello from triage - this has sat idle for the last 5 days.
@eddyb I can't tell if this is waiting to be merged or additional tests from @tanriol
CC: @spastorino

spastorino · 2019-10-12T04:15:48Z

Seemed to be waiting for additional tests. Looks good to me btw :).

tanriol · 2019-10-12T05:08:36Z

Going to add a test in a couple days.

tmandry · 2019-10-15T22:27:59Z

I'm confused why we would add this test in-tree instead of as a benchmark to rustc-perf. (Example PR, btw)

tanriol · 2019-10-16T08:57:07Z

@tmandry Thank you for the example.

Is rustc-perf a correct place for, basically, a regression test for error reporting performance in a very specific case of a huge generated function with borrow-check errors? It feels to me a little heavyweight for such a small problem (cluttering of the "all benchmarks" view with an additional graph, for example).

Basically, my feeling was that rustc-perf / perf.rust-lang.org is a human-oriented resource, not an automated testing oriented one, and the exact performance of this use case is not something worth permanent human attention. However, if you consider this position wrong, please tell me so and I'll add the test there.

nagisa · 2019-10-16T12:27:10Z

rustc perf tests will not block landing and will only test retroactively, which is not terrible, but not ideal for issues like these either.

That being said glandium suggests there are some sort of a "failure test" mode in rustc-perf? In that case it sounds like a fairly good compromise.

nagisa · 2019-10-16T12:30:35Z

@bors r+

I am approving this PR for landing due to the large improvement on perf that this gives, with an expectation that a regression test is added as a follow-up PR (either to rustc itself or to rustc-perf or somewhere else entirely?)

To track this follow-up work, I am a) removing the Fixes #65131 annotation so that the issue does not get closed and tagging the issue with E-needs-test.

bors · 2019-10-16T12:30:37Z

📌 Commit 4cc707c has been approved by nagisa

bors · 2019-10-16T12:32:05Z

💡 This pull request was already approved, no need to approve it again.

There's another pull request that is currently being tested, blocking this pull request: [dev-nightly] Testing out rustc-dev #65453

bors · 2019-10-16T12:32:06Z

📌 Commit 4cc707c has been approved by nagisa

bors · 2019-10-16T12:49:58Z

⌛ Testing commit 4cc707c with merge 53aca55...

… r=nagisa use precalculated dominators in explain_borrow This looks like the only place calculating dominators from the MIR body every time instead of using the ones stored on the `MirBorrowckCtxt`. For example, in #65131 a big generated function with a number of borrowck errors takes a few hours(!) recalculating the dominators while explaining the errors. I don't know enough about this part of rustc codebase to know for sure that this change is correct, but no tests seem to fail as a result of this change in local testing.

bors · 2019-10-16T16:48:37Z

☀️ Test successful - checks-azure
Approved by: nagisa
Pushing 53aca55 to master...

pnkfelix · 2019-10-17T14:28:50Z

Discussed at T-compiler meeting. Accepted for beta backport.

[beta] backport rollup This includes a bunch of PRs: * Fix redundant semicolon lint interaction with proc macro attributes #64387 * Upgrade async/await to "used" keywords. #64875 * syntax: fix dropping of attribute on first param of non-method assocated fn #64894 * async/await: improve not-send errors #64895 * Silence unreachable code lint from await desugaring #64930 * Always mark rust and rust-call abi's as unwind #65020 * Account for macro invocation in `let mut $pat` diagnostic. #65123 * Ensure that associated `async fn`s have unique fresh param names #65142 * Add troubleshooting section to PGO chapter in rustc book. #65402 * Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302 * Optimize `try_expand_impl_trait_type` #65293 * use precalculated dominators in explain_borrow #65172 * Fix ICE #64964 #64989

[beta] backport rollup This includes a bunch of PRs: * Fix redundant semicolon lint interaction with proc macro attributes #64387 * Upgrade async/await to "used" keywords. #64875 * syntax: fix dropping of attribute on first param of non-method assocated fn #64894 * async/await: improve not-send errors #64895 * Silence unreachable code lint from await desugaring #64930 * Always mark rust and rust-call abi's as unwind #65020 * Account for macro invocation in `let mut $pat` diagnostic. #65123 * Ensure that associated `async fn`s have unique fresh param names #65142 * Add troubleshooting section to PGO chapter in rustc book. #65402 * Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302 * Optimize `try_expand_impl_trait_type` #65293 * use precalculated dominators in explain_borrow #65172 * Fix ICE #64964 #64989 * [beta] Revert "Auto merge of #62948 - matklad:failable-file-loading, r=petro… #65273 * save-analysis: Don't ICE when resolving qualified type paths in struct members #65353 * save-analysis: Nest tables when processing impl block definitions #65511 * Avoid ICE when checking `Destination` of `break` inside a closure #65518 * Avoid ICE when adjusting bad self ty #65755 * workaround msys2 bug #65762

rust-highfive assigned eddyb Oct 6, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Oct 6, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 7, 2019

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Oct 7, 2019

spastorino approved these changes Oct 7, 2019

View reviewed changes

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 16, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Oct 16, 2019

bors merged commit 4cc707c into rust-lang:master Oct 16, 2019

nagisa added the beta-nominated Nominated for backporting to the compiler in the beta channel. label Oct 17, 2019

pnkfelix added the beta-accepted Accepted for backporting to the compiler in the beta channel. label Oct 17, 2019

Mark-Simulacrum removed the beta-nominated Nominated for backporting to the compiler in the beta channel. label Oct 22, 2019

Mark-Simulacrum mentioned this pull request Oct 22, 2019

[beta] backport rollup #65708

Merged

Skepfyr mentioned this pull request Jul 25, 2021

Exponential compile time with recursive opaque type #87450

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

use precalculated dominators in explain_borrow #65172

use precalculated dominators in explain_borrow #65172

tanriol commented Oct 6, 2019 •

edited by nagisa

Loading

rust-highfive commented Oct 6, 2019

nagisa commented Oct 7, 2019

bors commented Oct 7, 2019

nagisa commented Oct 7, 2019 •

edited

Loading

bors commented Oct 7, 2019

bors commented Oct 7, 2019

nagisa commented Oct 7, 2019

kpp commented Oct 7, 2019

tanriol commented Oct 7, 2019 •

edited

Loading

glandium commented Oct 7, 2019

nagisa commented Oct 7, 2019

tanriol commented Oct 7, 2019

bjorn3 commented Oct 7, 2019 •

edited

Loading

nagisa commented Oct 7, 2019 •

edited

Loading

JohnCSimon commented Oct 12, 2019

spastorino commented Oct 12, 2019

tanriol commented Oct 12, 2019

tmandry commented Oct 15, 2019

tanriol commented Oct 16, 2019

nagisa commented Oct 16, 2019 •

edited

Loading

nagisa commented Oct 16, 2019 •

edited

Loading

bors commented Oct 16, 2019

bors commented Oct 16, 2019

bors commented Oct 16, 2019

bors commented Oct 16, 2019

bors commented Oct 16, 2019

pnkfelix commented Oct 17, 2019

use precalculated dominators in explain_borrow #65172

use precalculated dominators in explain_borrow #65172

Conversation

tanriol commented Oct 6, 2019 • edited by nagisa Loading

rust-highfive commented Oct 6, 2019

nagisa commented Oct 7, 2019

bors commented Oct 7, 2019

nagisa commented Oct 7, 2019 • edited Loading

bors commented Oct 7, 2019

bors commented Oct 7, 2019

nagisa commented Oct 7, 2019

kpp commented Oct 7, 2019

tanriol commented Oct 7, 2019 • edited Loading

glandium commented Oct 7, 2019

nagisa commented Oct 7, 2019

tanriol commented Oct 7, 2019

bjorn3 commented Oct 7, 2019 • edited Loading

nagisa commented Oct 7, 2019 • edited Loading

JohnCSimon commented Oct 12, 2019

spastorino commented Oct 12, 2019

tanriol commented Oct 12, 2019

tmandry commented Oct 15, 2019

tanriol commented Oct 16, 2019

nagisa commented Oct 16, 2019 • edited Loading

nagisa commented Oct 16, 2019 • edited Loading

bors commented Oct 16, 2019

bors commented Oct 16, 2019

bors commented Oct 16, 2019

bors commented Oct 16, 2019

bors commented Oct 16, 2019

pnkfelix commented Oct 17, 2019

tanriol commented Oct 6, 2019 •

edited by nagisa

Loading

nagisa commented Oct 7, 2019 •

edited

Loading

tanriol commented Oct 7, 2019 •

edited

Loading

bjorn3 commented Oct 7, 2019 •

edited

Loading

nagisa commented Oct 7, 2019 •

edited

Loading

nagisa commented Oct 16, 2019 •

edited

Loading

nagisa commented Oct 16, 2019 •

edited

Loading