Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use precalculated dominators in explain_borrow #65172

Merged

Conversation

@tanriol
Copy link
Contributor

tanriol commented Oct 6, 2019

This looks like the only place calculating dominators from the MIR body every time instead of using the ones stored on the MirBorrowckCtxt. For example, in #65131 a big generated function with a number of borrowck errors takes a few hours(!) recalculating the dominators while explaining the errors.

I don't know enough about this part of rustc codebase to know for sure that this change is correct, but no tests seem to fail as a result of this change in local testing.

This looks like the only place calculating dominators from the MIR body every time instead of using the ones stored on the `MirBorrowckCtxt`. For example, in #65131 a big generated function with a number of borrowck errors takes a few hours(!) recalculating the dominators while explaining the errors.

I don't know enough about this part of rustc codebase to know for sure that this change is correct, but no tests seem to fail as a result of this change in local testing.
@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Oct 6, 2019

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @eddyb (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Oct 7, 2019

@bors r+

The only case where this change can be wrong is in case we have wrong code elsewhere that does not invalidate the cache in some way after doing some modification. Since borrowck does not mutate anything, this change seems perfectly fine to me.

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 7, 2019

📌 Commit 4cc707c has been approved by nagisa

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Oct 7, 2019

@bors r- delegate+

@tanriol would be great if you also added a regression test case. A good test could be just something that runs for a dozen minutes without this fix and completes in under a second with the fix. If that is not feasible, then something that checks whether the compiler terminates with the minimized reproducer from #65131 would also do.

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 7, 2019

✌️ @tanriol can now approve this pull request

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 7, 2019

✌️ @tanriol can now approve this pull request

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Oct 7, 2019

I have also added Fixes #65131 to the issue description body for you.

@kpp

This comment has been minimized.

Copy link
Contributor

kpp commented Oct 7, 2019

On this case: #65131 (comment)

19.8s (nightly 1.40) vs 0.3s (this branch)

<3

@tanriol

This comment has been minimized.

Copy link
Contributor Author

tanriol commented Oct 7, 2019

@nagisa What's the proper way of testing this? As written currently this is a performance-only change, and compiletest does not seem to have any test timeouts. Sure, I can scale up this test to take an hour and hope there are CI timeouts that'll fail, but this does not seem to be a good idea.

The options I'm seeing are:

  • work on adding timeout support to compiletest,
  • (probably easier) add to compiletest support for success time limit without terminating rustc,
  • make a run-make test and run rustc with a timeout from the Makefile (probably UNIX-only),
  • (alternatively) add some checks to rustc to ensure that body.dominators() is not called "too many" times,
  • any other ideas?

UPD: sorry, this reply was written based on the email notification without noticing the edits.

@glandium

This comment has been minimized.

Copy link
Contributor

glandium commented Oct 7, 2019

How about adding failure tests on https://perf.rust-lang.org/ ?

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Oct 7, 2019

The easiest approach is probably a run-pass test similar to this one which spawns itself as another process with an extra argument and then calculates the time necessary for the child to complete from the parent process.

@tanriol

This comment has been minimized.

Copy link
Contributor Author

tanriol commented Oct 7, 2019

The slowdown is not to runtime, but to compilation process, so the alternative would be to... spawn rustc, I guess?

@bjorn3

This comment has been minimized.

Copy link
Contributor

bjorn3 commented Oct 7, 2019

Useful trick: When you use - as source file, rustc will read from stdin. Just make sure you close stdin, as rustc will otherwise keep waiting for more input.

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Oct 7, 2019

The slowdown is not to runtime, but to compilation process

Ah yeah, sorry, I didn’t wake up entirely before writing that down. In that case this one is an example of a test that invokes rustc within some arbitrary Rust executable.

@JohnCSimon

This comment has been minimized.

Copy link
Member

JohnCSimon commented Oct 12, 2019

Hello from triage - this has sat idle for the last 5 days.
@eddyb I can't tell if this is waiting to be merged or additional tests from @tanriol
CC: @spastorino

@spastorino

This comment has been minimized.

Copy link
Member

spastorino commented Oct 12, 2019

Seemed to be waiting for additional tests. Looks good to me btw :).

@tanriol

This comment has been minimized.

Copy link
Contributor Author

tanriol commented Oct 12, 2019

Going to add a test in a couple days.

@tmandry

This comment has been minimized.

Copy link
Contributor

tmandry commented Oct 15, 2019

I'm confused why we would add this test in-tree instead of as a benchmark to rustc-perf. (Example PR, btw)

@tanriol

This comment has been minimized.

Copy link
Contributor Author

tanriol commented Oct 16, 2019

@tmandry Thank you for the example.

Is rustc-perf a correct place for, basically, a regression test for error reporting performance in a very specific case of a huge generated function with borrow-check errors? It feels to me a little heavyweight for such a small problem (cluttering of the "all benchmarks" view with an additional graph, for example).

Basically, my feeling was that rustc-perf / perf.rust-lang.org is a human-oriented resource, not an automated testing oriented one, and the exact performance of this use case is not something worth permanent human attention. However, if you consider this position wrong, please tell me so and I'll add the test there.

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Oct 16, 2019

rustc perf tests will not block landing and will only test retroactively, which is not terrible, but not ideal for issues like these either.

That being said glandium suggests there are some sort of a "failure test" mode in rustc-perf? In that case it sounds like a fairly good compromise.

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Oct 16, 2019

@bors r+

I am approving this PR for landing due to the large improvement on perf that this gives, with an expectation that a regression test is added as a follow-up PR (either to rustc itself or to rustc-perf or somewhere else entirely?)

To track this follow-up work, I am a) removing the Fixes #65131 annotation so that the issue does not get closed and tagging the issue with E-needs-test.

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 16, 2019

📌 Commit 4cc707c has been approved by nagisa

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 16, 2019

💡 This pull request was already approved, no need to approve it again.

  • There's another pull request that is currently being tested, blocking this pull request: #65453
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 16, 2019

📌 Commit 4cc707c has been approved by nagisa

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 16, 2019

⌛️ Testing commit 4cc707c with merge 53aca55...

bors added a commit that referenced this pull request Oct 16, 2019
… r=nagisa

use precalculated dominators in explain_borrow

This looks like the only place calculating dominators from the MIR body every time instead of using the ones stored on the `MirBorrowckCtxt`. For example, in #65131 a big generated function with a number of borrowck errors takes a few hours(!) recalculating the dominators while explaining the errors.

I don't know enough about this part of rustc codebase to know for sure that this change is correct, but no tests seem to fail as a result of this change in local testing.
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Oct 16, 2019

☀️ Test successful - checks-azure
Approved by: nagisa
Pushing 53aca55 to master...

@bors bors added the merged-by-bors label Oct 16, 2019
@bors bors merged commit 4cc707c into rust-lang:master Oct 16, 2019
5 checks passed
5 checks passed
homu Test successful
Details
pr Build #20191006.63 succeeded
Details
pr (Linux mingw-check) Linux mingw-check succeeded
Details
pr (Linux x86_64-gnu-llvm-6.0) Linux x86_64-gnu-llvm-6.0 succeeded
Details
pr (LinuxTools) LinuxTools succeeded
Details
@pnkfelix

This comment has been minimized.

Copy link
Member

pnkfelix commented Oct 17, 2019

Discussed at T-compiler meeting. Accepted for beta backport.

bors added a commit that referenced this pull request Oct 23, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
bors added a commit that referenced this pull request Oct 24, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
bors added a commit that referenced this pull request Oct 24, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
bors added a commit that referenced this pull request Oct 24, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
bors added a commit that referenced this pull request Oct 24, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
bors added a commit that referenced this pull request Oct 25, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
bors added a commit that referenced this pull request Oct 25, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
bors added a commit that referenced this pull request Oct 25, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
 *  [beta] Revert "Auto merge of #62948 - matklad:failable-file-loading, r=petro… #65273
 *  save-analysis: Don't ICE when resolving qualified type paths in struct members #65353
 *  save-analysis: Nest tables when processing impl block definitions #65511
 *  Avoid ICE when checking `Destination` of `break` inside a closure #65518
 *  Avoid ICE when adjusting bad self ty #65755
 *  workaround msys2 bug #65762
bors added a commit that referenced this pull request Oct 25, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
 *  [beta] Revert "Auto merge of #62948 - matklad:failable-file-loading, r=petro… #65273
 *  save-analysis: Don't ICE when resolving qualified type paths in struct members #65353
 *  save-analysis: Nest tables when processing impl block definitions #65511
 *  Avoid ICE when checking `Destination` of `break` inside a closure #65518
 *  Avoid ICE when adjusting bad self ty #65755
 *  workaround msys2 bug #65762
bors added a commit that referenced this pull request Oct 26, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
 *  [beta] Revert "Auto merge of #62948 - matklad:failable-file-loading, r=petro… #65273
 *  save-analysis: Don't ICE when resolving qualified type paths in struct members #65353
 *  save-analysis: Nest tables when processing impl block definitions #65511
 *  Avoid ICE when checking `Destination` of `break` inside a closure #65518
 *  Avoid ICE when adjusting bad self ty #65755
 *  workaround msys2 bug #65762
bors added a commit that referenced this pull request Oct 26, 2019
[beta] backport rollup

This includes a bunch of PRs:
 *  Fix redundant semicolon lint interaction with proc macro attributes #64387
 *  Upgrade async/await to "used" keywords. #64875
 *  syntax: fix dropping of attribute on first param of non-method assocated fn #64894
 *  async/await: improve not-send errors #64895
 *  Silence unreachable code lint from await desugaring #64930
 *  Always mark rust and rust-call abi's as unwind #65020
 *  Account for macro invocation in `let mut $pat` diagnostic. #65123
 *  Ensure that associated `async fn`s have unique fresh param names #65142
 *  Add troubleshooting section to PGO chapter in rustc book. #65402
 *  Upgrade GCC to 8.3.0, glibc to 1.17.0 and crosstool-ng to 1.24.0 for dist-armv7-linux #65302
 *  Optimize `try_expand_impl_trait_type` #65293
 *  use precalculated dominators in explain_borrow #65172
 *  Fix ICE #64964 #64989
 *  [beta] Revert "Auto merge of #62948 - matklad:failable-file-loading, r=petro… #65273
 *  save-analysis: Don't ICE when resolving qualified type paths in struct members #65353
 *  save-analysis: Nest tables when processing impl block definitions #65511
 *  Avoid ICE when checking `Destination` of `break` inside a closure #65518
 *  Avoid ICE when adjusting bad self ty #65755
 *  workaround msys2 bug #65762
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.