New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up item_bodies for large match statements involving regions #57494

Merged
merged 2 commits into from Jan 13, 2019

Conversation

Projects
None yet
8 participants
@dotdash
Copy link
Contributor

dotdash commented Jan 10, 2019

These changes don't change anything about the complexity of the algorithms, but use some easy shortcuts or modifications to cut down some overhead.

The first change, which incrementally removes the constraints from the set we're iterating over probably introduces some overhead for small to medium sized constraint sets, but it's not big enough for me to observe it in any project I tested against (not that many though).

Though most other crates probably won't improve much at all, because huge matches aren't that common, the changes seemed simple enough for me to make them.

Ref #55528

cc unicode-rs/unicode-normalization#29

r? @nikomatsakis

Drop "solved" constraints during region expansion
Once a region has been expanded to cover a fixed region, a corresponding
RegSubVar constraint won't have any effect on the expansion anymore, the
same is true for constraints where the variable on the RHS has already
reached static scope. By removing those constraints from the set that
we're iterating over, we remove a lot of needless overhead in case of
slow convergences (i.e. lots of iterations).

For the unicode_normalization crate, this about cuts the time required
for item_bodies checking in half.
@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

rust-highfive commented Jan 10, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:0934da20:start=1547147821963856316,finish=1547147824303879101,duration=2340022785
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---
[01:02:24] .......................i............................................................................ 2100/5298
[01:02:28] .................................................................................................... 2200/5298
[01:02:32] .................................................................................................... 2300/5298
[01:02:36] .................................................................................................... 2400/5298
[01:02:40] ...............................F.................................................................... 2500/5298
[01:02:48] .................................................................................................... 2700/5298
[01:02:52] .................................................................................................... 2800/5298
[01:02:56] .................................................................................................... 2900/5298
[01:02:59] .................................................................................................... 3000/5298
---
[01:03:24] ....................................ii.............................................................. 3700/5298
[01:03:26] ......................................................i............................................. 3800/5298
[01:03:28] .................................................................................................... 3900/5298
[01:03:30] ..........i......................................................................................... 4000/5298
[01:03:34] .......................................F............................................................ 4100/5298
[01:03:48] .................................................................................................... 4300/5298
[01:03:51] .................................................................................................... 4400/5298
[01:03:55] .......................................................i............................................ 4500/5298
[01:04:01] .................................................................................................... 4600/5298
---
[01:04:24] ---- [ui] ui/issues/issue-30240-b.rs stdout ----
[01:04:24] 
[01:04:24] error: ui test compiled successfully!
[01:04:24] status: exit code: 0
[01:04:24] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/issues/issue-30240-b.rs" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/issues/issue-30240-b/a" "-Crpath" "-O" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/issues/issue-30240-b/auxiliary" "-A" "unused"
[01:04:24] ------------------------------------------
[01:04:24] 
[01:04:24] ------------------------------------------
[01:04:24] stderr:
---
[01:04:24] ---- [ui] ui/pattern/slice-pattern-const-3.rs stdout ----
[01:04:24] 
[01:04:24] error: ui test compiled successfully!
[01:04:24] status: exit code: 0
[01:04:24] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/pattern/slice-pattern-const-3.rs" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/pattern/slice-pattern-const-3/a" "-Crpath" "-O" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/pattern/slice-pattern-const-3/auxiliary" "-A" "unused"
[01:04:24] ------------------------------------------
[01:04:24] 
[01:04:24] ------------------------------------------
[01:04:24] stderr:
---
[01:04:24] 
[01:04:24] thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:495:22
[01:04:24] 
[01:04:24] 
[01:04:24] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:04:24] 
[01:04:24] 
[01:04:24] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:04:24] Build completed unsuccessfully in 0:04:12
[01:04:24] Build completed unsuccessfully in 0:04:12
[01:04:24] make: *** [check] Error 1
[01:04:24] Makefile:48: recipe for target 'check' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:03a32fd4
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Thu Jan 10 20:21:39 UTC 2019
---
travis_time:end:198eb9d3:start=1547151700816659387,finish=1547151700821657444,duration=4998057
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0b86a790
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|');

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Add a fast path for identical regions in lub_concrete_regions
In functions with lots of region constraint, if the fixed point
iteration converges only slowly, a lot of the var/var constraints will
have equal regions most of the time. Yet, we still perform the LUB
calculation and try to intern the result. Especially the latter incurs
quite some overhead.

This reduces the take taken by the item bodies checking pass for the
unicode_normalization crate by about 75%.

@dotdash dotdash force-pushed the dotdash:expand branch from d3153f8 to 5f402b8 Jan 10, 2019

@dotdash dotdash changed the title Speed up item_bodies and match check for large match statements Speed up item_bodies for large match statements involving regions Jan 10, 2019

@dotdash

This comment has been minimized.

Copy link
Contributor

dotdash commented Jan 10, 2019

Removed the commit that modifies the match checking code for now, as that caused a test failure.

@lzutao

This comment has been minimized.

Copy link

lzutao commented Jan 10, 2019

@bors try

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 10, 2019

@lzutao: 🔑 Insufficient privileges: not in try users

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

Mark-Simulacrum commented Jan 10, 2019

@bors try

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 10, 2019

⌛️ Trying commit 5f402b8 with merge 960c730...

bors added a commit that referenced this pull request Jan 10, 2019

Auto merge of #57494 - dotdash:expand, r=<try>
Speed up item_bodies for large match statements involving regions

These changes don't change anything about the complexity of the algorithms, but use some easy shortcuts or modifications to cut down some overhead.

The first change, which incrementally removes the constraints from the set we're iterating over probably introduces some overhead for small to medium sized constraint sets, but it's not big enough for me to observe it in any project I tested against (not that many though).

Though most other crates probably won't improve much at all, because huge matches aren't that common, the changes seemed simple enough for me to make them.

Ref #55528

cc unicode-rs/unicode-normalization#29

r? @nikomatsakis
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 11, 2019

☀️ Test successful - checks-travis
State: approved= try=True

@lzutao

This comment has been minimized.

Copy link

lzutao commented Jan 11, 2019

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Jan 11, 2019

Insufficient permissions to issue commands to rust-timer.

@dotdash

This comment has been minimized.

Copy link
Contributor

dotdash commented Jan 11, 2019

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Jan 11, 2019

Insufficient permissions to issue commands to rust-timer.

@nagisa

This comment has been minimized.

Copy link
Contributor

nagisa commented Jan 11, 2019

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Jan 11, 2019

Success: Queued 960c730 with parent 6ecad33, comparison URL.

@rust-timer

This comment has been minimized.

Copy link

rust-timer commented Jan 11, 2019

Finished benchmarking try commit 960c730

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

nikomatsakis commented Jan 11, 2019

@bors r+

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 11, 2019

📌 Commit 5f402b8 has been approved by nikomatsakis

pietroalbini added a commit to pietroalbini/rust that referenced this pull request Jan 12, 2019

Rollup merge of rust-lang#57494 - dotdash:expand, r=nikomatsakis
Speed up item_bodies for large match statements involving regions

These changes don't change anything about the complexity of the algorithms, but use some easy shortcuts or modifications to cut down some overhead.

The first change, which incrementally removes the constraints from the set we're iterating over probably introduces some overhead for small to medium sized constraint sets, but it's not big enough for me to observe it in any project I tested against (not that many though).

Though most other crates probably won't improve much at all, because huge matches aren't that common, the changes seemed simple enough for me to make them.

Ref rust-lang#55528

cc unicode-rs/unicode-normalization#29

r? @nikomatsakis

bors added a commit that referenced this pull request Jan 12, 2019

Auto merge of #57554 - pietroalbini:rollup, r=pietroalbini
Rollup of 15 pull requests

Successful merges:

 - #57351 (Don't actually create a full MIR stack frame when not needed)
 - #57353 (Optimise floating point `is_finite` (2x) and `is_infinite` (1.6x).)
 - #57392 (Always calculate glob map but only for glob uses)
 - #57412 (Improve the wording)
 - #57436 (save-analysis: use a fallback when access levels couldn't be computed)
 - #57442 (Simplify `ConstValue::ScalarPair`)
 - #57453 (lldb_batchmode.py: try `import _thread` for Python 3)
 - #57454 (Some cleanups for core::fmt)
 - #57461 (Change `String` to `&'static str` in `ParseResult::Failure`.)
 - #57473 (std: Render large exit codes as hex on Windows)
 - #57474 (save-analysis: Get path def from parent in case there's no def for the path itself.)
 - #57494 (Speed up item_bodies for large match statements involving regions)
 - #57496 (re-do docs for core::cmp)
 - #57508 (rustdoc: Allow inlining of reexported crates and crate items)
 - #57547 (Use `ptr::eq` where applicable)

Failed merges:

r? @ghost

Centril added a commit to Centril/rust that referenced this pull request Jan 13, 2019

Rollup merge of rust-lang#57494 - dotdash:expand, r=nikomatsakis
Speed up item_bodies for large match statements involving regions

These changes don't change anything about the complexity of the algorithms, but use some easy shortcuts or modifications to cut down some overhead.

The first change, which incrementally removes the constraints from the set we're iterating over probably introduces some overhead for small to medium sized constraint sets, but it's not big enough for me to observe it in any project I tested against (not that many though).

Though most other crates probably won't improve much at all, because huge matches aren't that common, the changes seemed simple enough for me to make them.

Ref rust-lang#55528

cc unicode-rs/unicode-normalization#29

r? @nikomatsakis

bors added a commit that referenced this pull request Jan 13, 2019

Auto merge of #57561 - Centril:rollup, r=Centril
Rollup of 15 pull requests

Successful merges:

 - #57351 (Don't actually create a full MIR stack frame when not needed)
 - #57353 (Optimise floating point `is_finite` (2x) and `is_infinite` (1.6x).)
 - #57392 (Always calculate glob map but only for glob uses)
 - #57412 (Improve the wording)
 - #57436 (save-analysis: use a fallback when access levels couldn't be computed)
 - #57453 (lldb_batchmode.py: try `import _thread` for Python 3)
 - #57454 (Some cleanups for core::fmt)
 - #57461 (Change `String` to `&'static str` in `ParseResult::Failure`.)
 - #57473 (std: Render large exit codes as hex on Windows)
 - #57474 (save-analysis: Get path def from parent in case there's no def for the path itself.)
 - #57494 (Speed up item_bodies for large match statements involving regions)
 - #57496 (re-do docs for core::cmp)
 - #57508 (rustdoc: Allow inlining of reexported crates and crate items)
 - #57547 (Use `ptr::eq` where applicable)
 - #57560 (hygiene: Do not treat `Self` ctor as a local variable)

Failed merges:

r? @ghost

Centril added a commit to Centril/rust that referenced this pull request Jan 13, 2019

Rollup merge of rust-lang#57494 - dotdash:expand, r=nikomatsakis
Speed up item_bodies for large match statements involving regions

These changes don't change anything about the complexity of the algorithms, but use some easy shortcuts or modifications to cut down some overhead.

The first change, which incrementally removes the constraints from the set we're iterating over probably introduces some overhead for small to medium sized constraint sets, but it's not big enough for me to observe it in any project I tested against (not that many though).

Though most other crates probably won't improve much at all, because huge matches aren't that common, the changes seemed simple enough for me to make them.

Ref rust-lang#55528

cc unicode-rs/unicode-normalization#29

r? @nikomatsakis

bors added a commit that referenced this pull request Jan 13, 2019

Auto merge of #57568 - Centril:rollup, r=Centril
Rollup of 16 pull requests

Successful merges:

 - #57351 (Don't actually create a full MIR stack frame when not needed)
 - #57353 (Optimise floating point `is_finite` (2x) and `is_infinite` (1.6x).)
 - #57412 (Improve the wording)
 - #57436 (save-analysis: use a fallback when access levels couldn't be computed)
 - #57453 (lldb_batchmode.py: try `import _thread` for Python 3)
 - #57454 (Some cleanups for core::fmt)
 - #57461 (Change `String` to `&'static str` in `ParseResult::Failure`.)
 - #57473 (std: Render large exit codes as hex on Windows)
 - #57474 (save-analysis: Get path def from parent in case there's no def for the path itself.)
 - #57494 (Speed up item_bodies for large match statements involving regions)
 - #57496 (re-do docs for core::cmp)
 - #57508 (rustdoc: Allow inlining of reexported crates and crate items)
 - #57547 (Use `ptr::eq` where applicable)
 - #57557 (resolve: Mark extern crate items as used in more cases)
 - #57560 (hygiene: Do not treat `Self` ctor as a local variable)
 - #57564 (Update the const fn tracking issue to the new metabug)

Failed merges:

r? @ghost

bors added a commit that referenced this pull request Jan 13, 2019

Auto merge of #57568 - Centril:rollup, r=Centril
Rollup of 16 pull requests

Successful merges:

 - #57351 (Don't actually create a full MIR stack frame when not needed)
 - #57353 (Optimise floating point `is_finite` (2x) and `is_infinite` (1.6x).)
 - #57412 (Improve the wording)
 - #57436 (save-analysis: use a fallback when access levels couldn't be computed)
 - #57453 (lldb_batchmode.py: try `import _thread` for Python 3)
 - #57454 (Some cleanups for core::fmt)
 - #57461 (Change `String` to `&'static str` in `ParseResult::Failure`.)
 - #57473 (std: Render large exit codes as hex on Windows)
 - #57474 (save-analysis: Get path def from parent in case there's no def for the path itself.)
 - #57494 (Speed up item_bodies for large match statements involving regions)
 - #57496 (re-do docs for core::cmp)
 - #57508 (rustdoc: Allow inlining of reexported crates and crate items)
 - #57547 (Use `ptr::eq` where applicable)
 - #57557 (resolve: Mark extern crate items as used in more cases)
 - #57560 (hygiene: Do not treat `Self` ctor as a local variable)
 - #57564 (Update the const fn tracking issue to the new metabug)

Failed merges:

r? @ghost

@bors bors merged commit 5f402b8 into rust-lang:master Jan 13, 2019

1 check passed

homu Test successful
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment