Fix exponential projection complexity on nested types #48296

ishitatsuyuki · 2018-02-17T14:27:43Z

This implements solution 1 from #38528 (comment).

The code quality was extremely poor, but it has improved during review.

Blocking issues:

we probably don't want a quadratic deduplication for obligations.
is there an alternative to deduplication?

Based on #48315.

Needs changelog. Noticable improvement on compile time is expected.

Fix #38528
Close #39684
Close #43757

rust-highfive · 2018-02-17T14:27:53Z

r? @nikomatsakis

(rust_highfive has picked a reviewer for you, use r? to override)

ishitatsuyuki

Here are some self-reminders.

ishitatsuyuki · 2018-02-17T15:14:05Z

src/librustc/traits/project.rs

@@ -818,55 +822,74 @@ fn project_type<'cx, 'gcx, 'tcx>(
                                       &obligation_trait_ref,
                                       &mut candidates);

+    let decide_commit = |candidates: &mut ProjectionTyCandidateSet<'tcx>| {


I have no way but to reorder this before the error handling. It looks non intuitive.

ishitatsuyuki · 2018-02-17T15:14:05Z

src/librustc/traits/project.rs

+            });
+            debug!("resulting candidate set: {:?}", candidates.vec);
+            if candidates.vec.len() != 1 {
+                candidates.ambiguous = true;


This is nearly a hack.

ishitatsuyuki · 2018-02-17T15:14:05Z

src/librustc/traits/project.rs

            }
        };

-        match vtable {
+        match vtable.clone() {


This copy can be avoided.

ishitatsuyuki · 2018-02-18T03:37:00Z

The other part of the blowup (typeck related) is yet to be solved, but I think the code is ready for a round of review. I have removed whitespace changes and clarified the description.

ishitatsuyuki · 2018-02-18T04:01:32Z

@kennytm A reminder for you to change the review labels.

kennytm · 2018-02-18T07:20:58Z

@bors try

Let’s do a perf check while we’re waiting for review :)

bors · 2018-02-18T07:21:09Z

⌛ Trying commit 3d81698 with merge e35a327...

[wip] Fix exponential projection complexity on nested types This implements solution 1 from #38528 (comment). The code quality is currently extremely poor, but we can improve them during review. Based on #48315. Needs changelog. Noticable improvement on compile time is expected. Fix #38528 (in progress, another exponential blowup exist) Close #39684 Close #43757

kennytm · 2018-02-18T07:22:13Z

@ishitatsuyuki please remove the [wip] from the PR title if it is ready for review.

ishitatsuyuki · 2018-02-18T07:24:25Z

Well, I'm currently tackling another part of the exponential issue (so this is WIP), but I need comments on improving the code organization.

bors · 2018-02-18T09:32:00Z

☀️ Test successful - status-travis
State: approved= try=True

ishitatsuyuki · 2018-02-18T12:26:52Z

By deduplicating the obligations the issue is completely resolved. I'm not sure if this approach is the best way though, and I copied the dumb deduplication becuase I didn't want to implement Ord everywhere. I will fix them later.

ishitatsuyuki · 2018-02-18T12:28:12Z

@kennytm I need another try.

Mark-Simulacrum · 2018-02-18T13:29:24Z

@bors try

bors · 2018-02-18T13:29:34Z

⌛ Trying commit 4a40c55 with merge 4ac2425...

ishitatsuyuki · 2018-02-22T22:26:57Z

r? @nikomatsakis

Please change the label...

nikomatsakis

Looks great =)

nikomatsakis · 2018-02-22T23:22:09Z

src/librustc/traits/project.rs

+        } else {
+            Err(())
+        }
+    });


I like the change to push errors into the candidate set. This is less confusing this way.

I'm not sure if I understand. I already handle SelectionError above, and these Ok and Err is the return value to commit_if_ok.

nikomatsakis · 2018-02-22T23:22:09Z

src/librustc/traits/select.rs

+        // unknown. What we know is that the deduplication avoids exponential
+        // amount of predicates being propogated when processing deeply nested
+        // types.
+        let mut seen = FxHashSet();


Oh, it seems like duplicates could come from many sources -- in particular, normalization might introduce the same obligation a few times.

nikomatsakis · 2018-02-22T23:29:32Z

@bors r+

@ishitatsuyuki -- it'd be great to add at least one of these exponential blow-up cases to the perf website so we can monitor for regressions. Perhaps @Mark-Simulacrum can walk you through it.

bors · 2018-02-22T23:29:33Z

📌 Commit 5a2bec9 has been approved by nikomatsakis

…sakis Fix exponential projection complexity on nested types This implements solution 1 from rust-lang#38528 (comment). The code quality is currently extremely poor, but we can improve them during review. Blocking issues: - we probably don't want a quadratic deduplication for obligations. - is there an alternative to deduplication? Based on rust-lang#48315. Needs changelog. Noticable improvement on compile time is expected. Fix rust-lang#38528 Close rust-lang#39684 Close rust-lang#43757

Rollup of 15 pull requests - Successful merges: #47689, #48110, #48197, #48296, #48386, #48392, #48404, #48415, #48441, #48448, #48452, #48481, #48490, #48499, #48503 - Failed merges:

This effectively reverts rust-lang#43546 as it seems that it does affect performance more than the PR has anticipated. This also removes the deduplication code from rust-lang#48296 as duplications were primarily coming from rust-lang#43546 and after removing that code it probably doesn't worth paying the cost of using a hash map.

Revert rust-lang#43546 This effectively reverts rust-lang#43546 as it seems that it does affect performance more than the PR has anticipated. This also removes the deduplication code from rust-lang#48296 as duplications were primarily coming from rust-lang#43546 and after removing that code it probably doesn't worth paying the cost of using a hash map. Interestingly, the regression tests introduced in rust-lang#43546 didn't fail after reverting, but some other tests had breakage. For now I just applied `--bless`, but I'd like to figure out if it could be a problem later on. Preliminary perf results confirms that we no longer need deduplication to avoid exponential behavior. ![image](https://user-images.githubusercontent.com/12389383/94511110-64b42200-0253-11eb-9dec-1f2632783719.png) This PR will be draft until I figure out the potential breakage. r? `@ghost` Need a perf run. A crater run will help as well to identify potential breakage.

This effectively reverts rust-lang#43546 as it seems that it does affect performance more than the PR has anticipated. Follow-up changes from rust-lang#44269 are also reverted. This also removes the deduplication code from rust-lang#48296 as duplications were primarily coming from rust-lang#43546 and after removing that code it probably doesn't worth paying the cost of using a hash map.

rust-highfive assigned nikomatsakis Feb 17, 2018

rust-highfive added the S-waiting-on-review label Feb 17, 2018

ishitatsuyuki changed the title ~~[wip] (PoC) Fix exponential blowup on nested types~~ [wip] (PoC) Fix exponential projection complexity on nested types Feb 17, 2018

ishitatsuyuki force-pushed the ishitatsuyuki:exp-unblow branch from 0b84c09 to 91d7792 Feb 17, 2018

This was referenced Feb 17, 2018

Compiler hangs during recursive monomorphisation #39684

Closed

Nonlinear compile time blow-up with deeply nested types #38528

Closed

ishitatsuyuki reviewed Feb 17, 2018

View changes

kennytm added S-waiting-on-author and removed S-waiting-on-review labels Feb 17, 2018

ishitatsuyuki force-pushed the ishitatsuyuki:exp-unblow branch from 91d7792 to 365bf21 Feb 18, 2018

ishitatsuyuki changed the title ~~[wip] (PoC) Fix exponential projection complexity on nested types~~ [wip] Fix exponential projection complexity on nested types Feb 18, 2018

ishitatsuyuki force-pushed the ishitatsuyuki:exp-unblow branch from 365bf21 to 3d81698 Feb 18, 2018

kennytm added S-waiting-on-review and removed S-waiting-on-author labels Feb 18, 2018

ishitatsuyuki changed the title ~~[wip] Fix exponential projection complexity on nested types~~ Fix exponential projection complexity on nested types Feb 18, 2018

ishitatsuyuki mentioned this pull request Feb 18, 2018

cache the results of type projection and normalization #20304

Open

Mark-Simulacrum added the relnotes label Feb 18, 2018

kennytm added S-waiting-on-review and removed S-waiting-on-author labels Feb 22, 2018

nikomatsakis approved these changes Feb 22, 2018

View changes

bors added S-waiting-on-bors and removed S-waiting-on-review labels Feb 22, 2018

Manishearth mentioned this pull request Feb 24, 2018

Rollup of 15 pull requests #48520

Merged

bors merged commit 5a2bec9 into rust-lang:master Feb 25, 2018
1 check passed

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

ishitatsuyuki mentioned this pull request Feb 26, 2018

type checker takes O(~1.5^recursion_limit) time to reject simple-ish code #40353

Open

Antti mentioned this pull request Feb 26, 2018

Compiler panic while building alga #48542

Closed

ishitatsuyuki mentioned this pull request Feb 26, 2018

Unexpected panic with cgmath #48551

Closed

ishitatsuyuki deleted the ishitatsuyuki:exp-unblow branch Mar 1, 2018

ishitatsuyuki mentioned this pull request Mar 1, 2018

Add benchmark for deeply nested types rust-lang/rustc-perf#185

Merged

cuviper mentioned this pull request Apr 3, 2020

Implement Chain with Fuses #70332

Closed

ishitatsuyuki mentioned this pull request Sep 29, 2020

Revert #43546 "save subobligations in the projection cache" #77325

Open

rust-lang / rust

Fix exponential projection complexity on nested types #48296

Fix exponential projection complexity on nested types #48296

ishitatsuyuki commented Feb 17, 2018 •

edited

rust-highfive commented Feb 17, 2018

ishitatsuyuki left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

ishitatsuyuki commented Feb 18, 2018

ishitatsuyuki commented Feb 18, 2018

kennytm commented Feb 18, 2018

bors commented Feb 18, 2018

kennytm commented Feb 18, 2018

ishitatsuyuki commented Feb 18, 2018

bors commented Feb 18, 2018

ishitatsuyuki commented Feb 18, 2018

ishitatsuyuki commented Feb 18, 2018

Mark-Simulacrum commented Feb 18, 2018

bors commented Feb 18, 2018

ishitatsuyuki commented Feb 22, 2018

nikomatsakis left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

nikomatsakis commented Feb 22, 2018

bors commented Feb 22, 2018

rust-lang / rust

Fix exponential projection complexity on nested types #48296

Fix exponential projection complexity on nested types #48296

Conversation

ishitatsuyuki commented Feb 17, 2018 • edited

rust-highfive commented Feb 17, 2018

ishitatsuyuki left a comment

This comment has been minimized.

ishitatsuyuki Feb 17, 2018 Author Contributor

This comment has been minimized.

ishitatsuyuki Feb 17, 2018 Author Contributor

This comment has been minimized.

ishitatsuyuki Feb 17, 2018 Author Contributor

ishitatsuyuki commented Feb 18, 2018

ishitatsuyuki commented Feb 18, 2018

kennytm commented Feb 18, 2018

bors commented Feb 18, 2018

kennytm commented Feb 18, 2018

ishitatsuyuki commented Feb 18, 2018

bors commented Feb 18, 2018

ishitatsuyuki commented Feb 18, 2018

ishitatsuyuki commented Feb 18, 2018

Mark-Simulacrum commented Feb 18, 2018

bors commented Feb 18, 2018

ishitatsuyuki commented Feb 22, 2018

nikomatsakis left a comment

This comment has been minimized.

nikomatsakis Feb 22, 2018 Contributor

This comment has been minimized.

ishitatsuyuki Feb 22, 2018 Author Contributor

This comment has been minimized.

nikomatsakis Feb 22, 2018 Contributor

nikomatsakis commented Feb 22, 2018

bors commented Feb 22, 2018

ishitatsuyuki commented Feb 17, 2018 •

edited

ishitatsuyuki Feb 17, 2018
Author Contributor

ishitatsuyuki Feb 17, 2018
Author Contributor

ishitatsuyuki Feb 17, 2018
Author Contributor

nikomatsakis Feb 22, 2018
Contributor

ishitatsuyuki Feb 22, 2018
Author Contributor

nikomatsakis Feb 22, 2018
Contributor