Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustc runs out of memory when compiling trait heavy code #43613

Closed
Marwes opened this issue Aug 2, 2017 · 23 comments · Fixed by #44269
Closed

rustc runs out of memory when compiling trait heavy code #43613

Marwes opened this issue Aug 2, 2017 · 23 comments · Fixed by #44269
Assignees
Labels
C-bug Category: This is a bug. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. P-high High priority regression-from-stable-to-beta Performance or correctness regression from stable to beta. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Milestone

Comments

@Marwes
Copy link
Contributor

Marwes commented Aug 2, 2017

Been working on an update to combine and ran into #43132. Since the fix has been merged I updated using rustup but now rustc runs out of memory when compiling instead.

This branch reproduces the issue https://github.com/Marwes/combine/tree/ice_allocator_exhausted. I haven't bothered to reduce to a minimal production given that it is likely related to the slow down mentioned in #43132 and removing code will just make it compile and make the problem less pronounced (possibly the minimal example here reproduces the same problem as well https://github.com/rust-lang/rust/pull/32062/files#diff-6a2bc45424fb5e8e0f7eb99f2e05f9cf).

$ rustc --version --verbose
rustc 1.21.0-nightly (dd53dd5f9 2017-08-01)
binary: rustc
commit-hash: dd53dd5f9e21dce1fbc06b7f9f451d1009bdcfd8
commit-date: 2017-08-01
host: x86_64-pc-windows-msvc
release: 1.21.0-nightly
LLVM version: 4.0
@Mark-Simulacrum Mark-Simulacrum added C-bug Category: This is a bug. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 3, 2017
@Marwes
Copy link
Contributor Author

Marwes commented Aug 4, 2017

EDIT I may have been mistaken about the example from #32062. While compilation takes far longer than one would expect, that has likely always been the case as well (still a potential issue but perhaps not a reproduction for the one I encountered).

The example added for #32062 reproduces as long as few more calls to chain gets added

// Copyright 2016 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.

// pretty-expanded FIXME #23616

fn main() {
    let _ = test(Some(0).into_iter());
}

trait Parser {
    type Input: Iterator;
    type Output;
    fn parse(self, input: Self::Input) -> Result<(Self::Output, Self::Input), ()>;
    fn chain<P>(self, p: P) -> Chain<Self, P> where Self: Sized {
        Chain(self, p)
    }
}

struct Token<T>(T::Item) where T: Iterator;

impl<T> Parser for Token<T> where T: Iterator {
    type Input = T;
    type Output = T::Item;
    fn parse(self, _input: Self::Input) -> Result<(Self::Output, Self::Input), ()> {
        Err(())
    }
}

struct Chain<L, R>(L, R);

impl<L, R> Parser for Chain<L, R> where L: Parser, R: Parser<Input = L::Input> {
    type Input = L::Input;
    type Output = (L::Output, R::Output);
    fn parse(self, _input: Self::Input) -> Result<(Self::Output, Self::Input), ()> {
        Err(())
    }
}

fn test<I>(i: I) -> Result<((), I), ()> where I: Iterator<Item = i32> {
    Chain(Token(0), Token(1))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .chain(Chain(Token(0), Token(1)))
        .parse(i)
        .map(|(_, i)| ((), i))
}

@alexcrichton
Copy link
Member

@Marwes did this compile previously? Just wondering if we can tag it as a regression!

@aturon
Copy link
Member

aturon commented Aug 9, 2017

cc @arielb1 for potential regression tracking

@Marwes
Copy link
Contributor Author

Marwes commented Aug 10, 2017

@alexcrichton I am afraid the the example from #32062 would fail to compile before as well so it is likely unrelated. https://github.com/Marwes/combine/tree/ice_allocator_exhausted did not compile before due to #43132 so this is probably not a regression.

@jonhoo
Copy link
Contributor

jonhoo commented Aug 22, 2017

A tangentially related comment: on nightly, no error is printed when the compiler fails this way beyond Could not compile "foo".. On beta the situation is a little better, as it at least prints fatal runtime error: allocator memory exhausted.

@alexcrichton
Copy link
Member

Ah unfortunately this is a beta regression. Compiling this source I get:

$ rustc -V
rustc 1.19.0 (0ade33941 2017-07-17)
$ time rustc foo.rs
rustc foo.rs  3.00s user 0.02s system 99% cpu 3.040 total
$ rustc +beta -V
rustc 1.20.0-beta.2 (54279dfac 2017-08-13)
$ time rustc +beta foo.rs
rustc +beta foo.rs  6.17s user 1.26s system 99% cpu 7.437 total

@alexcrichton alexcrichton added the regression-from-stable-to-beta Performance or correctness regression from stable to beta. label Aug 23, 2017
@alexcrichton
Copy link
Member

cc @rust-lang/compiler for a new beta regression

@alexcrichton alexcrichton modified the milestone: 1.20 Aug 23, 2017
@nikomatsakis
Copy link
Contributor

triage: P-high

@rust-highfive rust-highfive added the P-high High priority label Aug 23, 2017
@arielb1
Copy link
Contributor

arielb1 commented Aug 24, 2017

@acrichto

We backed out quite a few PRs for beta .3. Could you test again after that?

@alexcrichton
Copy link
Member

Looks like it helped a little but not a huge amount, the beta timing decreased from ~7 to ~5s

It looks like, however, that it may be constant overhead. Commenting out a few more chain calls and stable/beta don't diverge in the difference between them (1-2s).

Should this be closed?

@nikomatsakis nikomatsakis added regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. and removed regression-from-stable-to-beta Performance or correctness regression from stable to beta. labels Aug 24, 2017
@nikomatsakis
Copy link
Contributor

I am reclassifying as a regression against nightly. Still relevant.

@alexcrichton alexcrichton modified the milestones: 1.21, 1.20 Aug 24, 2017
@alexcrichton
Copy link
Member

Ok, switching milestone in that case as well.

Thanks!

@alexcrichton alexcrichton added regression-from-stable-to-beta Performance or correctness regression from stable to beta. and removed regression-from-stable-to-nightly Performance or correctness regression from stable to nightly. labels Aug 28, 2017
@alexcrichton
Copy link
Member

1.21 is now in beta

@nikomatsakis
Copy link
Contributor

We should retest -- @arielb1, have your various improvements landed? I'm assigning to us both to try and investigate.

@nikomatsakis
Copy link
Contributor

OK, so, using nightly, I am able to build the original combine branch, but there is definitely still a loss on beta/nightly relative to stable, both in memory use and time. I'll do a bit more profiling and see what turns up.

lunch-box. for x in stable beta nightly; do echo === $x ===; rustup update $x; /usr/bin/time rustc +$x foo.rs; done
=== stable ===
info: syncing channel updates for 'stable-x86_64-unknown-linux-gnu'

  stable-x86_64-unknown-linux-gnu unchanged - rustc 1.20.0 (f3d6973f4 2017-08-27)

5.97user 0.04system 0:06.02elapsed 100%CPU (0avgtext+0avgdata 121132maxresident)k
0inputs+7680outputs (0major+12912minor)pagefaults 0swaps
=== beta ===
info: syncing channel updates for 'beta-x86_64-unknown-linux-gnu'

  beta-x86_64-unknown-linux-gnu unchanged - rustc 1.21.0-beta.1 (198109911 2017-08-29)

9.99user 1.50system 0:11.49elapsed 100%CPU (0avgtext+0avgdata 387528maxresident)k
0inputs+8056outputs (0major+1408205minor)pagefaults 0swaps
=== nightly ===
info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'

  nightly-x86_64-unknown-linux-gnu unchanged - rustc 1.21.0-nightly (97b01abf3 2017-08-31)

10.16user 1.58system 0:11.74elapsed 100%CPU (0avgtext+0avgdata 368428maxresident)k
0inputs+8008outputs (0major+1709226minor)pagefaults 0swaps

@nikomatsakis
Copy link
Contributor

OK, I did some digging. There is a massive spike (when compiling foo.rs) directly attributable to the projection cache. I have to assume this is due to us now including the obligations in the cache. Will investigate some more, but I suspect one of the improvements that @arielb1 and I have talked about (like skipping the obligations if there are no type variables in the result) might be relevant here.

@nikomatsakis
Copy link
Contributor

Relevant backtrace from memcache report:

99.50% (286,155,244B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->57.95% (166,652,149B) 0x5352FDF: __rdl_alloc (lib.rs:131)
| ->51.10% (146,960,384B) 0x92C8978: _$LT$alloc..raw_vec..RawVec$LT$T$C$$u20$A$GT$$GT$::allocate_in::h79ee2a1009556b6d (heap.rs:84)
| | ->51.10% (146,960,384B) 0x8DCB78E: _$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..clone..Clone$GT$::clone::h9c3d151fd71233a3 (raw_vec.rs:141)
| | | ->40.89% (117,600,896B) 0x92607AC: rustc::traits::project::ProjectionCache::insert_ty::h0cf47cfd76f17eb9 (project.rs:340)
| | | | ->40.89% (117,600,896B) 0x925B678: rustc::traits::project::opt_normalize_projection_type::h1383880954193dab (project.rs:530)
| | | |   ->40.89% (117,600,896B) 0x925A277: rustc::traits::project::normalize_projection_type::hf37038bf35e92756 (project.rs:365)
| | | |   | ->40.89% (117,600,896B) 0x9259FF6: _$LT$rustc..traits..project..AssociatedTypeNormalizer$LT$$u27$a$C$$u20$$u27$b$C$$u20$$u27$gcx$C$$u20$$u27$tcx$GT$$u20$as$u20$rustc..ty..fold..TypeFolder$LT$$
| | | |   |   ->30.69% (88,251,264B) 0x923A220: rustc::ty::structural_impls::_$LT$impl$u20$rustc..ty..fold..TypeFoldable$LT$$u27$tcx$GT$$u20$for$u20$$RF$$u27$tcx$u20$rustc..ty..TyS$LT$$u27$tcx$GT$$GT$::fo
| | | |   |   | ->30.69% (88,251,136B) 0x8F41703: core::ops::function::impls::_$LT$impl$u20$core..ops..function..FnOnce$LT$A$GT$$u20$for$u20$$RF$$u27$a$u20$mut$u20$F$GT$::call_once::h6574094d91ee375f (str
| | | |   |   | | ->30.69% (88,251,136B) 0x8E9412E: _$LT$rustc_data_structures..array_vec..ArrayVec$LT$A$GT$$u20$as$u20$core..iter..traits..Extend$LT$$LT$A$u20$as$u20$rustc_data_structures..array_vec..Arr
| | | |   |   | |   ->30.69% (88,251,136B) 0x9128645: _$LT$rustc_data_structures..accumulate_vec..AccumulateVec$LT$A$GT$$u20$as$u20$core..iter..traits..FromIterator$LT$$LT$A$u20$as$u20$rustc_data_structur
| | | |   |   | |     ->30.69% (88,251,136B) 0x922A1AD: rustc::ty::fold::TypeFoldable::fold_with::he61f397c4c6fc301 (iterator.rs:1302)
| | | |   |   | |       ->20.48% (58,891,648B) 0x9239AB6: rustc::ty::structural_impls::_$LT$impl$u20$rustc..ty..fold..TypeFoldable$LT$$u27$tcx$GT$$u20$for$u20$$RF$$u27$tcx$u20$rustc..ty..TyS$LT$$u27$tcx$G
| | | |   |   | |       | ->20.48% (58,891,648B) 0x9259E9B: _$LT$rustc..traits..project..AssociatedTypeNormalizer$LT$$u27$a$C$$u20$$u27$b$C$$u20$$u27$gcx$C$$u20$$u27$tcx$GT$$u20$as$u20$rustc..ty..fold..Ty
| | | |   |   | |       |   ->20.48% (58,891,648B) 0x923A220: rustc::ty::structural_impls::_$LT$impl$u20$rustc..ty..fold..TypeFoldable$LT$$u27$tcx$GT$$u20$for$u20$$RF$$u27$tcx$u20$rustc..ty..TyS$LT$$u27$t
| | | |   |   | |       |     ->10.26% (29,508,352B) 0x925B47B: rustc::traits::project::opt_normalize_projection_type::h1383880954193dab (project.rs:266)
| | | |   |   | |       |     | ->10.26% (29,508,352B) 0x925A277: rustc::traits::project::normalize_projection_type::hf37038bf35e92756 (project.rs:365)
| | | |   |   | |       |     | | ->10.26% (29,508,352B) 0x9259FF6: _$LT$rustc..traits..project..AssociatedTypeNormalizer$LT$$u27$a$C$$u20$$u27$b$C$$u20$$u27$gcx$C$$u20$$u27$tcx$GT$$u20$as$u20$rustc..ty..
| | | |   |   | |       |     | |   ->10.26% (29,508,352B) 0x923A220: rustc::ty::structural_impls::_$LT$impl$u20$rustc..ty..fold..TypeFoldable$LT$$u27$tcx$GT$$u20$for$u20$$RF$$u27$tcx$u20$rustc..ty..TyS$L

@jonhoo
Copy link
Contributor

jonhoo commented Sep 1, 2017

@nikomatsakis is it reasonable to assume that this and #43789 are really the same issue?

@nikomatsakis
Copy link
Contributor

nikomatsakis commented Sep 1, 2017

I did more investigation. In particular, I tried an experimental idea to reduce the size of the cache as soon as we can. This did reduce, the peak but not by much. Here is the "ideal" memory profile, based on the old caching policy around obligations:

    MB
22.60^                                             :
     |            :@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     |            :@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     |            :@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     |            :@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     |            :@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     |            :@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     |            :@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     |            :@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     |            :@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     |           @:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     | ::@:::::::@:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     | ::@:: :: :@:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     | ::@:: :: :@:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     | ::@:: :: :@:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     | ::@:: :: :@:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     | ::@:: :: :@:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     | ::@:: :: :@:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     | ::@:: :: :@:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
     | ::@:: :: :@:@:::::::@::::::::@::::::::::@:#::::@:::::@::::@:::::@:::::@
   0 +----------------------------------------------------------------------->Gi
     0                                                                   69.01

This is the profile of the current master:

    MB
274.3^                                           #
     |                @                          #
     |                @                          #
     |                @                          #          :               :
     |                @                          #          :               :
     |                @                          #          :               :
     |    @           @                          #          :               :
     |    @           @                          #          :               :
     |    @:::        @                          #          :               :
     |    @::         @                          #          :               :
     |    @::         @                          #          :               :
     |    @::         @                          #::        ::              ::
     |  @@@::         @::                        #:         ::              ::
     |  @ @::         @:                         #:         ::              ::
     |  @ @::         @:                         #:         ::              ::
     |  @ @::         @:                         #:         ::              ::
     | @@ @::         @:          :              #:         ::    :         ::
     | @@ @::         @:          :    :         #:         ::    :         ::
     | @@ @::         @:  :       :    :         #:         ::    :         ::
     |:@@ @::       : @: ::::::::@:::::::::::::::#: ::::::@:::@@:::@:::::::@::
   0 +----------------------------------------------------------------------->Gi
     0                                                                   95.04

Notice the several peaks. Each of those are an "episode" the cache in use.

This is the profile with my patch, which clears the cache obligations after a cache hit in which we observe that there are no unbound type variables in the projection result:

    MB
230.5^      ##
     |      #
     |      #
     |     @#
     |     @#
     |     @#
     |     @#
     |     @#  :
     |     @# ::
     |     @# ::
     |   @ @# ::
     |   @ @# ::
     |   @:@# ::
     |   @:@# ::
     |   @:@# ::
     |  @@:@# ::
     |  @@:@# ::
     |  @@:@# ::
     |  @@:@# ::
     | @@@:@# ::         ::::::::::::::::::@@:::@:::@@:::@::::@:::::@::::::@::
   0 +----------------------------------------------------------------------->Gi
     0                                                                   76.31

You can see it's improved, and the peak is lower in an absolute sense, but there is still a clear peak. I'm done for the day but will poke at this later on.

@nikomatsakis
Copy link
Contributor

nikomatsakis commented Sep 1, 2017

@jonhoo

is it reasonable to assume that this and #43789 are really the same issue?

I think that is very likely.

EDIT: Or some of the EndRegion business.

@nikomatsakis
Copy link
Contributor

OK, I tried a more aggressive pruning pass, and that seems to have worked out pretty well:

    MB
51.89^ ##
     | #
     | #
     | #
     | #
     | #
     | #
     | #
     | #
     | #
     | #
     | #
     | #           ::::::::::::::::::::::::::::::::::::::::::::::::@::::::@:::
     | #           ::: :: :: :::::: ::::::::::::::::::::: : :::::::@::::::@:::
     | #           ::: :: :: :::::: ::::::::::::::::::::: : :::::::@::::::@:::
     | #           ::: :: :: :::::: ::::::::::::::::::::: : :::::::@::::::@:::
     | # ::::::::::::: :: :: :::::: ::::::::::::::::::::: : :::::::@::::::@:::
     | # ::: ::::::::: :: :: :::::: ::::::::::::::::::::: : :::::::@::::::@:::
     | # ::: ::::::::: :: :: :::::: ::::::::::::::::::::: : :::::::@::::::@:::
     | # ::: ::::::::: :: :: :::::: ::::::::::::::::::::: : :::::::@::::::@:::
   0 +----------------------------------------------------------------------->Gi
     0                                                                   69.74

We're still using more memory, but it's a much more mild spike -- and execution time is about the same. Now I just have to make sure that I didn't break anything else along the way. =)

nikomatsakis added a commit to nikomatsakis/rust that referenced this issue Sep 2, 2017
Keep **all** the obligations for every projection is wasteful of
memory and compilation time. We only really care about those
subobligations that may inform the result of the projection (i.e., may
help to resolve any inference variables that appear within).
Therefore, we can clear the subobligations from the cache that don't
potentially affect the result of the projection. On every cache hit,
we also take the opportunity to check if the type variables have been
resolved *yet* and, if so, clear out the pending obligations.

Fixes rust-lang#43613
@nikomatsakis
Copy link
Contributor

PR opened (#44269)

@nikomatsakis
Copy link
Contributor

nikomatsakis commented Sep 2, 2017

The version in the PR prunes mildly less aggressively -- I think my initial patches were incorrect -- but does remember to call shrink_to_fit(), and results in completely eliminating the memory spike:

    MB
22.39^                                :
     |           @::@:::::::::::@::@#::::::@::::::@:::::@:::::@::::@:::::@::::
     |           @::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     |           @::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     |           @::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     |           @::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     |           @::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     |           @::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     |           @::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     |           @::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     |           @::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     | @::::@::::@::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     | @::::@::::@::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     | @::::@::::@::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     | @::::@::::@::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     | @::::@::::@::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     | @::::@::::@::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     | @::::@::::@::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     | @::::@::::@::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
     | @::::@::::@::@:::::::::::@::@#:::: :@::::::@:::::@:::::@::::@:::::@::::
   0 +----------------------------------------------------------------------->Gi
     0                                                                   66.92

bors added a commit that referenced this issue Sep 8, 2017
…ns, r=arielb1

limit and clear cache obligations opportunistically

Keeping **all** the obligations for every projection is wasteful of
memory and compilation time. We only really care about those
subobligations that may inform the result of the projection (i.e., may
help to resolve any inference variables that appear within).
Therefore, we can clear the subobligations from the cache that don't
potentially affect the result of the projection. On every cache hit,
we also take the opportunity to check if the type variables have been
resolved *yet* and, if so, clear out the pending obligations.

Fixes #43613.

r? @arielb1

NB -- not sure how to test for this. Probably we should add the #43613 test case to perf.
nikomatsakis added a commit to nikomatsakis/rust that referenced this issue Sep 28, 2017
Keep **all** the obligations for every projection is wasteful of
memory and compilation time. We only really care about those
subobligations that may inform the result of the projection (i.e., may
help to resolve any inference variables that appear within).
Therefore, we can clear the subobligations from the cache that don't
potentially affect the result of the projection. On every cache hit,
we also take the opportunity to check if the type variables have been
resolved *yet* and, if so, clear out the pending obligations.

Fixes rust-lang#43613
arielb1 added a commit to arielb1/rust that referenced this issue Oct 6, 2017
We want to retain obligations that *contain* inference variables, not
obligations that *don't contain* them, in order to fix rust-lang#43132. Because
of surrounding changes to inference, the ICE doesn't occur in its
original case, but I believe it could still be made to occur on master.

Maybe I should try to write a new test case? Certainly not right now
(I'm mainly trying to get us a beta that we can ship) but maybe before
we land this PR on nightly?

This seems to cause a 10% performance regression in my imprecise
attempt to benchmark item-body checking for rust-lang#43613, but it's better to
be slow and right than fast and wrong. If we want to recover that, I
think we can change the constrained-type-parameter code to actually
give a list of projections that are important for resolving inference
variables and filter everything else out.
nikomatsakis pushed a commit to nikomatsakis/rust that referenced this issue Oct 6, 2017
We want to retain obligations that *contain* inference variables, not
obligations that *don't contain* them, in order to fix rust-lang#43132. Because
of surrounding changes to inference, the ICE doesn't occur in its
original case, but I believe it could still be made to occur on master.

Maybe I should try to write a new test case? Certainly not right now
(I'm mainly trying to get us a beta that we can ship) but maybe before
we land this PR on nightly?

This seems to cause a 10% performance regression in my imprecise
attempt to benchmark item-body checking for rust-lang#43613, but it's better to
be slow and right than fast and wrong. If we want to recover that, I
think we can change the constrained-type-parameter code to actually
give a list of projections that are important for resolving inference
variables and filter everything else out.
bors added a commit that referenced this issue Oct 6, 2017
fix logic error in #44269's `prune_cache_value_obligations`

We want to retain obligations that *contain* inference variables, not
obligations that *don't contain* them, in order to fix #43132. Because
of surrounding changes to inference, the ICE doesn't occur in its
original case, but I believe it could still be made to occur on master.

Maybe I should try to write a new test case? Certainly not right now
(I'm mainly trying to get us a beta that we can ship) but maybe before
we land this PR on nightly?

This seems to cause a 10% performance regression in my imprecise
attempt to benchmark item-body checking for #43613, but it's better to
be slow and right than fast and wrong. If we want to recover that, I
think we can change the constrained-type-parameter code to actually
give a list of projections that are important for resolving inference
variables and filter everything else out.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. I-compilemem Issue: Problems and improvements with respect to memory usage during compilation. P-high High priority regression-from-stable-to-beta Performance or correctness regression from stable to beta. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants