Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

forego caching for all participants in cycles, apart from root node #60444

Merged

Conversation

Projects
None yet
9 participants
@nikomatsakis
Copy link
Contributor

commented May 1, 2019

This is a targeted fix for #60010, which uncovered a pretty bad failure of our caching strategy in the face of coinductive cycles. The problem is explained in the comment in the PR on the new field, in_cycle, but I'll reproduce it here:

Starts out as false -- if, during evaluation, we encounter a
cycle, then we will set this flag to true for all participants
in the cycle (apart from the "head" node). These participants
will then forego caching their results. This is not the most
efficient solution, but it addresses #60010. The problem we
are trying to prevent:

  • If you have A: AutoTrait requires B: AutoTrait and C: NonAutoTrait
  • B: AutoTrait requires A: AutoTrait (coinductive cycle, ok)
  • C: NonAutoTrait requires A: AutoTrait (non-coinductive cycle, not ok)

you don't want to cache that B: AutoTrait or A: AutoTrait
is EvaluatedToOk; this is because they were only considered
ok on the premise that if A: AutoTrait held, but we indeed
encountered a problem (later on) with A: AutoTrait. So we currently set a flag on the stack node for B: AutoTrait(as well as the second instance ofA: AutoTrait`) to supress
caching.

This is a simple, targeted fix. The correct fix requires
deeper changes, but would permit more caching: we could
basically defer caching until we have fully evaluated the
tree, and then cache the entire tree at once.

I'm not sure what the impact of this fix will be in terms of existing crates or performance: we were accepting incorrect code before, so there will perhaps be some regressions, and we are now caching less.

As the comment above notes, we could do a lot better than this fix, but that would involve more invasive rewrites. I thought it best to start with something simple.

r? @pnkfelix -- but let's do crater/perf run
cc @arielb1

@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

commented May 1, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:07b81db5:start=1556731637715198881,finish=1556731638482573283,duration=767374402
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---

[00:04:02] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:04:02] tidy error: /checkout/src/librustc/traits/select.rs:874: line longer than 100 chars
[00:04:02] tidy error: /checkout/src/test/ui/traits/cycle-cache-err-60010.rs:18: line longer than 100 chars
[00:04:02] tidy error: /checkout/src/test/ui/traits/cycle-cache-err-60010.rs:24: line longer than 100 chars
[00:04:02] tidy error: /checkout/src/test/ui/traits/cycle-cache-err-60010.rs:26: line longer than 100 chars
[00:04:02] tidy error: /checkout/src/test/ui/traits/cycle-cache-err-60010.rs:27: line longer than 100 chars
[00:04:02] tidy error: /checkout/src/test/ui/traits/cycle-cache-err-60010.rs:30: line longer than 100 chars
[00:04:04] some tidy checks failed
[00:04:04] 
[00:04:04] 
[00:04:04] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:04:04] 
[00:04:04] 
[00:04:04] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:04:04] Build completed unsuccessfully in 0:00:45
[00:04:04] Build completed unsuccessfully in 0:00:45
[00:04:04] make: *** [tidy] Error 1
[00:04:04] Makefile:67: recipe for target 'tidy' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:1bafcee4
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Wed May  1 17:31:34 UTC 2019
---
travis_time:end:1de6d40e:start=1556731895489899076,finish=1556731895494582882,duration=4683806
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:13e8ac13
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:04a67700
travis_time:start:04a67700
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:234bbfc2
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@nikomatsakis nikomatsakis force-pushed the nikomatsakis:issue-60010-cycle-error-investigation branch from fdd7a95 to 1da50d2 May 1, 2019

@rust-highfive

This comment was marked as outdated.

Copy link
Collaborator

commented May 1, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:01587ca2:start=1556733278992615532,finish=1556733279775283181,duration=782667649
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
$ export GCP_CACHE_BUCKET=rust-lang-ci-cache
$ export AWS_ACCESS_KEY_ID=AKIA46X5W6CZEJZ6XT55
---
[01:12:28] .................................................................................................... 4700/5476
[01:12:33] .................................................................................................... 4800/5476
[01:12:36] .................................................................................................... 4900/5476
[01:12:40] .................................................................................................... 5000/5476
[01:12:44] ....................................F............................................................... 5100/5476
[01:12:51] .................................................................................................... 5300/5476
[01:12:53] .................................................................................................... 5400/5476
[01:12:56] ..............i.............................................................
[01:12:56] failures:
[01:12:56] failures:
[01:12:56] 
[01:12:56] ---- [ui] ui/traits/cycle-cache-err-60010.rs stdout ----
[01:12:56] normalized stderr:
[01:12:56] error[E0275]: overflow evaluating the requirement `RootDatabase: SourceDatabase`
[01:12:56]   --> $DIR/cycle-cache-err-60010.rs:27:5
[01:12:56]    |
[01:12:56] LL |     _parse: <ParseQuery as Query<RootDatabase>>::Data,
[01:12:56]    |
[01:12:56]    |
[01:12:56]    = note: required because of the requirements on the impl of `Query<RootDatabase>` for `ParseQuery`
[01:12:56] 
[01:12:56] error[E0275]: overflow evaluating the requirement `RootDatabase: SourceDatabase`
[01:12:56]   --> $DIR/cycle-cache-err-60010.rs:30:6
[01:12:56]    |
[01:12:56] LL | impl Database for RootDatabase {
[01:12:56]    |
[01:12:56]    |
[01:12:56]    = note: required because of the requirements on the impl of `Query<RootDatabase>` for `ParseQuery`
[01:12:56]    = note: required because it appears within the type `SalsaStorage`
[01:12:56] error: aborting due to 2 previous errors
[01:12:56] 
[01:12:56] For more information about this error, try `rustc --explain E0275`.
[01:12:56] 
[01:12:56] 
[01:12:56] 
[01:12:56] 
[01:12:56] The actual stderr differed from the expected stderr.
[01:12:56] Actual stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/traits/cycle-cache-err-60010/cycle-cache-err-60010.stderr
[01:12:56] To update references, rerun the tests and pass the `--bless` flag
[01:12:56] To only update this specific test, also pass `--test-args traits/cycle-cache-err-60010.rs`
[01:12:56] error: 1 errors occurred comparing output.
[01:12:56] status: exit code: 1
[01:12:56] status: exit code: 1
[01:12:56] command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/traits/cycle-cache-err-60010.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "-o" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/traits/cycle-cache-err-60010/a" "-Crpath" "-O" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/traits/cycle-cache-err-60010/auxiliary" "-A" "unused"
[01:12:56] ------------------------------------------
[01:12:56] 
[01:12:56] ------------------------------------------
[01:12:56] stderr:
[01:12:56] stderr:
[01:12:56] ------------------------------------------
[01:12:56] error[E0275]: overflow evaluating the requirement `RootDatabase: SourceDatabase`
[01:12:56]   --> /checkout/src/test/ui/traits/cycle-cache-err-60010.rs:27:5
[01:12:56]    |
[01:12:56] LL |     _parse: <ParseQuery as Query<RootDatabase>>::Data,
[01:12:56]    |
[01:12:56]    |
[01:12:56]    = note: required because of the requirements on the impl of `Query<RootDatabase>` for `ParseQuery`
[01:12:56] 
[01:12:56] error[E0275]: overflow evaluating the requirement `RootDatabase: SourceDatabase`
[01:12:56]   --> /checkout/src/test/ui/traits/cycle-cache-err-60010.rs:30:6
[01:12:56]    |
[01:12:56] LL | impl Database for RootDatabase {
[01:12:56]    |
[01:12:56]    |
[01:12:56]    = note: required because of the requirements on the impl of `Query<RootDatabase>` for `ParseQuery`
[01:12:56]    = note: required because it appears within the type `SalsaStorage`
[01:12:56] error: aborting due to 2 previous errors
[01:12:56] 
[01:12:56] For more information about this error, try `rustc --explain E0275`.
[01:12:56] 
---
[01:12:56] thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:519:22
[01:12:56] note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
[01:12:56] 
[01:12:56] 
[01:12:56] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
[01:12:56] 
[01:12:56] 
[01:12:56] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
[01:12:56] Build completed unsuccessfully in 0:04:23
[01:12:56] Build completed unsuccessfully in 0:04:23
[01:12:56] make: *** [check] Error 1
[01:12:56] Makefile:48: recipe for target 'check' failed
The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:18c16d00
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
Wed May  1 19:07:46 UTC 2019
---
travis_time:end:1b4f988a:start=1556737668306194413,finish=1556737668313158650,duration=6964237
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:00c10f95
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:01c722fc
travis_time:start:01c722fc
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:0b2e759d
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@nikomatsakis nikomatsakis force-pushed the nikomatsakis:issue-60010-cycle-error-investigation branch from 1da50d2 to bd005a2 May 1, 2019

/// well as the second instance of `A: AutoTrait`) to supress
/// caching.
///
/// This is a simple, targeted fix. The correct fix requires

This comment has been minimized.

Copy link
@pnkfelix

pnkfelix May 2, 2019

Member

in what sense are you using the word "correct" here?

Is this use of "correct" meant to imply that there are cases of unsound code that on master we currently incorrectly accept, and a correct fix would reject, but this targeted fix continues to accept?

Or is this use of "correct" just meant to emphasize that this fix is not ideal with respect to compiler efficiency?

This comment has been minimized.

Copy link
@nikomatsakis

nikomatsakis May 2, 2019

Author Contributor

Oh, the latter -- efficiency.

@pnkfelix

This comment has been minimized.

Copy link
Member

commented May 2, 2019

@bors try

@bors

This comment has been minimized.

Copy link
Contributor

commented May 2, 2019

⌛️ Trying commit bd005a2 with merge cf8e5a0...

bors added a commit that referenced this pull request May 2, 2019

Auto merge of #60444 - nikomatsakis:issue-60010-cycle-error-investiga…
…tion, r=<try>

forego caching for all participants in cycles, apart from root node

This is a targeted fix for #60010, which uncovered a pretty bad failure of our caching strategy in the face of coinductive cycles. The problem is explained in the comment in the PR on the new field, `in_cycle`, but I'll reproduce it here:

> Starts out as false -- if, during evaluation, we encounter a
> cycle, then we will set this flag to true for all participants
> in the cycle (apart from the "head" node). These participants
> will then forego caching their results. This is not the most
> efficient solution, but it addresses #60010. The problem we
> are trying to prevent:
>
> - If you have `A: AutoTrait` requires `B: AutoTrait` and `C: NonAutoTrait`
> - `B: AutoTrait` requires `A: AutoTrait` (coinductive cycle, ok)
> - `C: NonAutoTrait` requires `A: AutoTrait` (non-coinductive cycle, not ok)
>
> you don't want to cache that `B: AutoTrait` or `A: AutoTrait`
> is `EvaluatedToOk`; this is because they were only considered
> ok on the premise that if `A: AutoTrait` held, but we indeed
> encountered a problem (later on) with `A: AutoTrait. So we
> currently set a flag on the stack node for `B: AutoTrait` (as
> well as the second instance of `A: AutoTrait`) to supress
> caching.
>
> This is a simple, targeted fix. The correct fix requires
> deeper changes, but would permit more caching: we could
> basically defer caching until we have fully evaluated the
> tree, and then cache the entire tree at once.

I'm not sure what the impact of this fix will be in terms of existing crates or performance: we were accepting incorrect code before, so there will perhaps be some regressions, and we are now caching less.

As the comment above notes, we could do a lot better than this fix, but that would involve more invasive rewrites. I thought it best to start with something simple.

r? @pnkfelix -- but let's do crater/perf run
cc @arielb1
@pnkfelix

This comment has been minimized.

Copy link
Member

commented May 2, 2019

(the change seems fine, though I would like a small revision to the comment where I noted a somewhat ambiguous of the word "correct")

I'll try to get the crater and perf runs going.

@bors

This comment has been minimized.

Copy link
Contributor

commented May 2, 2019

☀️ Try build successful - checks-travis
Build commit: cf8e5a0

@nikomatsakis

This comment has been minimized.

Copy link
Contributor Author

commented May 7, 2019

@rust-timer

This comment has been minimized.

Copy link

commented May 7, 2019

Success: Queued cf8e5a0 with parent 92b5e20, comparison URL.

@rust-timer

This comment has been minimized.

Copy link

commented May 7, 2019

Finished benchmarking try commit cf8e5a0

@nikomatsakis

This comment has been minimized.

Copy link
Contributor Author

commented May 7, 2019

Seems like it is a wash performance wise, which is great.

@nikomatsakis

This comment has been minimized.

Copy link
Contributor Author

commented May 7, 2019

@craterbot run mode=check-only

@craterbot

This comment has been minimized.

Copy link
Collaborator

commented May 7, 2019

👌 Experiment pr-60444 created and queued.
🤖 Automatically detected try build cf8e5a0
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot

This comment has been minimized.

Copy link
Collaborator

commented May 7, 2019

🚧 Experiment pr-60444 is now running on agent aws-2.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@arielb1

This comment has been minimized.

Copy link
Contributor

commented May 9, 2019

This looks like an annoying problem, and I agree we shouldn't fix it the "right way" if Chalk is going to be merged soon-ish.

I was afraid for some time that a problem like this might be present, but didn't have time to dig in deeper. I would have preferred to do a Tarjan's-style mechanism using EvaluatedToOk etc. (which would cache cycles as a unit), but this is OK too if doesn't break some edge-case too horribly.

@arielb1

This comment has been minimized.

Copy link
Contributor

commented May 9, 2019

I also think the performance impact here shouldn't be so horrible: every time this is hit, we do cache at least one trait, so we only evaluate each member of a cycle up to N times, where N is the length of the cycle. This means the performance impact is bounded and we shouldn't have any terrible worst-cases.

Would be nice to write this in a comment. r=me with that comment and a clean crater.

@craterbot

This comment has been minimized.

Copy link
Collaborator

commented May 10, 2019

🎉 Experiment pr-60444 is completed!
📊 6 regressed and 0 fixed (60951 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the blacklist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

modify comment
modify the comment on `in_cycle` to reflect changes requested by ariel and myself.
@nikomatsakis

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2019

Dear crate authors,

This PR fixes a soundness hole in our trait resolution caching scheme. I'm not sure how easily we'll be able to do a warning system for this fix (I can't immediately think of how to do one), so I'm contacting you to let you know that your crates may be affected. If you'd like to try out this build, you should be able to install it using @kennytm's rustup-toolchain-install-master script and installing the toolchain cf8e5a06b289fbcdaf75ca55c057b2cb09fff33b.

There are two crates from crates.io that appear to be affected:

I will take a look at those crates and see if I can offer any suggestions.

There are also two crates appear unmaintained (last update was >2 years ago) and a git repository that appears similar (last commit was Sep 2018). Apologies if this is incorrect!

As these appear unmaintained, I don't plan to investigate them further.

@ipetkov

This comment has been minimized.

Copy link
Contributor

commented May 14, 2019

Hey @nikomatsakis, thanks for flagging!

conch-parser is maintained in the sense that I intend to add more to it, but without anyone using it/sending bug reports, me trying to work on its sister crate conch-runtime, and, well life happening in general, I haven't updated it in a while 😅

I wasn't able to build this locally as the install script failed due to a missing component:
detecting the channel of the `cf8e5a06b289fbcdaf75ca55c057b2cb09fff33b` toolchain...
downloading ...
error: missing component `rustc` on toolchain `cf8e5a06b289fbcdaf75ca55c057b2cb09fff33b` on channel `nightly` for target `x86_64-apple-darwin`
I peeked at the build logs, it appears due to the compiler getting stuck when attempting to verify Send/Sync bounds on a very deeply recursive and generic type hierarchy. I'm happy to elaborate more on what the crate does so you don't have to do too much spelunking.

The crate offers the functionality to parse shell programs. Each piece of the grammar is represented as a node which can hold generic sub-nodes. The reasoning for this is so that the crate consumer could customize their AST with different/custom nodes, while reusing existing implementations.

The shell grammar is deeply recursive. Basically each command can vary in complexity (compound commands such as case, for, or simple commands like echo foo), but is ultimately made up of a list of shell words (literals, interpolations, etc.). Because each word can contain a command substitution, the AST type is recursive (a Command<W> has a Word<C> type, which gives us Command<Word<Command<...>>).

There are two "top-level" type definitions which seek to unify the entire AST tree concretely which are basically TopLevelCommand(Command<TopLevelWord>) TopLevelWord(Word<TopLevelCommand>). There are a few tests which assert that the appropriate concrete AST trees are Send/Sync as appropriate, which is where the build is appearing to fail.

@pnkfelix

This comment has been minimized.

Copy link
Member

commented May 14, 2019

I'm going to take the plunge here and r+ this. This is fixing a pretty serious bug, and the cases that regress seem like they are either: 1. instances of cycles (which can't be fixed), or 2. cases where increasing the threshold will fix things.

Overall i think the impact is reasonable.

@pnkfelix

This comment has been minimized.

Copy link
Member

commented May 14, 2019

@bors r+

@bors

This comment has been minimized.

Copy link
Contributor

commented May 14, 2019

📌 Commit decd6d3 has been approved by pnkfelix

Centril added a commit to Centril/rust that referenced this pull request May 14, 2019

Rollup merge of rust-lang#60444 - nikomatsakis:issue-60010-cycle-erro…
…r-investigation, r=pnkfelix

forego caching for all participants in cycles, apart from root node

This is a targeted fix for rust-lang#60010, which uncovered a pretty bad failure of our caching strategy in the face of coinductive cycles. The problem is explained in the comment in the PR on the new field, `in_cycle`, but I'll reproduce it here:

> Starts out as false -- if, during evaluation, we encounter a
> cycle, then we will set this flag to true for all participants
> in the cycle (apart from the "head" node). These participants
> will then forego caching their results. This is not the most
> efficient solution, but it addresses rust-lang#60010. The problem we
> are trying to prevent:
>
> - If you have `A: AutoTrait` requires `B: AutoTrait` and `C: NonAutoTrait`
> - `B: AutoTrait` requires `A: AutoTrait` (coinductive cycle, ok)
> - `C: NonAutoTrait` requires `A: AutoTrait` (non-coinductive cycle, not ok)
>
> you don't want to cache that `B: AutoTrait` or `A: AutoTrait`
> is `EvaluatedToOk`; this is because they were only considered
> ok on the premise that if `A: AutoTrait` held, but we indeed
> encountered a problem (later on) with `A: AutoTrait. So we
> currently set a flag on the stack node for `B: AutoTrait` (as
> well as the second instance of `A: AutoTrait`) to supress
> caching.
>
> This is a simple, targeted fix. The correct fix requires
> deeper changes, but would permit more caching: we could
> basically defer caching until we have fully evaluated the
> tree, and then cache the entire tree at once.

I'm not sure what the impact of this fix will be in terms of existing crates or performance: we were accepting incorrect code before, so there will perhaps be some regressions, and we are now caching less.

As the comment above notes, we could do a lot better than this fix, but that would involve more invasive rewrites. I thought it best to start with something simple.

r? @pnkfelix -- but let's do crater/perf run
cc @arielb1

bors added a commit that referenced this pull request May 14, 2019

Auto merge of #60834 - Centril:rollup-fikyi9i, r=Centril
Rollup of 9 pull requests

Successful merges:

 - #60130 (Add implementations of last in terms of next_back on a bunch of DoubleEndedIterators)
 - #60443 (as_ptr returns a read-only pointer)
 - #60444 (forego caching for all participants in cycles, apart from root node)
 - #60719 (Allow subdirectories to be tested by x.py test)
 - #60780 (fix Miri)
 - #60788 (default to $ARCH-apple-macosx10.7.0 LLVM triple for darwin targets)
 - #60799 (Allow late-bound regions in existential types)
 - #60808 (Improve the "must use" lint for `Future`)
 - #60819 (submodules: update clippy from 3710ec59 to ad3269c4)

Failed merges:

r? @ghost

@bors bors merged commit decd6d3 into rust-lang:master May 14, 2019

1 check passed

Travis CI - Pull Request Build Passed
Details

ipetkov added a commit to ipetkov/conch-parser that referenced this pull request May 15, 2019

@ipetkov

This comment has been minimized.

Copy link
Contributor

commented May 15, 2019

cases where increasing the threshold will fix things

Confirming that increasing the recursion limit to 128 has fixed building my crates. However, I'm seeing a significant perf issue when building. I'll open a separate issue for this

ipetkov added a commit to ipetkov/conch-runtime that referenced this pull request May 15, 2019

ipetkov added a commit to ipetkov/conch-runtime that referenced this pull request May 15, 2019

ipetkov added a commit to ipetkov/conch-runtime that referenced this pull request May 15, 2019

ipetkov added a commit to ipetkov/conch-runtime that referenced this pull request May 15, 2019

ipetkov added a commit to ipetkov/conch-runtime that referenced this pull request May 15, 2019

matklad added a commit to rust-analyzer/rust-analyzer that referenced this pull request May 18, 2019

Assert that DB is unwind-safe, instead of proving
Unfortunately, that `: RefUnwindSafe` bound gives rustc a hard time,
so let's remove it for know.

See

* #1283
* rust-lang/rust#60444
* rust-lang/rust#58291

closes #1283
@Zoxc

This comment has been minimized.

Copy link
Contributor

commented May 18, 2019

I think this PR makes rustc unable to build itself with [rust] parallel-compiler = true. Building libsyntax ends up taking too much time, probably when trying to prove Send or Sync bounds for the AST.

Marwes added a commit to Marwes/gluon that referenced this pull request May 22, 2019

Marwes added a commit to Marwes/gluon that referenced this pull request May 22, 2019

Marwes added a commit to Marwes/gluon that referenced this pull request May 22, 2019

bors added a commit that referenced this pull request May 27, 2019

Auto merge of #60967 - Zoxc:fix-syntax-sync, r=michaelwoerister
Short circuit Send and Sync impls for TokenTree

Workaround to make the parallel compiler build after #60444.

r? @nikomatsakis

Marwes added a commit to Marwes/gluon that referenced this pull request Jun 9, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.