Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Still more `ObligationForest` improvements. #64805

Merged
merged 11 commits into from Oct 2, 2019

Conversation

@nnethercote
Copy link
Contributor

commented Sep 26, 2019

Following on from #64627, more readability improvements, but negligible effects on speed.

r? @nikomatsakis

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

commented Sep 26, 2019

This is the end of my spate of ObligationForest changes. I didn't manage to get rid of the memcpys caused by the swap calls within compress, which was the original motivation for looking into ObligationForest. (Well... not without (a) regressing performance, (b) causing test failures, or (c) using unsafe.) drain_filter() is the right fit conceptually, but caused a slowdown in practice :(

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Sep 26, 2019

The job x86_64-gnu-llvm-6.0 of your PR failed (pretty log, raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
2019-09-26T07:20:08.1718424Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-09-26T07:20:08.1904525Z ##[command]git config gc.auto 0
2019-09-26T07:20:08.1998033Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-09-26T07:20:08.2060658Z ##[command]git config --get-all http.proxy
2019-09-26T07:20:08.2223453Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/64805/merge:refs/remotes/pull/64805/merge
---
2019-09-26T08:27:04.2521920Z .................................................................................................... 1500/9044
2019-09-26T08:27:10.5786708Z .................................................................................................... 1600/9044
2019-09-26T08:27:23.4963227Z ..........................................................................i...............i......... 1700/9044
2019-09-26T08:27:30.6911019Z .................................................................................................... 1800/9044
2019-09-26T08:27:39.7858775Z .................................................................iiiii.............................. 1900/9044
2019-09-26T08:28:00.7517794Z .................................................................................................... 2100/9044
2019-09-26T08:28:03.4777073Z .................................................................................................... 2200/9044
2019-09-26T08:28:06.8425148Z .................................................................................................... 2300/9044
2019-09-26T08:28:15.9527139Z .................................................................................................... 2400/9044
---
2019-09-26T08:31:25.4262808Z ........................................................i...............i........................... 4700/9044
2019-09-26T08:31:35.2498035Z .................................................................................................... 4800/9044
2019-09-26T08:31:44.3020027Z .................................................................................................... 4900/9044
2019-09-26T08:31:52.2854541Z .................................................................................................... 5000/9044
2019-09-26T08:32:02.6221164Z ...........................................ii.ii.................................................... 5100/9044
2019-09-26T08:32:13.4139297Z .................................................................................................... 5300/9044
2019-09-26T08:32:24.4760863Z .................................................................................................... 5400/9044
2019-09-26T08:32:32.2826532Z ........i........................................................................................... 5500/9044
2019-09-26T08:32:38.2460043Z .................................................................................................... 5600/9044
2019-09-26T08:32:38.2460043Z .................................................................................................... 5600/9044
2019-09-26T08:32:50.6272060Z .................................................................................................... 5700/9044
2019-09-26T08:33:04.2621025Z ...ii...i..ii............i.......................................................................... 5800/9044
2019-09-26T08:33:27.3196761Z .................................................................................................... 6000/9044
2019-09-26T08:33:37.1608865Z .................................................................................................... 6100/9044
2019-09-26T08:33:37.1608865Z .................................................................................................... 6100/9044
2019-09-26T08:33:53.8213664Z .....i..ii.......................................................................................... 6200/9044
2019-09-26T08:34:14.1316332Z .................................................................i.................................. 6400/9044
2019-09-26T08:34:16.4682969Z .................................................................................................... 6500/9044
2019-09-26T08:34:19.2523721Z .....................................i.............................................................. 6600/9044
2019-09-26T08:34:23.5707037Z .................................................................................................... 6700/9044
---
2019-09-26T08:35:21.3521453Z .................................................................................................... 7200/9044
2019-09-26T08:35:27.1248564Z .................................................................................................... 7300/9044
2019-09-26T08:35:35.0538076Z .................................................................................................... 7400/9044
2019-09-26T08:35:46.0213214Z .................................................................................................... 7500/9044
2019-09-26T08:35:57.0818971Z .........................................................................................ii......i.. 7600/9044
2019-09-26T08:36:09.4301033Z .................................................................................................... 7800/9044
2019-09-26T08:36:27.7952234Z .................................................................................................... 7900/9044
2019-09-26T08:36:36.7016606Z .................................................................................................... 8000/9044
2019-09-26T08:36:47.4054420Z .................................................................................................... 8100/9044
---
2019-09-26T08:39:12.4658297Z  finished in 5.766
2019-09-26T08:39:12.4806446Z Check compiletest suite=codegen mode=codegen (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:12.6761976Z 
2019-09-26T08:39:12.6762234Z running 150 tests
2019-09-26T08:39:16.2542326Z i....iii......iii..iiii....i.............................i..i..................i....i.........ii.i.i 100/150
2019-09-26T08:39:18.3635693Z ..iiii..............i.........iii.i.......ii......
2019-09-26T08:39:18.3638395Z 
2019-09-26T08:39:18.3643131Z  finished in 5.883
2019-09-26T08:39:18.3829952Z Check compiletest suite=codegen-units mode=codegen-units (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:18.5572358Z 
---
2019-09-26T08:39:20.7414149Z  finished in 2.358
2019-09-26T08:39:20.7628910Z Check compiletest suite=assembly mode=assembly (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:20.9311079Z 
2019-09-26T08:39:20.9338195Z running 9 tests
2019-09-26T08:39:20.9339024Z iiiiiiiii
2019-09-26T08:39:20.9339402Z 
2019-09-26T08:39:20.9339482Z  finished in 0.168
2019-09-26T08:39:20.9533216Z Check compiletest suite=incremental mode=incremental (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:21.1459137Z 
---
2019-09-26T08:39:41.1289030Z  finished in 20.174
2019-09-26T08:39:41.1575421Z Check compiletest suite=debuginfo mode=debuginfo-gdb+lldb (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:39:41.3785766Z 
2019-09-26T08:39:41.3788665Z running 123 tests
2019-09-26T08:40:07.8578268Z .iiiii...i.....i..i...i..i.i.i..i.ii..i.i.....i..i....ii..........iiii..........i...ii...i.......ii. 100/123
2019-09-26T08:40:12.9879767Z i.i.i......iii.i.....ii
2019-09-26T08:40:12.9881676Z 
2019-09-26T08:40:12.9887896Z  finished in 31.834
2019-09-26T08:40:12.9901901Z Uplifting stage1 rustc (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T08:40:12.9902353Z Copying stage2 rustc from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / x86_64-unknown-linux-gnu)
---
2019-09-26T08:54:17.0429340Z 
2019-09-26T08:54:17.0430114Z    Doc-tests core
2019-09-26T08:54:22.5590872Z 
2019-09-26T08:54:22.5591270Z running 2405 tests
2019-09-26T08:54:34.6884481Z ......iiiii......................................................................................... 100/2405
2019-09-26T08:54:46.6125539Z ................................................................................ii.................. 200/2405
2019-09-26T08:55:14.5686089Z .i.................................................................................................. 400/2405
2019-09-26T08:55:14.5686089Z .i.................................................................................................. 400/2405
2019-09-26T08:55:26.1961600Z ................................................i..i.................iiii........................... 500/2405
2019-09-26T08:55:48.6866484Z .................................................................................................... 700/2405
2019-09-26T08:56:00.2353757Z .................................................................................................... 800/2405
2019-09-26T08:56:11.5510730Z .................................................................................................... 900/2405
2019-09-26T08:56:22.8585477Z .................................................................................................... 1000/2405
---
2019-09-26T09:00:48.0831131Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/mutex.rs:689:13
2019-09-26T09:00:48.0844677Z .......thread '<unnamed>' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:00:48.0845128Z   left: `1`,
2019-09-26T09:00:48.0846002Z  right: `2`', src/libstd/sync/mutex.rs:653:13
2019-09-26T09:00:48.0913036Z ......thread '<unnamed>' panicked at 'test panic in inner thread to poison RwLock', src/libstd/sync/rwlock.rs:791:13
2019-09-26T09:00:48.0930478Z ...thread '.<unnamed>' panicked at 'test panic in inner thread to poison RwLock', src/libstd/sync/rwlock.rs:768:13
2019-09-26T09:00:48.0952645Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:705:13
2019-09-26T09:00:48.0961517Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:635:13
2019-09-26T09:00:48.0965441Z .thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:646:13
2019-09-26T09:00:48.0972700Z ..thread '<unnamed>' panicked at 'explicit panic', src/libstd/sync/rwlock.rs:611:13
2019-09-26T09:00:50.1621647Z ..........................thread '<unnamed>' panicked at 'What the answer to my lifetimes dilemma is?', src/libstd/sys_common/remutex.rs:233:13
2019-09-26T09:00:50.1667530Z ........................ 700/763
2019-09-26T09:00:50.1772633Z ...................................thread '<unnamed>' panicked at 'explicit panic', src/libstd/thread/mod.rs:1535:13
2019-09-26T09:00:50.7859922Z ............thread '<unnamed>' panicked at 'Box<Any>', src/libstd/thread/mod.rs:1667:13
---
2019-09-26T09:00:57.5786006Z 
2019-09-26T09:00:57.5787324Z running 992 tests
2019-09-26T09:01:20.7614874Z i................................................................................................... 100/992
2019-09-26T09:01:33.5081082Z .................................................................................................... 200/992
2019-09-26T09:01:42.4907211Z .................iii......i......i...i......i....................................................... 300/992
2019-09-26T09:01:48.7715508Z .................................................................................................... 400/992
2019-09-26T09:01:57.3470795Z ...................................i..i.................................ii.......................... 500/992
2019-09-26T09:02:14.5783242Z .................................................................................................... 700/992
2019-09-26T09:02:14.5783242Z .................................................................................................... 700/992
2019-09-26T09:02:23.9195530Z ..................iiii.............................................................................. 800/992
2019-09-26T09:02:40.7113266Z .................................................................................................... 900/992
2019-09-26T09:02:49.2074246Z ........................................iiii................................................
2019-09-26T09:02:49.2075723Z 
2019-09-26T09:02:49.2169224Z  finished in 212.065
2019-09-26T09:02:49.2186343Z Testing term stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
2019-09-26T09:02:49.4418675Z    Compiling term v0.0.0 (/checkout/src/libterm)
---
2019-09-26T09:05:03.4344108Z     Finished release [optimized] target(s) in 17.89s
2019-09-26T09:05:03.4385425Z      Running build/x86_64-unknown-linux-gnu/stage1-rustc/x86_64-unknown-linux-gnu/release/deps/rustc_data_structures-5f9a32e9b4b8b904
2019-09-26T09:05:03.4409830Z 
2019-09-26T09:05:03.4410192Z running 154 tests
2019-09-26T09:05:03.4776721Z .................................FFFF.F............................................................. 100/154
2019-09-26T09:05:03.4905546Z failures:
2019-09-26T09:05:03.4905698Z 
2019-09-26T09:05:03.4906507Z ---- obligation_forest::tests::diamond stdout ----
2019-09-26T09:05:03.4906507Z ---- obligation_forest::tests::diamond stdout ----
2019-09-26T09:05:03.4908483Z thread 'obligation_forest::tests::diamond' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4908859Z   left: `["A", "A.1", "A.2", "D"]`,
2019-09-26T09:05:03.4909384Z  right: `["D", "A.2", "A.1", "A"]`', src/librustc_data_structures/obligation_forest/tests.rs:264:5
2019-09-26T09:05:03.4909770Z 
2019-09-26T09:05:03.4910179Z ---- obligation_forest::tests::done_dependency stdout ----
2019-09-26T09:05:03.4910179Z ---- obligation_forest::tests::done_dependency stdout ----
2019-09-26T09:05:03.4910634Z thread 'obligation_forest::tests::done_dependency' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4911321Z   left: `["A: Sized", "B: Sized", "C: Sized"]`,
2019-09-26T09:05:03.4911906Z  right: `["C: Sized", "B: Sized", "A: Sized"]`', src/librustc_data_structures/obligation_forest/tests.rs:326:5
2019-09-26T09:05:03.4912479Z ---- obligation_forest::tests::orphan stdout ----
2019-09-26T09:05:03.4912479Z ---- obligation_forest::tests::orphan stdout ----
2019-09-26T09:05:03.4912941Z thread 'obligation_forest::tests::orphan' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4913133Z   left: `["C1", "C2"]`,
2019-09-26T09:05:03.4913530Z  right: `["C2", "C1"]`', src/librustc_data_structures/obligation_forest/tests.rs:364:5
2019-09-26T09:05:03.4913725Z 
2019-09-26T09:05:03.4914108Z ---- obligation_forest::tests::push_pop stdout ----
2019-09-26T09:05:03.4914544Z thread 'obligation_forest::tests::push_pop' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4914753Z   left: `["A.1", "A.3", "A.3.i"]`,
2019-09-26T09:05:03.4915187Z  right: `["A.3", "A.1", "A.3.i"]`', src/librustc_data_structures/obligation_forest/tests.rs:119:5
2019-09-26T09:05:03.4915757Z ---- obligation_forest::tests::success_in_grandchildren stdout ----
2019-09-26T09:05:03.4915757Z ---- obligation_forest::tests::success_in_grandchildren stdout ----
2019-09-26T09:05:03.4916221Z thread 'obligation_forest::tests::success_in_grandchildren' panicked at 'assertion failed: `(left == right)`
2019-09-26T09:05:03.4916432Z   left: `["A.1", "A.3"]`,
2019-09-26T09:05:03.4916833Z  right: `["A.3", "A.1"]`', src/librustc_data_structures/obligation_forest/tests.rs:175:5
2019-09-26T09:05:03.4917230Z 
2019-09-26T09:05:03.4917505Z failures:
2019-09-26T09:05:03.4917640Z     obligation_forest::tests::diamond
2019-09-26T09:05:03.4917789Z     obligation_forest::tests::done_dependency
2019-09-26T09:05:03.4917789Z     obligation_forest::tests::done_dependency
2019-09-26T09:05:03.4918304Z     obligation_forest::tests::orphan
2019-09-26T09:05:03.4918447Z     obligation_forest::tests::push_pop
2019-09-26T09:05:03.4918606Z     obligation_forest::tests::success_in_grandchildren
2019-09-26T09:05:03.4918728Z 
2019-09-26T09:05:03.4918870Z test result: FAILED. 149 passed; 5 failed; 0 ignored; 0 measured; 0 filtered out
2019-09-26T09:05:03.4918992Z 
2019-09-26T09:05:03.4919454Z error: test failed, to rerun pass '-p rustc_data_structures --lib'
2019-09-26T09:05:03.4963972Z 
2019-09-26T09:05:03.4965023Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "test" "-Zconfig-profile" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--locked" "--color" "always" "--features" "" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "-p" "rustc_data_structures" "--" "--quiet"
2019-09-26T09:05:03.4966835Z expected success, got: exit code: 101
2019-09-26T09:05:03.4967760Z 
2019-09-26T09:05:03.4967760Z 
2019-09-26T09:05:03.4967936Z 
2019-09-26T09:05:03.4968094Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-09-26T09:05:03.4971688Z Build completed unsuccessfully in 1:37:14
2019-09-26T09:05:03.4972006Z == clock drift check ==
2019-09-26T09:05:03.4998863Z   local time: Thu Sep 26 09:05:03 UTC 2019
2019-09-26T09:05:03.6527725Z   network time: Thu, 26 Sep 2019 09:05:03 GMT
2019-09-26T09:05:03.6530378Z == end clock drift check ==
2019-09-26T09:05:04.3452982Z ##[error]Bash exited with code '1'.
2019-09-26T09:05:04.3503490Z ##[section]Starting: Checkout
2019-09-26T09:05:04.3505447Z ==============================================================================
2019-09-26T09:05:04.3505518Z Task         : Get sources
2019-09-26T09:05:04.3505566Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@pnkfelix

This comment has been minimized.

Copy link
Member

commented Sep 26, 2019

@nnethercote what sort of test failures were you observing from (b) above?

Are we talking about injecting unsoundness bugs into compiler? Or changes to diagnostic output? Or something else?

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

commented Sep 26, 2019

Are we talking about injecting unsoundness bugs into compiler? Or changes to diagnostic output? Or something else?

They were all changes in error messages that arose from changing the order of pending obligations. Some of them looked pretty trivial (e.g. change the order of two errors) but some of them were more complex, to the point where I was struggling to tell if the new messages were as valid as the old ones. And there were quite a few in the latter basket. So I ended up preserving the order because the potential speed benefits of reordering were small.

@nnethercote

This comment has been minimized.

Copy link
Contributor Author

commented Sep 26, 2019

The test failures on automation are trivial; some vectors of erroneous obligations have changed order, but (AIUI) the order is not significant. I will fix those on Monday.

nnethercote added 11 commits Sep 24, 2019
It's not necessary; cycles (which are rare) can be detected by looking
at the node stack.

This change speeds things up slightly, as well as simplifying the code a
little.
For consistency with other variables in this file.
It has a single call site, and the code is easier to read this way.
They don't help performance at all, and just complicate things.
These make the code more concise.
By collecting the done obligations (when necessary) in the main loop.
This makes the code cleaner.

The commit also changes the order in which successful obligations are
returned -- they are now returned in the registered order, rather than
reversed. Because this order doesn't actually matter, being only used by
tests, the commit uses `sort()` to make the test agnostic w.r.t. the
order.
@nnethercote nnethercote force-pushed the nnethercote:ObligForest-still-more branch from 3af2b25 to a820672 Sep 29, 2019
@nnethercote

This comment has been minimized.

Copy link
Contributor Author

commented Sep 29, 2019

I fixed the test failures and rebased. Ready for review, @nikomatsakis!

@nikomatsakis

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2019

@bors r+

@bors

This comment has been minimized.

Copy link
Contributor

commented Oct 1, 2019

📌 Commit a820672 has been approved by nikomatsakis

tmandry added a commit to tmandry/rust that referenced this pull request Oct 2, 2019
… r=nikomatsakis

Still more `ObligationForest` improvements.

Following on from rust-lang#64627, more readability improvements, but negligible effects on speed.

r? @nikomatsakis
bors added a commit that referenced this pull request Oct 2, 2019
Rollup of 11 pull requests

Successful merges:

 - #64649 (Avoid ICE on return outside of fn with literal array)
 - #64722 (Make all alt builders produce parallel-enabled compilers)
 - #64801 (Avoid `chain()` in `find_constraint_paths_between_regions()`.)
 - #64805 (Still more `ObligationForest` improvements.)
 - #64840 (SelfProfiler API refactoring and part one of event review)
 - #64885 (use try_fold instead of try_for_each to reduce compile time)
 - #64942 (Fix clippy warnings)
 - #64952 (Update cargo.)
 - #64974 (Fix zebra-striping in generic dataflow visualization)
 - #64978 (Fully clear `HandlerInner` in `Handler::reset_err_count`)
 - #64979 (Update books)

Failed merges:

 - #64959 (syntax: improve parameter without type suggestions)

r? @ghost
@bors bors merged commit a820672 into rust-lang:master Oct 2, 2019
4 checks passed
4 checks passed
pr Build #20190929.41 succeeded
Details
pr (Linux mingw-check) Linux mingw-check succeeded
Details
pr (Linux x86_64-gnu-llvm-6.0) Linux x86_64-gnu-llvm-6.0 succeeded
Details
pr (LinuxTools) LinuxTools succeeded
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.