Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A fix for 51821 #51855

Merged
merged 10 commits into from Jul 1, 2018
Merged

A fix for 51821 #51855

merged 10 commits into from Jul 1, 2018

Conversation

Eh2406
Copy link
Contributor

@Eh2406 Eh2406 commented Jun 27, 2018

This dedupe the vec of OutlivesConstraint using a FxHashSet<(RegionVid, RegionVid)> it alsow adds a struct ConstraintSet to encapsulate/ensure this behavere.

@rust-highfive
Copy link
Collaborator

r? @estebank

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 27, 2018
Copy link
Contributor

@nikomatsakis nikomatsakis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great so far; the main thing that is sort of missing for me is that I'd like to see the ConstraintSet do a better job encapsulating its members. I'll leave some notes on the main issue with what I'm thinking of.

type_tests,
universal_regions,
};

for c in outlives_constraints {
result.add_outlives_iner(c);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "inner"

}
}

pub fn iner(&self) -> &IndexVec<ConstraintIndex, OutlivesConstraint> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "inner" -- but this method seems surprising, perhaps it'll disappear in future commits though =)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will fix the typo.

As to removing it in the future, iif that is what you want to get this merged. The alternative is to wrap all the iter methods that get used all over the place.

@lqd
Copy link
Member

lqd commented Jun 27, 2018

Just dropping a link to #51821 for easy navigation :)

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-3.9 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

[00:05:09] travis_fold:start:tidy
travis_time:start:tidy
tidy check
[00:05:10] tidy error: /checkout/src/librustc_mir/borrow_check/nll/constraint_set.rs:17: line longer than 100 chars
[00:05:10] tidy error: /checkout/src/librustc_mir/borrow_check/nll/constraint_set.rs:31: line longer than 100 chars
[00:05:10] tidy error: /checkout/src/librustc_mir/borrow_check/nll/constraint_set.rs: incorrect license
[00:05:11] some tidy checks failed
[00:05:11] 
[00:05:11] 
[00:05:11] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/tidy" "/checkout/src" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "--no-vendor" "--quiet"
[00:05:11] 
[00:05:11] 
[00:05:11] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test src/tools/tidy
[00:05:11] Build completed unsuccessfully in 0:01:54
[00:05:11] Build completed unsuccessfully in 0:01:54
[00:05:11] Makefile:79: recipe for target 'tidy' failed
[00:05:11] make: *** [tidy] Error 1

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:0d1389d0
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)
---
travis_time:end:04a65d5b:start=1530133238118435305,finish=1530133238124393049,duration=5957744
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:0866b175
$ head -30 ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
head: cannot open ‘./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers’ for reading: No such file or directory
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:036c4460
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@nikomatsakis
Copy link
Contributor

I expect ConstraintSet to have an API like this:

impl ConstraintSet {
  fn new() -> Self { .. }

  fn push(&mut self, constraint: Constraint) { .. }

  fn link(&mut self) { /* updates the `next` fields */ }

  fn each_affected_by_dirty(&mut self, r: RegionVid, op: impl FnMut(ConstraintIndex)) {
    /* given that the region `a` has changed, invoke `op` with each constraint `b: a` */
  }
}

@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-3.9 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
[00:13:12]    Compiling rustc_mir v0.0.0 (file:///checkout/src/librustc_mir)
[00:13:53] error: method is never used: `new`
[00:13:53]   --> librustc_mir/borrow_check/nll/constraint_set.rs:26:5
[00:13:53]    |
[00:13:53] 26 |     pub fn new() -> Self {
[00:13:53]    |
[00:13:53]    = note: `-D dead-code` implied by `-D warnings`
[00:13:53] 
[00:13:54] error: aborting due to previous error
[00:13:54] error: aborting due to previous error
[00:13:54] 
[00:13:54] error: Could not compile `rustc_mir`.
[00:13:54] 
[00:13:54] Caused by:
[00:13:54]   process didn't exit successfully: `/checkout/obj/build/bootstrap/debug/rustc --crate-name rustc_mir librustc_mir/lib.rs --color always --error-format json --crate-type dylib --emit=dep-info,link -C prefer-dynamic -C opt-level=2 -C metadata=6ef51208e3ed61cd -C extra-filename=-6ef51208e3ed61cd --out-dir /checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps --target x86_64-unknown-linux-gnu -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps -L dependency=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/release/deps --extern rustc_data_structures=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_data_structures-a7df2fc298b4fb06.so --extern rustc_errors=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_errors-c9d6678b1c0f0b46.so --extern rustc_target=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_target-58741ed9de9aae4f.so --extern log_settings=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/liblog_settings-f16e9eee09cda644.rlib --extern graphviz=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libgraphviz-be994de8b3050feb.so --extern log=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/liblog-5073f1296cd24b67.rlib --extern rustc=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc-f81250751e0ebed7.so --extern polonius_engine=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libpolonius_engine-6e0be5cf77966185.rlib --extern byteorder=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libbyteorder-78ebc049d4f53f46.rlib --extern bitflags=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libbitflags-99b4534960f92449.rlib --extern either=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libeither-11fcd35bf55f48b3.rlib --extern arena=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libarena-ccde2368d50449de.so --extern syntax_pos=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libsyntax_pos-670f6bac60d99b34.so --extern rustc_apfloat=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/librustc_apfloat-642a7efe79e24835.rlib --extern syntax=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libsyntax-6a2f0731783c2bd3.so --extern serialize=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libserialize-5d0a8a65bb9fe29f.so --extern serialize=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/deps/libserialize-5d0a8a65bb9fe29f.rlib -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/backtrace-sys-88673787176f9d86/out -L native=/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-rustc/x86_64-unknown-linux-gnu/release/build/miniz-sys-62d80197b9ec531e/out` (exit code: 101)
[00:15:14] error: build failed
[00:15:14] command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "build" "--target" "x86_64-unknown-linux-gnu" "-j" "4" "--release" "--locked" "--color" "always" "--features" " jemalloc" "--manifest-path" "/checkout/src/rustc/Cargo.toml" "--message-format" "json"
[00:15:14] expected success, got: exit code: 101
[00:15:14] expected success, got: exit code: 101
[00:15:14] thread 'main' panicked at 'cargo must succeed', bootstrap/compile.rs:1091:9
[00:15:14] travis_fold:end:stage0-rustc

[00:15:14] travis_time:end:stage0-rustc:start=1530136507658864149,finish=1530137133098669700,duration=625439805551


[00:15:14] failed to run: /checkout/obj/build/bootstrap/debug/bootstrap build
[00:15:14] Build completed unsuccessfully in 0:10:37
[00:15:14] make: *** [all] Error 1
[00:15:14] Makefile:28: recipe for target 'all' failed

The command "stamp sh -x -c "$RUN_SCRIPT"" exited with 2.
travis_time:start:00cca128
$ date && (curl -fs --head https://google.com | grep ^Date: | sed 's/Date: //g' || true)

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Copy link
Contributor

@nikomatsakis nikomatsakis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Left some more nits.

}
}

pub fn inner(&self) -> &IndexVec<ConstraintIndex, OutlivesConstraint> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Instead of Inner, we could implement IndexVec for ConstraintSet. Something like this;

impl Index<ConstraintIndex> for ConstraintSet {
    type Output = OutlivesConstriant;

    fn index(&self, index: ConstraintIndex) -> &OutlivesConstraint {
        &self.constraints[index]
    }
}

then you can just do constraint_set[index] instead of constraint_set.inner()[index]

&self.constraints
}

pub fn link(&mut self, len: usize) -> IndexVec<RegionVid, Option<ConstraintIndex>> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a comment here:


Once all constraints have been added, link() is used to thread together the constraints based on which would be affected when a particular region changes. See the next field of OutlivesContraint for more deatils. link returns a map that is needed later by each_affected_by_dirty.

map
}

pub fn each_affected_by_dirty(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment:


When a region R1 changes, we need to reprocess all constraints R2: R1 to take into account any new elements that R1 now has. This method will quickly enumerate all such constraints (that is, constraints where R1 is in the "subregion" position). To use it, invoke with map[R1] where map is the map returned by link; the callback op will be invoked for each affected constraint.

@@ -494,20 +462,18 @@ impl<'tcx> RegionInferenceContext<'tcx> {
while let Some(constraint_idx) = dirty_list.pop() {
clean_bit_vec.insert(constraint_idx.index());

let constraint = &self.constraints[constraint_idx];
let constraint = &self.constraints.inner()[constraint_idx];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this would be outlives_constraints[constraint_idx] with the Index impl I proposed

@@ -971,7 +928,7 @@ impl<'tcx> RegionInferenceContext<'tcx> {
);

let blame_index = self.blame_constraint(longer_fr, shorter_fr);
let blame_span = self.constraints[blame_index].span;
let blame_span = self.constraints.inner()[blame_index].span;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similarly here

@@ -1062,7 +1019,7 @@ impl<'tcx> RegionInferenceContext<'tcx> {
// - `fr1: X` transitively
// - and `Y` is live at `elem`
let index = self.blame_constraint(fr1, elem);
let region_sub = self.constraints[index].sub;
let region_sub = self.constraints.inner()[index].sub;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and here

@nikomatsakis
Copy link
Contributor

Actually, maybe we should just remove inner() and instead implement Deref:

impl Deref for ConstraintSet {
    type Target = IndexVec<ConstraintIndex, OutlivesConstraint>;

    fn deref(&self) -> &Self::Target { &self.constriants }
}

then you can do things like constraint_set[idx] but also constraint_set.iter() and so forth.

It seems pretty reasonable to say that a constraint set can be viewed as an IndexVec<ConstriantIndex, OutlivesConstriant>

@nikomatsakis
Copy link
Contributor

r=me once travis is happy

@Eh2406
Copy link
Contributor Author

Eh2406 commented Jun 27, 2018

do you want to profile/benchmark to confirm that this had the intended effect?

@Eh2406
Copy link
Contributor Author

Eh2406 commented Jun 28, 2018

Travis is happy.

@bors
Copy link
Contributor

bors commented Jun 28, 2018

☔ The latest upstream changes (presumably #51538) made this pull request unmergeable. Please resolve the merge conflicts.

@nikomatsakis
Copy link
Contributor

@Eh2406 merge conflicts :(

Regarding profiling, we could -- but we also profile the effect of every PR that lands, so either way we'll find out, and it's hard to imagine how this could make things worse.

@Eh2406
Copy link
Contributor Author

Eh2406 commented Jun 28, 2018

I did a rebase and pushed. I am getting a local error, but it is something really random so, I think it is just me.

@nikomatsakis
Copy link
Contributor

@bors r+

@bors
Copy link
Contributor

bors commented Jun 28, 2018

📌 Commit 6e0cefe has been approved by nikomatsakis

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 28, 2018
@rust-highfive
Copy link
Collaborator

Your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_fold:end:services

travis_fold:start:git.checkout
travis_time:start:114f52e8
$ git clone --depth=2 --branch=try https://github.com/rust-lang/rust.git rust-lang/rust
---
Cloning into 'rust-lang/rust'...
travis_time:end:114f52e8:start=1530235919998108762,finish=1530235925383248501,duration=5385139739
$ cd rust-lang/rust
$ git checkout -qf fbf3a9f0e182e3e202e674cc355ad3598956bb90
fatal: reference is not a tree: fbf3a9f0e182e3e202e674cc355ad3598956bb90
The command "git checkout -qf fbf3a9f0e182e3e202e674cc355ad3598956bb90" failed and exited with 128 during .
Your build has been stopped.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@bors bors added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jun 29, 2018
@kennytm
Copy link
Member

kennytm commented Jun 29, 2018

@bors retry

@bors
Copy link
Contributor

bors commented Jun 29, 2018

⌛ Trying commit 6e0cefe with merge 9bc7781a7a9a89400bceb10d87ac3580e61c78f8...

@bors
Copy link
Contributor

bors commented Jun 29, 2018

☀️ Test successful - status-travis
State: approved= try=True

@nikomatsakis
Copy link
Contributor

cc @Mark-Simulacrum ok, bors try is done, perf run please 😁

@Mark-Simulacrum
Copy link
Member

Perf queued.

@Eh2406
Copy link
Contributor Author

Eh2406 commented Jun 29, 2018

do we have a ETA on perf?

@Mark-Simulacrum
Copy link
Member

About 2 hours, maybe a little less.

@kennytm kennytm removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 30, 2018
@nikomatsakis
Copy link
Contributor

Kind of surprising how little difference this makes =(

Still, I think the code is nicer this way. I'm still inclined to r+

@nikomatsakis
Copy link
Contributor

Looking at some graphs, I think I see why this has minimal impact. Most of the duplicate edges I see appear to be self loops (X: X), and the merge operation already detects and optimizes that case:

    pub fn merge(&mut self, read: R, write: R) -> bool {
        let mut changed = false;

        if read != write {
            let (bit_set_read, bit_set_write) = self.vector.pick2_mut(read, write);

            for read_chunk in bit_set_read.chunks() {
                changed = changed | bit_set_write.insert_chunk(read_chunk).any();
            }
        }

        changed
    }

@nikomatsakis
Copy link
Contributor

So my estimate was mistaken.

It turns out that we don't have duplicates, just self-cycles.
@nikomatsakis
Copy link
Contributor

@bors r+

OK, so, although this didn't yield the perf win I expected, I think it's nice to have the constraint set extracted, and I also think it might lay the groundwork for some future changes I was contemplating experimenting with. However, since the hashset is not really helping us, I took the liberty of removing it with a quick commit.

I have some ideas for follow-up work here though it will be a bit more involved. I may take a stab at it today if I have some free time -- we'll see though -- but otherwise I'll open an issue. @Eh2406, maybe you'd be interested in doing that?

@bors
Copy link
Contributor

bors commented Jul 1, 2018

📌 Commit ac5bd5d has been approved by nikomatsakis

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 1, 2018
@bors
Copy link
Contributor

bors commented Jul 1, 2018

⌛ Testing commit ac5bd5d with merge e100ce4...

bors added a commit that referenced this pull request Jul 1, 2018
A fix for 51821

This dedupe the vec of `OutlivesConstraint` using a `FxHashSet<(RegionVid, RegionVid)>` it alsow adds a `struct ConstraintSet` to encapsulate/ensure this behavere.
@bors
Copy link
Contributor

bors commented Jul 1, 2018

☀️ Test successful - status-appveyor, status-travis
Approved by: nikomatsakis
Pushing e100ce4 to master...

@Eh2406
Copy link
Contributor Author

Eh2406 commented Jul 1, 2018

Feel free to ping me, but don't wait on me. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants