Fixes to mir dataflow #33667

pnkfelix · May 16, 2016

Fixes to mir dataflow

This collects a bunch of changes to rustc_borrowck::borrowck::dataflow (which others have pointed out should probably migrate to some crate that isn't tied to the borrow-checker -- but I have not attempted that here, especially since there are competing approaches to dataflow that we should also evaluate).

These changes:

Provide a family of related analyses: MovingOutStatements (which is what the old AST-based dataflo computed), as well as MaybeInitialized, MaybeUninitalized, and DefinitelyInitialized.
- (The last two are actually inverses of each other; we should pick one and drop the other.)
Fix bugs in the pre-existing analysis implementation, which was untested and thus some obvious bugs went unnoticed, which brings us to the third point:
Add a unit test infrastructure for the MIR dataflow analysis.
- The tests work by adding a new intrinsic that is able to query the analysis state for a particular expression (technically, a particular L-value).
- See the examples in compile-fail/mir-dataflow/inits-1.rs and compile-fail/mir-dataflow/uninits-1.rs
- These tests are only checking the results for MaybeInitialized, MaybeUninitalized, and DefinitelyInitialized; I am not sure if it will be feasible to generalize this testing strategy to the MovingOutStatements dataflow operator.

rust-highfive · May 16, 2016

r? @Aatch

(rust_highfive has picked a reviewer for you, use r? to override)

pnkfelix · May 16, 2016

r? @nikomatsakis

cc @arielb1

sanxiyn · May 16, 2016

sanxiyn reviewed May 16, 2016

View changes

sanxiyn · May 16, 2016

src/libcore/intrinsics.rs

+    /// attached to the function.
+    ///
+    /// For example, dataflow uses this to inject static assertions so
+    /// that `rustc_oeek(potentially_uninitialized)` would actually


Typo: oeek.

arielb1 · May 16, 2016

arielb1 reviewed May 16, 2016

View changes

arielb1 · May 16, 2016

src/librustc_borrowck/borrowck/mir/dataflow/sanity_check.rs

+        // it will always appear like Lvalues are initialized; e.g. in
+        // a case like:
+        //
+        // <bitset maps var1 to 0>


What is this talking about? rustc_peek(&var1) should be translated to

tmp1 = &var0; tmp0 = rustc_peek(tmp1) -> bb1;

If it copies the lvalue before taking a reference, that would be totally broken (consider Cell). So it doesn't. Unless something funny is going on here.

Oh you're right; I didn't update that comment correctly (it was originally authored for a different version of rustc_peek that looked like rustc_peek(var1) (instead of rustc_peek(&var1)), and I erroneously translated its content without thinking about what the actual translation of rustc_peek(&var1) is).

(The original version of rustc_peek had a more fragile API, where one would write rustc_peek(expr) (instead of rustc_peek(&expr)).)

The main point I'm trying to make is this: even with the corrected translation that you wrote:

tmp1 = &var0; tmp0 = rustc_peek(tmp1) -> bb1;

The normal (naive) dataflow GEN/KILL sets are going to say that tmp1 is initialized. If the semantics of rustc_peek were a naive "look directly at your argument's dataflow bit state", then rustc_peek would query the state of tmp1, which would always indicate that it has been initialized under the naive rules.

So I had written a comment about a work-around I was using for the naive system.

With the new API the story is somewhat related, but the example I gave in the comment isn't helping explain things.

I will update it.

I guessed. Yeah, you want the state of var0, not of tmp1. I think you could just go to the initialization of tmp1 and use that.

(I also expected to find the definition of tmp1 when @pnkfelix and I discussed in the work week; but this solution works too.)

~~I guess the reason not to do rustc_peek(&x) is that this code runs before borrowck anyway, so e.g. moves don't matter?~~ never mind, I see you are doing &x

nikomatsakis · May 17, 2016

nikomatsakis reviewed May 17, 2016

View changes

nikomatsakis · May 17, 2016

src/librustc_borrowck/borrowck/mir/dataflow.rs

+/// you if an l-value *might* be uninitialized at a given point in the
+/// control flow. But `MovingOutStatements` also includes the added
+/// data of *which* particular statement causing the deinitialization
+/// that the borrow checker's error meessage may need to report.


Intention is to compute this lazilly, right? Maybe note that?

nikomatsakis · May 17, 2016

nikomatsakis reviewed May 17, 2016

View changes

nikomatsakis · May 17, 2016

src/test/compile-fail/mir-dataflow/README.md

+
+These unit tests check the dataflow analysis by embedding calls to a
+special `rustc_peek` intrinsic within the code, in tandem with an
+attribute `#[rustc_mir(rustc_peek_maybe_init)]` (*). With that


Nit: use (\*) to disable markdown interpretation

nikomatsakis · May 17, 2016

I gave this a brief review and it seems quite reasonable. I'll try to read a bit more in detail tomorrow though.

pnkfelix · May 24, 2016

@arielb1 @nikomatsakis The one thing I'd like someone to take a look at is the new indexed_set abstraction that I am using as a layer atop bitvectors.

My goal was to have a breakdown analogous to Vec<T>/[T] (namely, one is owned, and the other is unsized and can be used with either &[T] or &mut [T]).

My analogous types in indexed_set are OwnIdxSet<T> and IdxSet<T>, where in both cases the phantom type parameter T must implement the trait Idx (also introduced in this module), since the T must be able to be mapped to a usize for the subsequent indexing into the bitvector.

My problem is that I used a little bit of nasty unsafe code to accomplish this goal. :( (Namely, I did not see a way to turn a &[usize] slice reference into a &IdxSet<T> besides using transmute.)

Maybe the right answer here is to not attempt to provide an unsized IdxSet<T> type, but instead to provide RefIdxSet<'a, T> and MutIdxSet<'a, T>. I briefly considered this but for some reason my initial drafts seemed really unappealing. Still, if either of you wants to insist that this be composed of 100% safe code, I can go back in and do that replacement.

pnkfelix · May 24, 2016

pnkfelix reviewed May 24, 2016

View changes

pnkfelix · May 24, 2016

src/librustc_borrowck/borrowck/mir/dataflow/mod.rs

+    /// "transfer-function" represnting the overall-effect of the
+    /// block, represented via GEN and KILL sets.
+    ///
+    /// The statement here is `idx_stmt.1`; `idx_stmt.0` is just


(note to self: this, like many comments, is out of date and needs to be fixed to reflect changes to the API here.)

nikomatsakis · May 25, 2016

nikomatsakis reviewed May 25, 2016

View changes

nikomatsakis · May 25, 2016

src/librustc_borrowck/indexed_set.rs

+// requires a transmute relying on representation guarantees that may
+// not hold in the future.
+
+pub struct IdxSet<T: Idx> {


I'd have appreciated a comment explaining what T is.

nikomatsakis · May 25, 2016

nikomatsakis reviewed May 25, 2016

View changes

nikomatsakis · May 25, 2016

src/librustc_borrowck/indexed_set.rs

+    fn idx(&self) -> usize;
+}
+
+pub struct OwnIdxSet<T: Idx> {


I would prefer the name IdxSetBuf, following the precedent of PathBuf and Path

I'll change the name.

nikomatsakis · May 25, 2016

nikomatsakis reviewed May 25, 2016

View changes

nikomatsakis · May 25, 2016

src/librustc_borrowck/indexed_set.rs

+}
+
+pub struct OwnIdxSet<T: Idx> {
+    _pd: PhantomData<fn(&[T], usize) -> &T>,


it'd be good to explain where this phantomdata comes from, since it's clearly quite deliberate...

In terms of the intuition for phantdomdata, I think of this is basically being a more optimized version of a FnvHashSet<T>, hence I would expect PhantomData<T>, effectively. Not that this matters much in practice. :)

Oh this PhantomData got things mixed up; you are right that PhantomData<T> would (probably) suffice here, (but I don't think its 100% accurate).

(I had originally attempted to put the actual represented type into the interface, e.g. make a connection from MovePathIndex to MovePath, and then the goal was to have a phantom data that would say Fn(&[MovePath], MovePathIndex) -> Option<&MovePath>. But as I reworked the API and removed references to the referenced data type, I didn't notice that I had mixed up the roles of T (the index type) and the referenced data type (now unnamed here).

I will replace with PhantomData<Fn(&T)> since that more properly reflects the relationship between the IdxSet and the Idx that each is fed.

IIRC Fn(A) is invariant over A so maybe you want fn(A)?

nikomatsakis · May 25, 2016

My feeling is that we should land this PR -- it seems like it contains many clear improvements and bug fixes, and it's big enough that we could keep iterating forever with nitpicks. I'd rather land it and then address further problems in follow-up PRs. @arielb1 -- thoughts?

arielb1 · May 25, 2016

I'll go for that

@bors r+

bors · May 25, 2016

📌 Commit df5c116 has been approved by arielb1

bors · May 25, 2016

⌛️ Testing commit df5c116 with merge 34fd686...

bors · May 25, 2016

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-32cross-opt, auto-linux-64-cargotest, auto-linux-64-cross-armhf, auto-linux-64-cross-armsf, auto-linux-64-cross-freebsd, auto-linux-64-cross-netbsd, auto-linux-64-debug-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-opt-mir, auto-linux-64-opt-rustbuild, auto-linux-64-x-android-t, auto-linux-cross-opt, auto-linux-musl-64-opt, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-mac-64-opt-rustbuild, auto-mac-cross-ios-opt, auto-win-gnu-32-nopt-t, auto-win-gnu-32-opt, auto-win-gnu-32-opt-rustbuild, auto-win-gnu-64-nopt-t, auto-win-gnu-64-opt, auto-win-msvc-32-cross-opt, auto-win-msvc-32-opt, auto-win-msvc-64-cargotest, auto-win-msvc-64-opt, auto-win-msvc-64-opt-mir, auto-win-msvc-64-opt-rustbuild

nikomatsakis · May 25, 2016

Yay!
On May 25, 2016 6:38 PM, "bors" notifications@github.com wrote:

☀️ Test successful - auto-linux-32-nopt-t
http://buildbot.rust-lang.org/builders/auto-linux-32-nopt-t/builds/9268,
auto-linux-32-opt
http://buildbot.rust-lang.org/builders/auto-linux-32-opt/builds/9305,
auto-linux-32cross-opt
http://buildbot.rust-lang.org/builders/auto-linux-32cross-opt/builds/388,
auto-linux-64-cargotest
http://buildbot.rust-lang.org/builders/auto-linux-64-cargotest/builds/580,
auto-linux-64-cross-armhf
http://buildbot.rust-lang.org/builders/auto-linux-64-cross-armhf/builds/391,
auto-linux-64-cross-armsf
http://buildbot.rust-lang.org/builders/auto-linux-64-cross-armsf/builds/395,
auto-linux-64-cross-freebsd
http://buildbot.rust-lang.org/builders/auto-linux-64-cross-freebsd/builds/394,
auto-linux-64-cross-netbsd
http://buildbot.rust-lang.org/builders/auto-linux-64-cross-netbsd/builds/394,
auto-linux-64-debug-opt
http://buildbot.rust-lang.org/builders/auto-linux-64-debug-opt/builds/2536,
auto-linux-64-nopt-t
http://buildbot.rust-lang.org/builders/auto-linux-64-nopt-t/builds/9243,
auto-linux-64-opt
http://buildbot.rust-lang.org/builders/auto-linux-64-opt/builds/9229,
auto-linux-64-opt-mir
http://buildbot.rust-lang.org/builders/auto-linux-64-opt-mir/builds/815,
auto-linux-64-opt-rustbuild
http://buildbot.rust-lang.org/builders/auto-linux-64-opt-rustbuild/builds/1200,
auto-linux-64-x-android-t
http://buildbot.rust-lang.org/builders/auto-linux-64-x-android-t/builds/9225,
auto-linux-cross-opt
http://buildbot.rust-lang.org/builders/auto-linux-cross-opt/builds/2436,
auto-linux-musl-64-opt
http://buildbot.rust-lang.org/builders/auto-linux-musl-64-opt/builds/4380,
auto-mac-32-opt
http://buildbot.rust-lang.org/builders/auto-mac-32-opt/builds/9285,
auto-mac-64-nopt-t
http://buildbot.rust-lang.org/builders/auto-mac-64-nopt-t/builds/9290,
auto-mac-64-opt
http://buildbot.rust-lang.org/builders/auto-mac-64-opt/builds/9255,
auto-mac-64-opt-rustbuild
http://buildbot.rust-lang.org/builders/auto-mac-64-opt-rustbuild/builds/1209,
auto-mac-cross-ios-opt
http://buildbot.rust-lang.org/builders/auto-mac-cross-ios-opt/builds/393,
auto-win-gnu-32-nopt-t
http://buildbot.rust-lang.org/builders/auto-win-gnu-32-nopt-t/builds/4323,
auto-win-gnu-32-opt
http://buildbot.rust-lang.org/builders/auto-win-gnu-32-opt/builds/4330,
auto-win-gnu-32-opt-rustbuild
http://buildbot.rust-lang.org/builders/auto-win-gnu-32-opt-rustbuild/builds/1218,
auto-win-gnu-64-nopt-t
http://buildbot.rust-lang.org/builders/auto-win-gnu-64-nopt-t/builds/4323,
auto-win-gnu-64-opt
http://buildbot.rust-lang.org/builders/auto-win-gnu-64-opt/builds/4309,
auto-win-msvc-32-cross-opt
http://buildbot.rust-lang.org/builders/auto-win-msvc-32-cross-opt/builds/679,
auto-win-msvc-32-opt
http://buildbot.rust-lang.org/builders/auto-win-msvc-32-opt/builds/3530,
auto-win-msvc-64-cargotest
http://buildbot.rust-lang.org/builders/auto-win-msvc-64-cargotest/builds/583,
auto-win-msvc-64-opt
http://buildbot.rust-lang.org/builders/auto-win-msvc-64-opt/builds/4321,
auto-win-msvc-64-opt-mir
http://buildbot.rust-lang.org/builders/auto-win-msvc-64-opt-mir/builds/822,
auto-win-msvc-64-opt-rustbuild
http://buildbot.rust-lang.org/builders/auto-win-msvc-64-opt-rustbuild/builds/1217

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#33667 (comment)

rust-highfive assigned Aatch May 16, 2016

rust-highfive assigned nikomatsakis and unassigned Aatch May 16, 2016

pnkfelix referenced this pull request May 16, 2016
Merged
[MIR] non-zeroing drop #33622

core::intrinsics: fix typo noted during review.

Loading status checks…

cd71b0d

pnkfelix added some commits May 25, 2016

Fix some comments.

d9680f5

Add notes that data-structures should potentially move to different c…
…rate.

Loading status checks…

25f37fd

bors merged commit df5c116 into rust-lang:master May 25, 2016
1 check passed

1 check passed

homu Test successful
Details

bors referenced this pull request May 25, 2016
Merged
MIR: Don't generate 3-armed boolean switch from match. #33583

rust-lang/rust

Join GitHub today

Fixes to mir dataflow #33667

Conversation

pnkfelix commented May 16, 2016 • edited

pnkfelix added some commits Apr 28, 2016

rust-highfive assigned Aatch May 16, 2016

This comment has been minimized.

rust-highfive commented May 16, 2016

This comment has been minimized.

pnkfelix commented May 16, 2016

rust-highfive assigned nikomatsakis and unassigned Aatch May 16, 2016

pnkfelix referenced this pull request May 16, 2016

sanxiyn reviewed May 16, 2016 View changes

This comment has been minimized.

arielb1 reviewed May 16, 2016 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

nikomatsakis May 17, 2016 • edited

nikomatsakis reviewed May 17, 2016 View changes

This comment has been minimized.

nikomatsakis May 17, 2016 • edited

nikomatsakis reviewed May 17, 2016 View changes

This comment has been minimized.

This comment has been minimized.

nikomatsakis commented May 17, 2016

pnkfelix added some commits May 17, 2016

pnkfelix added some commits May 24, 2016

This comment has been minimized.

pnkfelix commented May 24, 2016 • edited

pnkfelix reviewed May 24, 2016 View changes

This comment has been minimized.

pnkfelix added some commits May 25, 2016

nikomatsakis reviewed May 25, 2016 View changes

This comment has been minimized.

nikomatsakis reviewed May 25, 2016 View changes

This comment has been minimized.

This comment has been minimized.

nikomatsakis reviewed May 25, 2016 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

pnkfelix May 25, 2016 • edited

This comment has been minimized.

This comment has been minimized.

nikomatsakis commented May 25, 2016

pnkfelix added some commits May 25, 2016

This comment has been minimized.

arielb1 commented May 25, 2016

This comment has been minimized.

bors commented May 25, 2016

This comment has been minimized.

bors commented May 25, 2016

bors added a commit that referenced this pull request May 25, 2016

This comment has been minimized.

bors commented May 25, 2016

Hide details View details bors merged commit df5c116 into rust-lang:master May 25, 2016 1 check passed

1 check passed

bors referenced this pull request May 25, 2016

This comment has been minimized.

nikomatsakis commented May 25, 2016

pnkfelix commented May 16, 2016 •

edited

sanxiyn reviewed May 16, 2016

View changes

arielb1 reviewed May 16, 2016

View changes

nikomatsakis May 17, 2016 •

edited

nikomatsakis reviewed May 17, 2016

View changes

nikomatsakis May 17, 2016 •

edited

nikomatsakis reviewed May 17, 2016

View changes

pnkfelix commented May 24, 2016 •

edited

pnkfelix reviewed May 24, 2016

View changes

nikomatsakis reviewed May 25, 2016

View changes

nikomatsakis reviewed May 25, 2016

View changes

nikomatsakis reviewed May 25, 2016

View changes

pnkfelix May 25, 2016 •

edited

bors merged commit `df5c116` into rust-lang:master May 25, 2016
1 check passed