Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix overlapping references in BTree #58431

Merged
merged 2 commits into from Feb 23, 2019

Conversation

Projects
None yet
5 participants
@RalfJung
Copy link
Member

RalfJung commented Feb 13, 2019

This fixes two kinds of overlapping references in BTree (both found by running the BTree test suite in Miri).

In into_slices_mut, we did k.into_key_slice_mut() followed by self.into_val_slice_mut() (where k is a copy of self). Calling into_val_slice_mut calls self.len(), which creates a shared reference to NodeHeader, which unfortunately (due to padding) overlaps with the mutable reference returned by into_key_slice_mut. Hence the key slice got (partially) invalidated. The fix is to avoid creating an &NodeHeader after the first slice got created.

In the iterators, we used to first create the references that will be returned, and then perform the walk on the tree. Walking the tree creates references (such as &mut InternalNode) that overlap with all of the keys and values stored in a pointer; in particular, they overlap with the references the iterator will later return. This is fixed by reordering the operations of walking the tree and obtaining the inner references.

The test suite still passes (and it passes in Miri now!), but there is a lot of code here that I do not understand...

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

rust-highfive commented Feb 13, 2019

r? @Kimundi

(rust_highfive has picked a reviewer for you, use r? to override)

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

Mark-Simulacrum commented Feb 13, 2019

Since the ordering here changed, it might be worth trying to run some benchmarks as well (just to make sure we're not obviously regressing). I'm not sure if we have any in-tree though.

@RalfJung

This comment has been minimized.

Copy link
Member Author

RalfJung commented Feb 20, 2019

Without the patch:

test btree::map::find_rand_100                           ... bench:          20 ns/iter (+/- 4)
test btree::map::find_rand_10_000                        ... bench:          75 ns/iter (+/- 10)
test btree::map::find_seq_100                            ... bench:          20 ns/iter (+/- 9)
test btree::map::find_seq_10_000                         ... bench:          55 ns/iter (+/- 8)
test btree::map::insert_rand_100                         ... bench:          50 ns/iter (+/- 12)
test btree::map::insert_rand_10_000                      ... bench:          49 ns/iter (+/- 14)
test btree::map::insert_seq_100                          ... bench:          67 ns/iter (+/- 15)
test btree::map::insert_seq_10_000                       ... bench:         128 ns/iter (+/- 15)
test btree::map::iter_1000                               ... bench:       4,206 ns/iter (+/- 578)
test btree::map::iter_100000                             ... bench:     551,540 ns/iter (+/- 78,192)
test btree::map::iter_20                                 ... bench:          69 ns/iter (+/- 16)

With the patch:

running 11 tests
test btree::map::find_rand_100                           ... bench:          20 ns/iter (+/- 4)
test btree::map::find_rand_10_000                        ... bench:          78 ns/iter (+/- 31)
test btree::map::find_seq_100                            ... bench:          20 ns/iter (+/- 6)
test btree::map::find_seq_10_000                         ... bench:          55 ns/iter (+/- 7)
test btree::map::insert_rand_100                         ... bench:          50 ns/iter (+/- 11)
test btree::map::insert_rand_10_000                      ... bench:          50 ns/iter (+/- 17)
test btree::map::insert_seq_100                          ... bench:          63 ns/iter (+/- 16)
test btree::map::insert_seq_10_000                       ... bench:         131 ns/iter (+/- 12)
test btree::map::iter_1000                               ... bench:       4,132 ns/iter (+/- 448)
test btree::map::iter_100000                             ... bench:     555,513 ns/iter (+/- 86,294)
test btree::map::iter_20                                 ... bench:          70 ns/iter (+/- 21)

(I discarded a previous run because it reported a huge variance of >200,000ns)

Looks like no change to me.

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

Mark-Simulacrum commented Feb 20, 2019

@bors r+

Code changes look reasonable (and, well, don't actually change anything).

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Feb 20, 2019

📌 Commit f0bef49 has been approved by Mark-Simulacrum

Centril added a commit to Centril/rust that referenced this pull request Feb 22, 2019

Rollup merge of rust-lang#58431 - RalfJung:btree, r=Mark-Simulacrum
fix overlapping references in BTree

This fixes two kinds of overlapping references in BTree (both found by running the BTree test suite in Miri).

In `into_slices_mut`, we did `k.into_key_slice_mut()` followed by `self.into_val_slice_mut()` (where `k` is a copy of `self`). Calling `into_val_slice_mut` calls `self.len()`, which creates a shared reference to `NodeHeader`, which unfortunately (due to padding) overlaps with the mutable reference returned by `into_key_slice_mut`.  Hence the key slice got (partially) invalidated.  The fix is to avoid creating an `&NodeHeader` after the first slice got created.

In the iterators, we used to first create the references that will be returned, and then perform the walk on the tree.  Walking the tree creates references (such as `&mut InternalNode`) that overlap with all of the keys and values stored in a pointer; in particular, they overlap with the references the iterator will later return. This is fixed by reordering the operations of walking the tree and obtaining the inner references.

The test suite still passes (and it passes in Miri now!), but there is a lot of code here that I do not understand...

bors added a commit that referenced this pull request Feb 22, 2019

Auto merge of #58644 - Centril:rollup, r=Centril
Rollup of 17 pull requests

Successful merges:

 - #57656 (Deprecate the unstable Vec::resize_default)
 - #58059 (deprecate before_exec in favor of unsafe pre_exec)
 - #58064 (override `VecDeque::try_rfold`, also update iterator)
 - #58198 (Suggest removing parentheses surrounding lifetimes)
 - #58431 (fix overlapping references in BTree)
 - #58555 (Add a note about 2018e if someone uses `try {` in 2015e)
 - #58588 (remove a bit of dead code)
 - #58589 (cleanup macro after 2018 transition)
 - #58591 (Dedup a rustdoc diagnostic construction)
 - #58600 (fix small documentation typo)
 - #58601 (Search for target_triple.json only if builtin target not found)
 - #58606 (Docs: put Future trait into spotlight)
 - #58607 (Fixes #58586: Make E0505 erronous example fail for the 2018 edition)
 - #58615 (miri: explain why we use static alignment in ref-to-place conversion)
 - #58620 (introduce benchmarks of BTreeSet.intersection)
 - #58621 (Update miri links)
 - #58632 (Make std feature list sorted)

Failed merges:

r? @ghost

bors added a commit that referenced this pull request Feb 22, 2019

Auto merge of #58644 - Centril:rollup, r=Centril
Rollup of 17 pull requests

Successful merges:

 - #57656 (Deprecate the unstable Vec::resize_default)
 - #58059 (deprecate before_exec in favor of unsafe pre_exec)
 - #58064 (override `VecDeque::try_rfold`, also update iterator)
 - #58198 (Suggest removing parentheses surrounding lifetimes)
 - #58431 (fix overlapping references in BTree)
 - #58555 (Add a note about 2018e if someone uses `try {` in 2015e)
 - #58588 (remove a bit of dead code)
 - #58589 (cleanup macro after 2018 transition)
 - #58591 (Dedup a rustdoc diagnostic construction)
 - #58600 (fix small documentation typo)
 - #58601 (Search for target_triple.json only if builtin target not found)
 - #58606 (Docs: put Future trait into spotlight)
 - #58607 (Fixes #58586: Make E0505 erronous example fail for the 2018 edition)
 - #58615 (miri: explain why we use static alignment in ref-to-place conversion)
 - #58620 (introduce benchmarks of BTreeSet.intersection)
 - #58621 (Update miri links)
 - #58632 (Make std feature list sorted)

Failed merges:

r? @ghost

@bors bors merged commit f0bef49 into rust-lang:master Feb 23, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.