Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix LinkedList invalidating mutable references #60072

Merged
merged 1 commit into from Apr 19, 2019

Conversation

Projects
None yet
6 participants
@RalfJung
Copy link
Member

commented Apr 18, 2019

The test test_insert_prev failed in Miri due to what I consider a bug in LinkedList: in various places, NonNull::as_mut got called to modify the prev/next pointers of existing nodes. In particular, the unstable insert_next has to modify the next pointer of the node that was last handed out by the iterator; to this end it creates a mutable reference to the entire node that overlaps with the mutable reference to the node's content that was handed out by the iterator! Thus, the next use if said mutable reference is UB.

In code:

            loop {
                match it.next() { // mutable reference handed to us
                    None => break,
                    Some(elt) => {
                        it.insert_next(*elt + 1); // this invalidates `elt` because it creates an overlapping mutable reference
                        match it.peek_next() {
                            Some(x) => assert_eq!(*x, *elt + 2), // this use of `elt` now is a use of an invalid pointer
                            None => assert_eq!(8, *elt),
                        }
                    }
                }
            }

This PR fixes that by using as_ptr instead of as_mut. This avoids invalidating the mutable reference that was handed to the user. I did this in all methods called by iterators, just to be sure.

Cc @Gankro

@rust-highfive

This comment has been minimized.

Copy link
Collaborator

commented Apr 18, 2019

r? @shepmaster

(rust_highfive has picked a reviewer for you, use r? to override)

Some(mut head) => head.as_mut().prev = node,
// Not creating new mutable (unique!) references to not invalidate
// references we handed out.
Some(head) => (*head.as_ptr()).prev = node,

This comment has been minimized.

Copy link
@shepmaster

shepmaster Apr 18, 2019

Member

I'm not sure I even understand what is going on here...

Previously, as_mut returned a &mut Node<T>, which we then modified the prev field of. That much makes sense.

Now, we get a *mut Node<T>, dereference it to get a Node<T> then set the prev field of.

I've been under the impression that setting the field of a struct implicitly created a mutable reference — where am I wrong? If I'm not wrong, why aren't we creating a second mutable reference here?

This comment has been minimized.

Copy link
@Gankro

Gankro Apr 19, 2019

Contributor

I believe a mutable borrow is established only if you create an &mut that isn't immediately explicitly cast to a *mut. It's very janky immediate-action rules, with similar logic to why you can take a reference to the output of arr[x] and it's not a reference to a temporary.

Regardless for a lot of these the issue isn't a mutable borrow, but rather a mutable borrow that overlaps with node.elem. By writing the code this way we never claim unique ownership of the whole type, just that we can mutate the next/prev links.

This comment has been minimized.

Copy link
@RalfJung

RalfJung Apr 19, 2019

Author Member

I believe a mutable borrow is established only if you create an &mut that isn't immediately explicitly cast to a *mut.

No, that would be rust-lang/rfcs#2582 which is not yet in.

I've been under the impression that setting the field of a struct implicitly created a mutable reference

Writing to a field has all the same effects (in terms of invalidating other references) as creating a mutable reference to it, yes.

The key point is that we only modify the prev field, and the aliasing reference we are worried about points to the data field (element or whatever it is called). I will expand the comments to clarify this.

By writing the code this way we never claim unique ownership of the whole type, just that we can mutate the next/prev links.

s/type/node/. But yes.

This comment has been minimized.

Copy link
@RalfJung

RalfJung Apr 19, 2019

Author Member

@shepmaster I tried to improve the comments explaining why these changes are needed; does this make more sense now?

This comment has been minimized.

Copy link
@shepmaster

shepmaster Apr 19, 2019

Member

Writing to a field has all the same effects (in terms of invalidating other references) as creating a mutable reference to it.

Can you remove the pronoun here? Do you mean:

  • Writing to a struct's field has the same effects as creating a mutable reference to
    • the struct
    • the field

This comment has been minimized.

Copy link
@RalfJung

RalfJung Apr 19, 2019

Author Member

I mean "to the field".

It's really the other way around -- creating a mutable reference has the effects of a write to the memory that is "covered" by the reference (size_of::<T> bytes starting at where the pointer points).

@Gankro

Gankro approved these changes Apr 19, 2019

Copy link
Contributor

left a comment

seems fine, r=me, but I'm not actually a reviewer, so

@@ -143,14 +146,18 @@ impl<T> LinkedList<T> {
/// Adds the given node to the front of the list.
#[inline]
fn push_front_node(&mut self, mut node: Box<Node<T>>) {
// This method takes care not to create mutable references, to maintain
// validity of aliasing pointers into existing nodes.

This comment has been minimized.

Copy link
@Gankro

Gankro Apr 19, 2019

Contributor

could you clarify that the aliasing pointers are to the elements so it is specifically fine for us to prod at all the next/prev links with wild abandon?

This comment has been minimized.

Copy link
@RalfJung

RalfJung Apr 19, 2019

Author Member

Good point, done.

None => self.list.push_back(element), // push_back is okay with aliasing nodes
Some(head) => unsafe {
let prev = match head.as_ref().prev {
// push_back is okay with aliasing nodes

This comment has been minimized.

Copy link
@lqd

lqd Apr 19, 2019

Contributor

should that be "push_front is okay with aliasing nodes" ?

This comment has been minimized.

Copy link
@RalfJung

RalfJung Apr 19, 2019

Author Member

Indeed, thanks! Fixed.

@shepmaster

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

Thank you for the clarifications! r=me after squashing ❤️

@RalfJung RalfJung force-pushed the RalfJung:linked-list branch from 416fe81 to 8b09d04 Apr 19, 2019

@shepmaster

This comment has been minimized.

Copy link
Member

commented Apr 19, 2019

@bors r+ rollup

@bors

This comment has been minimized.

Copy link
Contributor

commented Apr 19, 2019

📌 Commit 8b09d04 has been approved by shepmaster

@bors

This comment has been minimized.

Copy link
Contributor

commented Apr 19, 2019

⌛️ Testing commit 8b09d04 with merge 130dc3e...

bors added a commit that referenced this pull request Apr 19, 2019

Auto merge of #60072 - RalfJung:linked-list, r=shepmaster
fix LinkedList invalidating mutable references

The test `test_insert_prev` failed in Miri due to what I consider a bug in `LinkedList`: in various places, `NonNull::as_mut` got called to modify the `prev`/`next` pointers of existing nodes. In particular, the unstable `insert_next` has to modify the `next` pointer of the node that was last handed out by the iterator; to this end it creates a mutable reference to the *entire node* that overlaps with the mutable reference to the node's content that was handed out by the iterator! Thus, the next use if said mutable reference is UB.

In code:
```rust
            loop {
                match it.next() { // mutable reference handed to us
                    None => break,
                    Some(elt) => {
                        it.insert_next(*elt + 1); // this invalidates `elt` because it creates an overlapping mutable reference
                        match it.peek_next() {
                            Some(x) => assert_eq!(*x, *elt + 2), // this use of `elt` now is a use of an invalid pointer
                            None => assert_eq!(8, *elt),
                        }
                    }
                }
            }
```

This PR fixes that by using `as_ptr` instead of `as_mut`. This avoids invalidating the mutable reference that was handed to the user.  I did this in all methods called by iterators, just to be sure.

Cc @Gankro
@bors

This comment has been minimized.

Copy link
Contributor

commented Apr 19, 2019

☀️ Test successful - checks-travis, status-appveyor
Approved by: shepmaster
Pushing 130dc3e to master...

@bors bors added the merged-by-bors label Apr 19, 2019

@bors bors merged commit 8b09d04 into rust-lang:master Apr 19, 2019

2 checks passed

Travis CI - Pull Request Build Passed
Details
homu Test successful
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.