`Drop` and leaking `&mut` references

[cuviper]

There's a way to leak a &mut member borrowed from a struct, and then manipulate that member from the Drop handler -- even while it is immutably borrowed!

struct VecWrapper<'a>(&'a mut Vec<u8>);

impl<'a> IntoIterator for VecWrapper<'a> {
    type Item = &'a u8;
    type IntoIter = std::slice::Iter<'a, u8>;

    fn into_iter(self) -> Self::IntoIter {
        self.0.iter()
    }
}

impl<'a> Drop for VecWrapper<'a> {
    fn drop(&mut self) {
        // aggressively free memory
        self.0.clear();
        self.0.shrink_to_fit();
    }
}

fn main() {
    let mut v = vec![1, 2, 3];
    for i in VecWrapper(&mut v) {
        let s = "foo".to_owned(); // reused allocation
        println!("{}!  {} {:#x} '{}'", s, i, i, *i as char);
    }
}

playground output:

foo!  102 0x66 'f'
foo!  111 0x6f 'o'
foo!  111 0x6f 'o'

So this drop frees the vec's buffer, then an unrelated string allocation reuses the same memory, and those new values come out of the vec::Iter.

I think if the member was immutable &, then this "leak" from into_iter() would be fine since they can both share that reference, and Drop can't mutate it. But &mut must be exclusive, of course...

[nagisa]

The behaviour feels somewhat intuitive to me, but is certainly a memory safety issue. Nominating.

cc @rust-lang/lang, I think?

EDIT: affects MIR as well, without an obvious fix.
EDIT2: Also @rust-lang/libs, because this is caused by IntoIter::into_iter.

[arielb1]

Minified:

struct VecWrapper<'a>(&'a mut S);

struct S(Box<u32>);

fn get_dangling<'a>(v: VecWrapper<'a>) -> &'a u32 {
    let s_inner: &'a S = &*v.0;
    &s_inner.0
}

impl<'a> Drop for VecWrapper<'a> {
    fn drop(&mut self) {
        *self.0 = S(Box::new(0));
    }
}

fn main() {
    let mut s = S(Box::new(11));
    let vw = VecWrapper(&mut s);
    let dangling = get_dangling(vw);
    println!("{}", dangling);
}

MIR borrowck would of course catch get_dangling borrowing a pointer and dropping its owner, given that the relevant MIR is

fn(arg0: VecWrapper<'a>) -> &'a u32 {
    let var0: VecWrapper<'a>; // v
    let var1: &'a S; // s_inner
    let mut tmp0: &'a S;
    let mut tmp1: ();
    let mut tmp2: &'a Box<u32>;

    bb0: {
        var0 = arg0;
        tmp0 = &(*(var0.0: &'a mut S)); // borrows *var0.0 for 'a
        var1 = &(*tmp0);
        tmp2 = &((*var1).0: Box<u32>);
        return = &(*(*tmp2));
        drop(var0) -> bb1; // inside 'a, drops var0
    }

    bb1: {
        return;
    }
}

[bluss]

Here's another minification (playground)

struct DropSet<'a>(&'a mut u8);

impl<'a> DropSet<'a> {
    fn get(self) -> &'a u8 {
        self.0
    }
}

impl<'a> Drop for DropSet<'a> {
    fn drop(&mut self) {
        *self.0 = 0;
    }
}

fn main() {
    let mut x = 64;
    let s = DropSet(&mut x);
    let r = s.get();
    println!("{}", r);
}

[arielb1]

I suppose we can leave this to fix-by-MIR-borrowck.

[nikomatsakis]

The problem (I think) is the special case rule in the borrow checker concerning &mut borrows whose owners go out of scope. That always seemed a bit suspicious to me. It does not consider destructors. Shouldn't be too hard to fix.

[nikomatsakis]

triage: P-high

[pnkfelix]

so, just a quick note on some ways @nikomatsakis and I have talked about resolving this:

• My first instinct was to try to change either ExprUseVisitor or borrowck to inject the effect of mutating any local binding of type carrying a destructor at the end of its scope.
  • One might think of this as trying to re-create the effect of the drop calls that we already explicit emit in the MIR.
  • (However, this may also be too conservative; in particular, it depends on what one infers to be the side-effects of a drop call -- see further discussion below)
• After discussing my attempts to explore the previous approach(es) with @nikomatsakis , he explained what he was thinking of in earlier comment -- namely, there is code in borrowck that already ensures, for any borrow, that it does not outlive the lifetime of the so-called "guarantor" for the borrow.
  • @nikomatsakis suggested exploring changing that code to account for types that carry destructors
  • However, he also noted that the guarantor code currently breaks out of its loop as soon as it sees any reference, rather than continuing to recurse until finding the underlying owner.
  • (So clearly it wasn't going to be a three-line diff; some investigation was required.)
• I spent today exploring how to extract the underlying "base guarantor", as I am calling it, for a cmt, and then checking that against the region of the borrow.
  • I was pretty happy; the stdlib and compiler source only had one regression caused by the new check
  • Namely, this code for cell::RefMut::map
  • As an explanation, the reason that my check is complaining about this code is that RefMut has an attached destructor (since its borrow field implements Drop), and the orig.value is going to extend the lifetime of the borrowed value past the lifetime of orig itself. So check reasons that the destructor for the RefMut might mutate orig.value, making the alias illegal.
  • However, now that I've taken the time to explain this scenario, I have come to realize that this case is reflecting a weakness in my proposed analysis.
  • In particular, RefMut itself does not carry an impl Drop. It is non-Copy, sure, but the only thing that its destructor can do is recursively invoke the destructor for each of its fields. And that means that the only actual code that runs is on the orig.borrow field -- it has no access to the orig.value field.
• This seems at least superficially similar to some of the issues associated with dropck.
  • In particular, a more precise analysis here would distinguish between a type that itself implements Drop versus a type who merely carries a field that implements Drop.
  • Such increased precision would allow the above code from cell::RefMut to go through, and thus reduce the breakage injected by the proposed change.

[arielb1]

In particular, a more precise analysis here would distinguish between a type that itself implements Drop versus a type who merely carries a field that implements Drop.

Surely you mean distinguish between a type that has a destructor and a type that may implement Drop? We already do that to forbid destructuring Drop structs.

[pnkfelix]

@arielb1 yes I just meant the more precise analysis needs to use the appropriate predicate (i.e. that the type implements Drop), on the appropriate collection of types (plural) according to the l-value expression being borrowed.

[arielb1]

@pnkfelix

I think we can just make it that a MIR drop conflicts with a loan of an lvalue only if there is a destructor "upstream" of that lvalue and not behind a &-reference. There is also a lifetime.end conflict which does not look through &mut, but we already do that.

[nikomatsakis]

@pnkfelix and I were talking. I think this can be cleanly integrated into regionck -- in fact, some of the rules that are enforced in borrowck (I suspect) ought to moved to regionck. I'm going to try and tinker a bit here.

[brson]

@rust-lang/compiler can you assign someone to this P-high bug?

[brson]

I tagged this a regression from a quick scan, but not sure.

[brson]

Ah, felix is assigned. Ignore me.

[eddyb]

@brson "stable", "nightly" or "version" do not appear in any comment here and the only mention of a regression is @pnkfelix discussing a potential solution.
IIUC, the problem here can be reproduced on any Rust version in existence, the needed checks were never there. I would untag as regression, or rather leave it to @pnkfelix since he's already on this.

[cuviper]

I can reproduce this all the way back to 1.0.0, at least.

[pnkfelix]

I suspect I won't be able to address this myself in this cycle, so keeping this at P-high is not a great idea.

@arielb1 suggests we should let this be fixed by switching to MIR-based borrowck.

[arielb1]

@stjepang

This errors correctly with -Z borrowck=mir without NLL, so this is an NLL issue - looks like what I reported as #47470

[nikomatsakis]

Fixed by #48411