New destructor semantics

[nikomatsakis]

Tracking issue for RFC #769, Sound Generic Drop.

Original description follows:

We agreed in a meeting to replace the unsafe destructor code with the rule that a value with a destructor must only contain values of lifetime strictly greater than the value to be destructed. The idea is to prevent values from referencing one another. Permitting borrowed values in destructors enables a number of RAII causes and helps to eliminate the need for once fns. This also relies on the change to prevent & pointers from being placed within managed boxes (which I think has already been made).

I have to figure out precisely how to formalize this rule still. =)

Nominating for backwards compat.

---

Update from @pnkfelix : Much of this is implemented, and so pnkfelix is listing out specific subtasks, issues, and/or PRs, to try to track what is done and what is left to do:

• Track finer-grained scopes (i.e. block suffixes): See PR #21657
• Land the drop-checker ("dropck"). See PR #21972
• Keep Drop from adding new bounds on a type param; Issue #8142
• Revise for-loops so that temporaries from the <iterator-expr> only live for the for-loop itself. (i.e. turn for-loops into a terminating scope). Addresses the bulk Issue #21114. Resolved by #21984
• Remove #[unsafe_destructor] attribute entirely; Issue #22196

Far future / No longer deemed necessary

• Make Vec<T>, Box<T>, Rc<T>, etc, covariant with respect to T. Discussed on issue #21198 (may not be a strict requirement for landing new destructor semantics).
• Track even finer grained scopes, e.g. destruction order of temporary rvalues in an expression. Issue #22323

(See also #22321)

[catamorphism]

Accepted for well-defined

[nikomatsakis]

Here is a specific example, from #13246, of unsound code we currently accept due to inadequate rules:

This program:

struct Foo { a: int }

impl Drop for Foo {
    fn drop(&mut self) {
        println!("{}", self.a);
    }
}

fn main() {
    {
        let _1 = Foo { a: 1 };
        let _2 = Foo { a: 2 };
        match Foo { a: 3 } {
            _ => {}
        }
    }
    let _4 = Foo { a: 4 };
}

should output

3
2
1
4

but instead outputs

2
1
3
4

It appears that if the bare block around the first part of main is removed, everything runs in the correct order. Moving the "3" Foo into a variable also makes everything run in the right order.

This was originally reported to me in sfackler/rust-postgres#31 as a segfault, but it looks like that's probably just due to heap corruption from a use-after-free caused by this bug.

Update

IR:

join:                                             ; preds = %case_body
  call void @_ZN3Foo14glue_drop.146917h02f639dd107d06daE(%struct.Foo* %_2)
  call void @_ZN3Foo14glue_drop.146917h02f639dd107d06daE(%struct.Foo* %_1)
  call void @_ZN3Foo14glue_drop.146917h02f639dd107d06daE(%struct.Foo* %0) ; <-- 3
  %7 = getelementptr inbounds %struct.Foo* %_4, i32 0, i32 1
  store i8 1, i8* %7
  %8 = getelementptr inbounds %struct.Foo* %_4, i32 0, i32 0
  store i64 4, i64* %8
  call void @_ZN3Foo14glue_drop.146917h02f639dd107d06daE(%struct.Foo* %_4)
  ret void

[pcwalton]

Another problem:

use std::cell::RefCell;
fn main() {
    let b = {
        let a = box RefCell::new(4);
        *a.borrow() + 1
    };
    println!("{}", b);
}

$ valgrind ./foo
==9880== Memcheck, a memory error detector
==9880== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al.
==9880== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
==9880== Command: ./foo
==9880== 
==9880== Invalid read of size 8
==9880==    at 0x403AC2: cell::Cell$LT$T$GT$::get::h14749520627650063431::v0.0 (in /home/alex/foo)
==9880==    by 0x403C2A: cell::Ref$LT$$x27b$C$$x20T$GT$.Drop::drop::h5517615880807730945::v0.0 (in /home/alex/foo)
==9880==    by 0x40406A: core..cell..Ref$LT$$x27_$C$int$GT$::glue_drop.1537::h7a74728680c53b3a (in /home/alex/foo)
==9880==    by 0x4031E1: main::hfd2b07b8610388fafaa::v0.0 (in /home/alex/foo)
==9880==    by 0x447512: start::closure.7199 (in /home/alex/foo)
==9880==    by 0x463862: task::Task::run::closure.5303 (in /home/alex/foo)
==9880==    by 0x466C4B: rust_try (in /home/alex/foo)
==9880==    by 0x465565: unwind::try::h7d720262cba71fd0fGd::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x463714: task::Task::run::hac578e5b353eedcbVVc::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x4472CD: start::h0149aeff7c3a4b69Jme::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x447098: lang_start::h3e00b03c17efac073le::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x40332E: main (in /home/alex/foo)
==9880==  Address 0x6014058 is 8 bytes inside a block of size 16 free'd
==9880==    at 0x471379: je_dallocx (in /home/alex/foo)
==9880==    by 0x40371F: heap::deallocate::h7bc897bb314e6fc3Bfa::v0.0 (in /home/alex/foo)
==9880==    by 0x403699: heap::exchange_free::h3a6062ae495b91fcefa::v0.0 (in /home/alex/foo)
==9880==    by 0x403FA0: _$UP$i8::glue_drop.1534::h08c160eb1ce0fd99 (in /home/alex/foo)
==9880==    by 0x4031D4: main::hfd2b07b8610388fafaa::v0.0 (in /home/alex/foo)
==9880==    by 0x447512: start::closure.7199 (in /home/alex/foo)
==9880==    by 0x463862: task::Task::run::closure.5303 (in /home/alex/foo)
==9880==    by 0x466C4B: rust_try (in /home/alex/foo)
==9880==    by 0x465565: unwind::try::h7d720262cba71fd0fGd::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x463714: task::Task::run::hac578e5b353eedcbVVc::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x4472CD: start::h0149aeff7c3a4b69Jme::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x447098: lang_start::h3e00b03c17efac073le::v0.11.0.pre (in /home/alex/foo)
==9880== 
==9880== Invalid write of size 8
==9880==    at 0x403B4A: cell::Cell$LT$T$GT$::set::h11020375487951902910::v0.0 (in /home/alex/foo)
==9880==    by 0x403DA8: cell::Ref$LT$$x27b$C$$x20T$GT$.Drop::drop::h5517615880807730945::v0.0 (in /home/alex/foo)
==9880==    by 0x40406A: core..cell..Ref$LT$$x27_$C$int$GT$::glue_drop.1537::h7a74728680c53b3a (in /home/alex/foo)
==9880==    by 0x4031E1: main::hfd2b07b8610388fafaa::v0.0 (in /home/alex/foo)
==9880==    by 0x447512: start::closure.7199 (in /home/alex/foo)
==9880==    by 0x463862: task::Task::run::closure.5303 (in /home/alex/foo)
==9880==    by 0x466C4B: rust_try (in /home/alex/foo)
==9880==    by 0x465565: unwind::try::h7d720262cba71fd0fGd::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x463714: task::Task::run::hac578e5b353eedcbVVc::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x4472CD: start::h0149aeff7c3a4b69Jme::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x447098: lang_start::h3e00b03c17efac073le::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x40332E: main (in /home/alex/foo)
==9880==  Address 0x6014058 is 8 bytes inside a block of size 16 free'd
==9880==    at 0x471379: je_dallocx (in /home/alex/foo)
==9880==    by 0x40371F: heap::deallocate::h7bc897bb314e6fc3Bfa::v0.0 (in /home/alex/foo)
==9880==    by 0x403699: heap::exchange_free::h3a6062ae495b91fcefa::v0.0 (in /home/alex/foo)
==9880==    by 0x403FA0: _$UP$i8::glue_drop.1534::h08c160eb1ce0fd99 (in /home/alex/foo)
==9880==    by 0x4031D4: main::hfd2b07b8610388fafaa::v0.0 (in /home/alex/foo)
==9880==    by 0x447512: start::closure.7199 (in /home/alex/foo)
==9880==    by 0x463862: task::Task::run::closure.5303 (in /home/alex/foo)
==9880==    by 0x466C4B: rust_try (in /home/alex/foo)
==9880==    by 0x465565: unwind::try::h7d720262cba71fd0fGd::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x463714: task::Task::run::hac578e5b353eedcbVVc::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x4472CD: start::h0149aeff7c3a4b69Jme::v0.11.0.pre (in /home/alex/foo)
==9880==    by 0x447098: lang_start::h3e00b03c17efac073le::v0.11.0.pre (in /home/alex/foo)
==9880== 
5
==9880== 
==9880== HEAP SUMMARY:
==9880==     in use at exit: 0 bytes in 0 blocks
==9880==   total heap usage: 18 allocs, 18 frees, 2,016 bytes allocated
==9880== 
==9880== All heap blocks were freed -- no leaks are possible
==9880== 
==9880== For counts of detected and suppressed errors, rerun with: -v
==9880== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

As much as I'd love to punt on things for 1.0, this seems like a serious issue we need to fix.

[pythonesque]

Does this rule mean code like this would be invalid?

extern crate arena;

fn main() {
    struct Foo<'a> { foo: ::std::cell::Cell<Option<&'a Foo<'a>>> }
    let foo = arena::TypedArena::new();
    let bar = foo.alloc(Foo { foo: ::std::cell::Cell::new(None) });
    bar.foo.set(Some(&*bar));
}

If so, I'm not really on board with it, as this is a very useful pattern (and is perfectly safe in this case), as this change seems to force the use of *const or *mut instead of lifetimes if you ever want cycles. This makes these cases less safe because the borrow checker will no longer verify that the stored collection doesn't move and isn't dropped until you're done with it.

[pythonesque]

As an addendum: this only seems to be a memory safety issue when two things occur together:

• A struct field has lifetime equal to that of the struct itself.
• The struct has an explicit destructor.

If the struct does not have an explicit destructor, then for each field, either the field is a reference (in which case there is no drop glue, so it can't cause memory unsafety), the field is a non-reference and has a lifetime equal to itself (in which case we can recursively perform the same check), or the field is a non-reference with a lifetime strictly greater than itself (in which case it's fine and we can move on to the next field).

This may not apply to TypedArena, depending on how the rule is interpreted, but it allows for many other safe patterns.

[aturon]

Nominating for -beta milestone.

[pnkfelix]

Moving ahead to the 1.0-beta milestone.

[pnkfelix]

for people who come to this issue url due to its many references on github, the code base, and IRC: I suspect the most relevant thing for you to look at right now is the Sound Generic Drop RFC PR: rust-lang/rfcs#769

[pnkfelix]

The description for #21972 accidentally closed this ticket, but there is still work to be done; reopening.

Also, assigning to myself since I pretty much own this topic at this point.

[pnkfelix]

reclassifying P-high, not 1.0 since everything for 1.0 is pretty much done (hopefully `unsafe_destructor will indeed be gone).

[pnkfelix]

closing; the remaining (hypothetical or far-future) work items are tracked on #22321