Separate "no UnsafeCell" property into separate Immutable trait; allow FromZeros, FromBytes, and AsBytes on types with UnsafeCells

[joshlf]

Summary

Introduce a Immutable trait which indicates that a type contains no UnsafeCells. Allow FromZeros, FromBytes, and AsBytes on types with UnsafeCells and use Immutable as a bound on individual methods/functions.

This enables by-value transmutations such as transmute! on types containing UnsafeCells.

See also #694.

Progress

• Add Immutable trait
  • Initial commit of NoCell #656
• Allow deriving Immutable
  • [derive] Support deriving NoCell #667
• Allow FromZeros, FromBytes, and AsBytes on UnsafeCell; add Immutable bound on methods as needed
  • Traits other than NoCell permit UnsafeCells #682
• Rename AsBytes to IntoBytes #695
• Only implement other traits on sized UnsafeCell
  • In #682, we implemented FromZeros, FromBytes, and AsBytes for unsized UnsafeCell<T> (ie, where T: ?Sized). This causes problems for TryFromBytes, and so we've decided to restrict to T: Sized (see the discussion below).
• Allow TryFromBytes on UnsafeCell; add Immutable bound on try_from_ref method
  • [TryFromBytes] Permit UnsafeCells #890
• Remove #[doc(hidden)] from Immutable and pub use zerocopy_derive::Immutable
  • Document NoCell #977
• Consider renaming to be consistent with whatever name Rust chooses for what is currently called Freeze since we will likely treat Immutable as merely a polyfill for that trait
• Loosen documentation to promise only "no interior mutability" (TODO: how do we define this formally?) and say that it is sufficient to prove that a type (recursively) contains no UnsafeCells, but not necessary, and that callers cannot assume that T: Immutable implies that T contains no UnsafeCells. Note that our code will need to rely on this implication, and all safety comments should be updated to point at an issue that tracks this change.
• Non-breaking changes
  • Consider adding blanket traits: RefFromZeros for FromZeros + Immutable, RefFromBytes for FromBytes + Immutable, and RefAsBytes for AsBytes + Immutable
    • We've decided not to do this
  • Improve error message on missing Immutable #1296

Details

We require that FromZeros, FromBytes, and AsBytes types don't permit interior mutability. This is a requirement in order for reference casts (e.g., &T to &[u8] or vice-versa) to be valid, but is not a requirement for value casts. As a result, we don't support some transmutations that are sound (namely, value transmutations on types with interior mutability).

If we added a separate Immutable trait to describe this "has no UnsafeCells" property, then we could decouple it from the other traits and allow them to support types with UnsafeCells. Immutable would then be required as a bound on some functions/methods/macros. This would make our API somewhat more complicated, but would unlock a new class of use cases.

Note that Immutable would be equivalent to the stdlib-internal Freeze trait. There is a proposal to make that trait public, but we shouldn't bank on that happening any time soon.

TryFromBytes and unsized UnsafeCell

In principle, there's no reason we couldn't implement FromZeros, FromBytes, and AsBytes for UnsafeCell<T> even when T: ?Sized. However, this would not play nicely with TryFromBytes (currently in progress in #5), which we want to eventually be a super-trait of FromZeros.

In particular, in order to implement TryFromBytes for a type, we have to implement the is_bit_valid method. For UnsafeCell<T> where T: Sized, we can make a copy of the referent UnsafeCell<T>, transmute it to a MaybeUninit<T>, and then construct a Ptr<T> to that value. This workaround isn't possible for unsized UnsafeCells.

Given that the whole problem with UnsafeCells arises when they're handled by-reference, and given that the only way to operate on unsized values in Rust (at least today) is by reference, supporting unsized UnsafeCells is unlikely to be useful to any of our users. Thus, by implementing TryFromBytes (and, by implication, the other traits) only for sized UnsafeCell, we get most of the benefit.

Motivation

This was originally motivated by chipsalliance/caliptra-sw#634. @korran has provided this minimized example to motivate the feature request:

#[repr(C)]
pub struct Prng {
    // It is critical for safety that every bit-pattern of this struct
    // is valid (no padding, no enums, no references), similar to the requirements for
    // zerocopy::FromBytes.
    s0: Cell<u32>,
    s1: Cell<u32>,
    s2: Cell<u32>,
    s3: Cell<u32>,
}

impl Prng {
    /// # Safety
    ///
    /// Caller must verify that the memory locations between `addr` and
    /// `addr + size_of::<Prng>()` are valid and meet the alignment
    /// requirements of Prng, and are not used for anything else.
    pub unsafe fn from_address(addr: u32) -> &'static Self {
        &*(addr as *const Self)
    }

    pub fn next(&self) -> u32 { ... }
}

If FromBytes didn't forbid interior mutability, the API of from_address would be the same, but it would not require the authors of from_address to justify that it's sound to construct a Prng at the given memory location regardless of its previous contents. Instead, the safety argument can just be "Prng: FromBytes, so it's sound regardless of the memory contents".

[joshlf]

Reopening; we still need to allow TryFromBytes on UnsafeCell and move the NoCell bound to try_from_ref.

[joshlf]

@korran Just wanted to let you know that this is available in 0.8.0-alpha.1. The NoCell trait itself is still #[doc(hidden)], but you can use it. FromZeros, FromBytes, and AsBytes are now implemented for UnsafeCell, and so you can derive those traits on types with UnsafeCell fields. If you're able to try it out, we'd love to hear how it works for you!

[arthurprs]

Will this work also enable some zerocopy functionality for Atomic* types?

[joshlf]

Will this work also enable some zerocopy functionality for Atomic* types?

Good point! I've filed #1009 to track.