Tracking issue for uninitialized constructors for Box, Rc, Arc

[SimonSapin]

Assigning MaybeUninit::<Foo>::uninit() to a local variable is usually free, even when size_of::<Foo>() is large. However, passing it for example to Arc::new causes at least one copy (from the stack to the newly allocated heap memory) even though there is no meaningful data. It is theoretically possible that a Sufficiently Advanced Compiler could optimize this copy away, but this is reportedly unlikely to happen soon in LLVM.

This issue tracks constructors for containers (Box, Rc, Arc) of MaybeUninit<T> or [MaybeUninit<T>] that do not initialized the data, and unsafe conversions to the known-initialized types (without MaybeUninit). The constructors are guaranteed not to make unnecessary copies.

PR #62451 adds:

impl<T> Box<T> { pub fn new_uninit() -> Box<MaybeUninit<T>> {…} }
impl<T> Box<MaybeUninit<T>> { pub unsafe fn assume_init(self) -> Box<T> {…} }
impl<T> Box<[T]> { pub fn new_uninit_slice(len: usize) -> Box<[MaybeUninit<T>]> {…} }
impl<T> Box<[MaybeUninit<T>]> { pub unsafe fn assume_init(self) -> Box<[T]> {…} }

impl<T> Rc<T> { pub fn new_uninit() -> Rc<MaybeUninit<T>> {…} }
impl<T> Rc<MaybeUninit<T>> { pub unsafe fn assume_init(self) -> Rc<T> {…} }
impl<T> Rc<[T]> { pub fn new_uninit_slice(len: usize) -> Rc<[MaybeUninit<T>]> {…} }
impl<T> Rc<[MaybeUninit<T>]> { pub unsafe fn assume_init(self) -> Rc<[T]> {…} }

impl<T> Arc<T> { pub fn new_uninit() -> Arc<MaybeUninit<T>> {…} }
impl<T> Arc<MaybeUninit<T>> { pub unsafe fn assume_init(self) -> Arc<T> {…} }
impl<T> Arc<[T]> { pub fn new_uninit_slice(len: usize) -> Arc<[MaybeUninit<T>]> {…} }
impl<T> Arc<[MaybeUninit<T>]> { pub unsafe fn assume_init(self) -> Arc<[T]> {…} }

PR #66128 adds:

impl<T> Box<T> { pub fn new_zeroed() -> Box<MaybeUninit<T>> {…} }
impl<T> Arc<T> { pub fn new_zeroed() -> Arc<MaybeUninit<T>> {…} }
impl<T> Rc<T> { pub fn new_zeroed() -> Rc<MaybeUninit<T>> {…} }

Unresolved question:

• The constructor that returns for example Box<MaybeUninit<T>> might “belong” more as an associated function of that same type, rather than Box<T>. (And similarly for other constructors.) However this would make a call like Box::<u32>::new_uninit() becomes Box::<MaybeUnint<u32>>::new_uninit() which feels unnecessarily verbose. I suspect that this turbofish will be needed in a lot of cases to appease type inference.

[TimDiekmann]

Just from looking at the current source code:

pub fn new_uninit() -> Box<mem::MaybeUninit<T>> {
    let layout = alloc::Layout::new::<mem::MaybeUninit<T>>();
    let ptr = unsafe {
        Global.alloc(layout)
            .unwrap_or_else(|_| alloc::handle_alloc_error(layout))
    };
    Box(ptr.cast().into())
}

When mem::size_of::<T>() == 0, a zero-sized layout is passed to Global.alloc but alloc mentions:

This function is unsafe because undefined behavior can result if the caller does not ensure that layout has non-zero size.

Did I have overseen something or is this a bug?

[SimonSapin]

Good point! I copied this pattern from Rc and Arc, but there the header (refcounts) cause the allocation to never be zero-size.

Box::new_uninit_slice has a similar bug with size_of() == 0 or len == 0.

[TimDiekmann]

I have noticed this when implementing Box for custom allocators with non-zero layouts. Turns out, that this is indeed useful 🙂

This is my implementation for the sliced version.

[SimonSapin]

#65174

[Kixunil]

What are the requirements for stabilizing the Box API?

I don't think using Rc<MaybeUninit<T>> (and Arc) is currently that useful, as Arc happens to be shared (no mutation). However, there's this pattern where one wants to create a value while it's uniquely owned and then share it later. Doing it with Box would mean copying, so I was thinking about having RcMut and ArcMut that work exactly as Box, but reserve space for reference counters, so that when you call share() on them, they just initialize ref count and convert to Rc/Arc. This is however another topic. Did anyone had this idea before or should I start discussion somewhere?

[SimonSapin]

there's this pattern where one wants to create a value while it's uniquely owned and then share it later

Yes, that’s exactly the idea with having Rc::new_uninit together with Rc::get_mut_unchecked #63292. It would also work with Rc::new_uninit + Rc::get_mut + unwrap, with an unnecessary run-time check (assuming sharing only after this).

[Kixunil]

Great! I still think it'd be better to provide safe abstraction for it, but at least it'd be possible to do in a library.

[rickvanprim]

Similar to new_zeroed() it would be nice to have a new_zeroed_slice().

[josephlr]

Okay, so there is no way to create an array on heap with some length in stable Rust, even in unsafe? -_-"

It is possible to create a zeroed array or slice directly on the heap on Stable Rust and without using unsafe:

const N: usize = 4096*4096;

pub fn make_array() -> Box<[u8; N]> {
    vec![0; N].into_boxed_slice().try_into().unwrap()
}

pub fn make_slice(n: usize) -> Box<[u8]> {
    vec![0; n].into_boxed_slice()
}

This doesn't blow the stack even with opt-level=0.

[SimonSapin]

That’s an optimization (based on specialization of a private trait) in current versions of the standard library, but it only works for some types and is not a documented guarantee.

[QuineDot]

Regarding the self parameter, using some_box_maybe_uninit.assume_init() currently resolves to MaybeUninit::assume_init, though it at least also triggers 48919. Stabilizing the method with the self receiver will be a breaking change as the return types differ.

[thomcc]

and is not a documented guarantee.

This is correct, but it's also true that we'd probably need a very good reason to regress it. If some code's correctness (somehow) relies on this, they should find another method, but for performance reasons I think it's safe enough to rely on.

That said, as you point out it does not work for all types.

[safinaskar]

API looks very strange. Box::new_uninit returns type, which is different from type this method lives in! new_uninit is located in Box<T>, but returns Box<MaybeUninit<T>>. This is absolute violation of principle of least surprise. I just have read #110715 (comment) and I initially was unable to understand the code. I wondered how Box::<Type>::new_uninit() can possibly work. Then I opened docs and then I was able to understand what is going on.

So, please make signature so:

impl<T> Box<MaybeUninit<T>, Global> {
  fn new_uninit() -> Box<MaybeUninit<T>, Global> { ... }
}

The same applies to other functions, for example, try_new_uninit

[bertptrs]

Regarding the self parameter, using some_box_maybe_uninit.assume_init() currently resolves to MaybeUninit::assume_init, though it at least also triggers 48919. Stabilizing the method with the self receiver will be a breaking change as the return types differ.

This could be resolved by not making Box::assume_init a method but rather an associated function like most of the Rc API.