How to safely initialize an array of Vec with MaybeUninit doc example

[leonardo-m]

In this page:
https://doc.rust-lang.org/nightly/std/mem/union.MaybeUninit.html

I suggest to add an equivalent example of "how to safely initialize an array of Vecs" using MaybeUninit instead of uninitialized, if possible:

https://doc.rust-lang.org/nightly/std/mem/fn.uninitialized.html

[cramertj]

What would the example look like? Something like this?

let arr: [MaybeUninit<Vec<u8>>; 5] = [MaybeUninit::uninitialized(); 5];
for v in &mut arr {
    v.set(vec![]);
}
let arr: [Vec<u8>; 5] = mem::transmute(arr);

cc @RalfJung

[leonardo-m]

As arr array length is better to use 50 instead of 5, because Default:default goes up to about 30.

[FenrirWolf]

One gotcha is that the example doesn't even work as-is because MaybeUninit isn't Copy:

https://play.rust-lang.org/?gist=1d2d5fa2a49bbbbceb8a3b113587f87a&version=nightly&mode=debug&edition=2015

If MaybeUninit is intended to be compatible with arrays then maybe that was overlooked?

[mbrubeck]

In 7c37c6d#diff-322c42fd5e8ec8e8f8093261d59a5782R87, @eddyb and @RalfJung mention that MaybeUninit “distributes over products” such as arrays, so you could instead do something like:

let mut arr: MaybeUninit<[Vec<u8>; 5]> = MaybeUninit::uninitialized();
for i in 0..5 {
    ptr::write(&mut arr.get_mut()[i], vec![]);
}
let arr: [Vec<u8>; 5] = arr.into_inner();

...if it is safe to get a pointer or reference to the uninitialized inner value or its fields/elements, as long as this pointer isn't used to read from uninitialized portions of the value. It would only be safe to read from an element after it is initialized, and only safe to read from the whole (e.g. using into_inner) when all the elements are initialized.

Note that the expression arr.get_mut() above creates an &mut T that points to an uninitialized T, so this can only work if we decide that such references are “unsafe but valid” (i.e., not instant UB). Also it may be a bit risky passing this to the safe Index::index function, so a more conservative version might use only raw pointers:

    (arr.as_mut_ptr() as *mut Vec<u8>).offset(i).write(vec![]);

[FenrirWolf]

So would this be a valid example of how to initialize an array of Vecs then?

#![feature(maybe_uninit)]

use std::mem::{self, MaybeUninit};

fn main() {
    let mut arr: [MaybeUninit<Vec<u8>>; 50] = {
        // The compiler complains about `Copy` bounds if we try to directly
        // create a `[MaybeUninit<Vec<u8>>; 50]`, so we use the fact that
        // MaybeUninit<[T; N]> == [MaybeUninit<T>; N]  to work around that
        let arr: MaybeUninit<[Vec<u8>; 50]> = MaybeUninit::uninitialized();
        unsafe { mem::transmute(arr) }
    };

    // Initialize the elements one at a time. The `Iterator` trait isn't
    // implemented for arrays over size 32 (integer generics when!?),
    // so let's use good ol' indexing instead.
    for idx in 0..arr.len() {
        arr[idx].set(vec![]);
    }

    // Now that everything is initialized (and they'd better be fully
    // initialized or things WILL blow up here), we can transmute to
    // a fully valid array of Vecs
    let arr: [Vec<u8>; 50] = unsafe { mem::transmute(arr) };

    // `Debug` is also unimplemented for arrays of this size, so let's
    // index-loop again to show our completed work!
    for idx in 0..arr.len() {
        println!("Index {}: {:?}", idx, arr[idx]);
    }
}

https://play.rust-lang.org/?gist=95140594f74290a294a7cea06d37c597&version=nightly&mode=release&edition=2015

[leonardo-m]

Is it worth adding to the Rust std lib a helper macro like this, to simplify the creation of arrays of noncopy values?

https://crates.io/crates/array-init

[mbrubeck]

So would this be a valid example of how to initialize an array of Vecs then?

My code from above was meant to be an entire solution, without any transmuting necessary. Here's a complete example program:

use std::mem::MaybeUninit;

fn main() {
    const N: usize = 5;

    let arr: [Vec<u8>; N] = unsafe {
        let mut arr = MaybeUninit::uninitialized();
        for i in 0..N {
            (arr.as_mut_ptr() as *mut Vec<u8>).add(i).write(vec![]);
        }
        arr.into_inner()
    };

    for idx in 0..arr.len() {
        println!("Index {}: {:?}", idx, arr[idx]);
    }
}

[FenrirWolf]

Ah nice, that's way more concise.

[RalfJung]

@mbrubeck One extremely subtle point about your code is the Nice to have no transmute, though as *mut Vec<u8> is pretty much the same thing. But yeah, I agree that is correct. Might be nicer to do the arr.as_mut_ptr() as *mut Vec<u8> only once? Also I think arr should have a type ascribed, inference is not entirely trivial here.

use std::mem::MaybeUninit;

fn main() {
    const N: usize = 5;

    let arr: [Vec<u8>; N] = unsafe {
        let mut arr: MaybeUninit<[Vec<u8>; N]> = MaybeUninit::uninitialized();
        let arr_ptr = arr.as_mut_ptr() as *mut Vec<u8>; // pointer to 1st element
        for i in 0..N {
            arr_ptr.add(i).write(vec![]);
        }
        arr.into_inner()
    };

    for idx in 0..arr.len() {
        println!("Index {}: {:?}", idx, arr[idx]);
    }
}

[DevQps]

@mbrubeck One extremely subtle point about your code is the Nice to have no transmute, though as *mut Vec<u8> is pretty much the same thing. But yeah, I agree that is correct. Might be nicer to do the arr.as_mut_ptr() as *mut Vec<u8> only once? Also I think arr should have a type ascribed, inference is not entirely trivial here.

use std::mem::MaybeUninit;

fn main() {
    const N: usize = 5;

    let arr: [Vec<u8>; N] = unsafe {
        let mut arr: MaybeUninit<[Vec<u8>; N]> = MaybeUninit::uninitialized();
        let arr_ptr = arr.as_mut_ptr() as *mut Vec<u8>; // pointer to 1st element
        for i in 0..N {
            arr_ptr.add(i).write(vec![]);
        }
        arr.into_inner()
    };

    for idx in 0..arr.len() {
        println!("Index {}: {:?}", idx, arr[idx]);
    }
}

I think the ascribed type is also very helpful!

When I try to run this on the Playground I get an error however:

error[E0599]: no method named `into_inner` found for type `std::mem::MaybeUninit<[std::vec::Vec<u8>; 5]>` in the current scope

When I try to run this on my own machine with nightly-1.35.0 I get the same error. Am I missing something?

Ultimately I think we should somehow create a good safe abstraction for this. I passed this to the security workgroup to see if there is something we can come up with.

[RalfJung]

The API has changed, the following should work now:

#![feature(maybe_uninit)]
use std::mem::MaybeUninit;

fn main() {
    const N: usize = 5;

    let arr: [Vec<u8>; N] = unsafe {
        let mut arr: MaybeUninit<[Vec<u8>; N]> = MaybeUninit::uninit();
        let arr_ptr = arr.as_mut_ptr() as *mut Vec<u8>; // pointer to 1st element
        for i in 0..N {
            arr_ptr.add(i).write(vec![]);
        }
        arr.assume_init()
    };

    for idx in 0..arr.len() {
        println!("Index {}: {:?}", idx, arr[idx]);
    }
}

But that's basically a hack. Better interaction of MaybeUninit with Box/arrays/slices is definitely on the roadmap, but let's first land the basics.

[eddyb]

Isn't it simpler and safer to turn &mut MaybeUninit<[Vec<u8>; N]> into &mut [MaybeUninit<Vec<u8>>; N] and iterate that normally, instead of iterating with raw pointers?