Return value packing for Option<T> and Result<T,E> on WebAssembly

[bvibber]

I'm looking at how Result<T,E> and Option<T> return values get handled on the WebAssembly target, using integral types as the T and small fieldless enums as the E type. It seems hard to predict when Rust will use stack allocation for the return value, and some optimizations that happen on native platforms like x86-64 Linux aren't happening where I expect them.

(I'm aware that internal Rust ABIs are unstable, but it'd nice to be able to reason about performance on critical paths, so I'm checking this out in detail before building a pipeline that depends on Results or Options in a tight interpreter loop.)

Tested some example code like:

// A small enum to annotate error conditions
#[derive(Copy, Clone, Debug)]
pub enum MyErr {
    PageFault,
    IllegalInstruction,
    Halt
}

#[inline(never)]
pub fn result_u16(i: u64) -> Result<u16, MyErr> {
    if i < 50 {
        Ok(i as u16)
    } else {
        Err(MyErr::IllegalInstruction)
    }
}

#[inline(never)]
pub fn option_u16(i: u64) -> Option<u16> {
    if i < 50 {
        Some(i as u16)
    } else {
        None
    }
}

with similar variants for u8, u16, u32, and u64 returns. The MyErr enum is small enough to fit in less than a byte itself for Result returns.

On a Linux or macOS x86-64 build (with the benefit of having native support for two return values in registers) I get:

• option_u8 returns two words in AL (discriminant) and EDX (payload)
• option_u16 returns two words in AL (discriminant) and EDX (payload)
• option_u32 returns two words in AL (discriminant) and EDX (payload)
• option_u64 returns two words in AL (discriminant) and RDX (payload)
• result_u8 returns two words in AL(discriminant) and EDX (payload)
• result_u16 returns a packed 32-bit word in EAX, with Ok payload in the top 16 bits
• result_u32 returns a packed 64-bit word in RAX, with Ok payload in the top 32 bits
• result_u64 returns on stack

On a WebAssembly build, I get:

• option_u8 returns on stack
• option_u16 returns on stack
• option_u32 returns on stack
• option_u64 returns on stack
• result_u8 returns on stack
• result_u16 returns a packed 32-bit word, with Ok payload in the top 16 bits
• result_u32 returns on stack
• result_u64 returns on stack

Couple big things surprised me here. First, while on native x86-64 Option<T> is well optimized using two registers, Result<T,E> doesn't consistently get the same treatment. Perhaps because conceptually the enum payload is a separate data byte and it complicates things?

Second, none of the Option types get optimized into integral values on WebAssembly, where there's no multiple return value handling yet but return types up to 64 bits are available.

Third, while Result<u16,MyErr> gets packed into a 32-bit word on WebAssembly, the u8 version is not though it would fit handily in a 32-bit word too, nor is the u32 version packed into a 64-bit word like on native. (There might be something weird with my u8 version, like it's optimizing out the MyErr or something.)

As for Result<u64,MyErr>, that's using stack as expected.

In summary, on WebAssembly:

• It's hard to predict when enum structures will get packed into a register versus transferred through stack memory, making it harder to reason about performance.
• Option<T> is not getting "register"-packed at any size, causing transfer of data through memory.
• Result<u8, MyErr> and Result<u32, MyErr> should be packable into 32-bit and 64-bit return values, but are not. It's unclear why.
• Result<u64, MyErr> as stack transfer remains optimal until some future day when WebAssembly gains multiple return values.

Thanks for any advice or explanations!