Segmentation fault when transmuting simd vector temporary

[ruuda]

This is a pretty subtle interaction of several things that produces code which causes a segmentation fault:

• A struct with #[repr(simd)].
• A struct with a member.
• A method on that struct.
• Transmuting the result of a method call.

#![feature(repr_simd)]

#[repr(simd)] // Without #[repr(simd)], it works fine.
pub struct Mu64(pub u64, pub u64, pub u64, pub u64);

pub struct Trouble {
    state: Mu64, // Without this field, it works fine.
}

impl Trouble {
    pub fn new() -> Trouble {
        Trouble {
            state: Mu64(1, 2, 3, 4),
        }
    }

    fn next(&self) -> Mu64 {
        Mu64(1, 2, 3, 4)
    }

    pub fn invoke_doom(&self) -> [u32; 8] {
        use std::mem::transmute;
        // transmute(Mu64(1, 2, 3, 4)) instead of transmute(self.next()) works
        // fine.
        unsafe { transmute(self.next()) }
    }
}

#[test]
fn this_causes_sigsegv() {
    let mut trouble = Trouble::new();
    println!("{}", trouble.invoke_doom()[0]);
}

Now cargo test crashes with a segmentation fault. (The test program that is, not Cargo.) cargo test --release runs fine. It could be that this is working as intended, and this is what I get for using unsafe code, but it certainly surprises me.

Workaround for now: use transmute_copy instead of transmute.

Version: rustc 1.9.0-nightly (a43eb4e 2016-04-12). It also happened on an older nightly.

[sanmai-NL]

Maybe you can write a PR with a test case so that this gets addressed?

[ruuda]

Tests are usually added after the fix to prevent regressing, but I wouldn’t know where to start in order to fix this.

[dotdash]

LLVM generates a movaps instruction which assumes that the value aligned on a 16 byte boundary. This happens because we're calling the memcpy intrinsic specifying a 32 byte alignment. But, the stack slot gets no alignment set, so it gets the default 8 byte alignment and we end up with a misaligned read.

[dotdash]

D'oh, nevermind that. I was looking at the wrong function in the IR. It's actually just transmute that just casts a [8 x i32]* to <4 x i64>* although the former isn't aligned enough.

[bltavares]

Triage: this fails on nightly.

rustc 1.11.0-nightly (6e00b5556 2016-05-29)
cargo 0.11.0-nightly (3ff108a 2016-05-24)

Running cargo test emit some warnings, and fail with segmentation fault:

rustup run nightly cargo test
   Compiling 32947 v0.1.0 (file:///private/tmp/32947)
src/lib.rs:7:5: 7:16 warning: struct field is never used: `state`, #[warn(dead_code)] on by default
src/lib.rs:7     state: Mu64, // Without this field, it works fine.
                 ^~~~~~~~~~~
src/lib.rs:7:5: 7:16 warning: struct field is never used: `state`, #[warn(dead_code)] on by default
src/lib.rs:7     state: Mu64, // Without this field, it works fine.
                 ^~~~~~~~~~~
src/lib.rs:31:9: 31:20 warning: variable does not need to be mutable, #[warn(unused_mut)] on by default
src/lib.rs:31     let mut trouble = Trouble::new();
                      ^~~~~~~~~~~
     Running target/debug/32947-044b564592bca0e2

running 1 test
error: Process didn't exit successfully: `/private/tmp/32947/target/debug/32947-044b564592bca0e2` (signal: 11, SIGSEGV: invalid memory reference)

cargo test --release compiles fine:

rustup run nightly cargo test --release
   Compiling 32947 v0.1.0 (file:///private/tmp/32947)
src/lib.rs:7:5: 7:16 warning: struct field is never used: `state`, #[warn(dead_code)] on by default
src/lib.rs:7     state: Mu64, // Without this field, it works fine.
                 ^~~~~~~~~~~
src/lib.rs:7:5: 7:16 warning: struct field is never used: `state`, #[warn(dead_code)] on by default
src/lib.rs:7     state: Mu64, // Without this field, it works fine.
                 ^~~~~~~~~~~
src/lib.rs:31:9: 31:20 warning: variable does not need to be mutable, #[warn(unused_mut)] on by default
src/lib.rs:31     let mut trouble = Trouble::new();
                      ^~~~~~~~~~~
     Running target/release/32947-044b564592bca0e2

running 1 test
test this_causes_sigsegv ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured

   Doc-tests 32947

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured

[cyplo]

Still failing on the newest nightly 1.14.0-nightly (f094206 2016-10-20)