[Mac M2 pro] Seg faults on multiple libraries

[shirshak55]

I am currently using m2 and have issues with multiple libraries like Tokio / Cargo what-features. It segfaults which means I am unable to use a rust compiler in my m2 pro 13inch 24 GB ram. I initially thought it was a crate issue, but I think it is a problem with rust in general and is related to formatted-related things.

This doesn't seg fault

Cargo.toml

[dependencies]
tokio = { version = "1.20.1", features = ["rt-multi-thread","macros"] }

main.rs

#[tokio::main]
async fn main() {
    println!("Hello, world!");
}

If I add process feature to tokio it segfaults

[dependencies]
tokio = { version = "1.20.1", features = ["rt-multi-thread","macros","process"] }

Running from lldb, I don't know why I don't get symbols but it shows the following output

lldb target/debug/delete_it
(lldb) target create "target/debug/delete_it"
Current executable set to '/Users/quantum/Desktop/projects/internal/delete_it/target/debug/delete_it' (arm64).
(lldb) run
Process 14100 launched: '/Users/quantum/Desktop/projects/internal/delete_it/target/debug/delete_it' (arm64)
Process 14100 stopped
* thread #1, name = 'main', queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x22)
    frame #0: 0x0000000100151a78 delete_it`ltmp0 + 1380136
delete_it`core::fmt::Formatter::debug_struct::h8fffffccd422503e:
->  0x100151a78 <+20>: ldr    x0, [x0, #0x20]
    0x100151a7c <+24>: ldr    x8, [x19, #0x28]
    0x100151a80 <+28>: ldr    x8, [x8, #0x18]
    0x100151a84 <+32>: blr    x8
Target 0: (delete_it) stopped.
(lldb)

I guess there is some issue with core::fmt::Formatter::debug_struct. I don't know the internals of rust so, I can't speak anything else.

Now lets run through Address sanitizer

export RUSTFLAGS=-Zsanitizer=address
cargo run
   Compiling proc-macro2 v1.0.43
   Compiling libc v0.2.132
   Compiling unicode-ident v1.0.3
   Compiling quote v1.0.21
   Compiling syn v1.0.99
   Compiling log v0.4.17
   Compiling cfg-if v1.0.0
   Compiling autocfg v1.1.0
   Compiling pin-project-lite v0.2.9
   Compiling once_cell v1.13.1
   Compiling bytes v1.2.1
   Compiling tokio v1.20.1
   Compiling mio v0.8.4
   Compiling signal-hook-registry v1.4.0
   Compiling num_cpus v1.13.1
   Compiling tokio-macros v1.8.0
==14297==ERROR: Interceptors are not working. This may be because AddressSanitizer is loaded too late (e.g. via dlopen). Please launch the executable with:
DYLD_INSERT_LIBRARIES=/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.asan.dylib
"interceptors not installed" && 0
error: could not compile `tokio`

Caused by:
  process didn't exit successfully: `rustc --crate-name tokio --edition=2018 /Users/quantum/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.20.1/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 --cfg 'feature="bytes"' --cfg 'feature="default"' --cfg 'feature="libc"' --cfg 'feature="macros"' --cfg 'feature="mio"' --cfg 'feature="num_cpus"' --cfg 'feature="once_cell"' --cfg 'feature="process"' --cfg 'feature="rt"' --cfg 'feature="rt-multi-thread"' --cfg 'feature="signal-hook-registry"' --cfg 'feature="tokio-macros"' -C metadata=710b81268173b05b -C extra-filename=-710b81268173b05b --out-dir /Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps -C linker=clang -L dependency=/Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps --extern bytes=/Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps/libbytes-6024d303b245d77d.rmeta --extern libc=/Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps/liblibc-5666b5f002ccdb48.rmeta --extern mio=/Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps/libmio-fc00bad52878b24f.rmeta --extern num_cpus=/Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps/libnum_cpus-8f0a3df89f175e22.rmeta --extern once_cell=/Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps/libonce_cell-5abf2d33226eb8d4.rmeta --extern pin_project_lite=/Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps/libpin_project_lite-b7768fffd0c2e379.rmeta --extern signal_hook_registry=/Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps/libsignal_hook_registry-d655ad958efd2a8c.rmeta --extern tokio_macros=/Users/quantum/Desktop/projects/internal/delete_it/target/debug/deps/libtokio_macros-14d631be2357c11b.dylib --cap-lints allow -Zsanitizer=address` (signal: 6, SIGABRT: process abort signal)

Shows some errors. I tried export DYLD_INSERT_LIBRARIES=/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/librustc-nightly_rt.asan.dylib but as expected it also doesn't work.

Lastly, I tried with cargo run -Zbuild-std --target aarch64-apple-darwin

cargo run -Zbuild-std --target aarch64-apple-darwin
   Compiling compiler_builtins v0.1.79
   Compiling core v0.0.0 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/core)
   Compiling libc v0.2.131
   Compiling cc v1.0.73
   Compiling memchr v2.5.0
   Compiling std v0.0.0 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/std)
   Compiling libc v0.2.132
   Compiling proc-macro2 v1.0.43
   Compiling quote v1.0.21
   Compiling unicode-ident v1.0.3
   Compiling log v0.4.17
   Compiling syn v1.0.99
   Compiling autocfg v1.1.0
   Compiling tokio v1.20.1
   Compiling unwind v0.0.0 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/unwind)
   Compiling tokio-macros v1.8.0
   Compiling rustc-std-workspace-core v1.99.0 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/rustc-std-workspace-core)
   Compiling alloc v0.0.0 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/alloc)
   Compiling cfg-if v0.1.10
   Compiling adler v0.2.3
   Compiling rustc-demangle v0.1.21
   Compiling rustc-std-workspace-alloc v1.99.0 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/rustc-std-workspace-alloc)
   Compiling panic_abort v0.0.0 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/panic_abort)
   Compiling panic_unwind v0.0.0 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/panic_unwind)
   Compiling gimli v0.25.0
   Compiling object v0.26.2
   Compiling std_detect v0.1.5 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/stdarch/crates/std_detect)
   Compiling miniz_oxide v0.4.0
   Compiling hashbrown v0.12.3
   Compiling addr2line v0.16.0
   Compiling proc_macro v0.0.0 (/Users/quantum/.rustup/toolchains/nightly-aarch64-apple-darwin/lib/rustlib/src/rust/library/proc_macro)
   Compiling cfg-if v1.0.0
   Compiling bytes v1.2.1
   Compiling pin-project-lite v0.2.9
   Compiling once_cell v1.13.1
   Compiling signal-hook-registry v1.4.0
   Compiling num_cpus v1.13.1
   Compiling mio v0.8.4
   Compiling delete_it v0.1.0 (/Users/quantum/Desktop/projects/internal/delete_it)
    Finished dev [unoptimized + debuginfo] target(s) in 12.67s
     Running `target/aarch64-apple-darwin/debug/delete_it`
Hello, world!

It works. So, what is the problem? Are the binaries provided by rustup not good?

Thanks.

[BGR360]

For future reference, try to clarify up front whether it's the Rust compiler that's segfaulting / crashing or the application that you compiled.

Could you please provide a link to a GitHub repository that demonstrates the issue so that someone can try to reproduce the problem?

@rustbot label O-macos O-aarch64

[thomcc]

Can you run sysctl -a hw.optional and post the output somewhere? It may be that we're compiling things with the wrong hardware flags for M2.

[shirshak55]

@thomcc

sysctl -a hw.optional
hw.optional.arm.FEAT_FlagM: 1
hw.optional.arm.FEAT_FlagM2: 1
hw.optional.arm.FEAT_FHM: 1
hw.optional.arm.FEAT_DotProd: 1
hw.optional.arm.FEAT_SHA3: 1
hw.optional.arm.FEAT_RDM: 1
hw.optional.arm.FEAT_LSE: 1
hw.optional.arm.FEAT_SHA256: 1
hw.optional.arm.FEAT_SHA512: 1
hw.optional.arm.FEAT_SHA1: 1
hw.optional.arm.FEAT_AES: 1
hw.optional.arm.FEAT_PMULL: 1
hw.optional.arm.FEAT_SPECRES: 0
hw.optional.arm.FEAT_SB: 1
hw.optional.arm.FEAT_FRINTTS: 1
hw.optional.arm.FEAT_LRCPC: 1
hw.optional.arm.FEAT_LRCPC2: 1
hw.optional.arm.FEAT_FCMA: 1
hw.optional.arm.FEAT_JSCVT: 1
hw.optional.arm.FEAT_PAuth: 1
hw.optional.arm.FEAT_PAuth2: 1
hw.optional.arm.FEAT_FPAC: 1
hw.optional.arm.FEAT_DPB: 1
hw.optional.arm.FEAT_DPB2: 1
hw.optional.arm.FEAT_BF16: 1
hw.optional.arm.FEAT_I8MM: 1
hw.optional.arm.FEAT_ECV: 1
hw.optional.arm.FEAT_LSE2: 1
hw.optional.arm.FEAT_CSV2: 1
hw.optional.arm.FEAT_CSV3: 1
hw.optional.arm.FEAT_FP16: 1
hw.optional.arm.FEAT_SSBS: 1
hw.optional.arm.FEAT_BTI: 1
hw.optional.floatingpoint: 1
hw.optional.neon: 1
hw.optional.neon_hpfp: 1
hw.optional.neon_fp16: 1
hw.optional.armv8_1_atomics: 1
hw.optional.armv8_2_fhm: 1
hw.optional.armv8_2_sha512: 1
hw.optional.armv8_2_sha3: 1
hw.optional.armv8_3_compnum: 1
hw.optional.watchpoint: 4
hw.optional.breakpoint: 6
hw.optional.armv8_crc32: 1
hw.optional.armv8_gpi: 1
hw.optional.AdvSIMD: 1
hw.optional.AdvSIMD_HPFPCvt: 1
hw.optional.ucnormal_mem: 1
hw.optional.arm64: 1

[shirshak55]

@BGR360 I have posted the code in my original post. It segfaults on tokio hello world if I use the process feature. I think the issue is probably due to some formatting in the std library.

And, it is not just one library, other libraries like cargo what features, etc. also seg faults. So, I think it is something from Rust Std library.

[thomcc]

It doesn't seem to be related to the hardware features (which are a strict superset of what I have on my M1Max, see below). I'm not quite sure then, given the origin of the error, it does seem to be from the formatting code, but it's probably an issue with how things are linked/loaded, somehow.

In case it matters, the features this has that my machine has disabled are:

hw.optional.arm.FEAT_PAuth2: 1
hw.optional.arm.FEAT_FPAC: 1
hw.optional.arm.FEAT_BF16: 1
hw.optional.arm.FEAT_I8MM: 1
hw.optional.arm.FEAT_BTI: 1

[shirshak55]

Solved!

I was using mold linker. Thanks for mentioning it. Also, sorry for the inconvenience.

[target.aarch64-apple-darwin]
linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=mold"]