FreeBSD arm64/aarch64 support

[nunotexbsd]

Feature

Add FreeBSD support on arm64/aarch64

Implementation

Add to https://github.com/bytecodealliance/wasmtime/blob/main/crates/runtime/src/traphandlers/unix.rs

 } else if #[cfg(all(target_os = "freebsd", target_arch = "aarch64"))] {
                    pc = context.uc_mcontext.mc_gpregs.gp_elr as usize;
                    sp = context.uc_mcontext.mc_gpregs.gp_sp as usize;

I take this code from https://github.com/wasmerio/wasmer/blob/dcfdea76999a/lib/vm/src/trap/traphandlers.rs#L302-L304 but don't know if some change needs to be made.

0.2.5 build log:
https://pkg-status.freebsd.org/ampere3/data/131arm64-default/b02e58aec16f/logs/lapce-0.2.5_1.log

[alexcrichton]

This is not currently a supported platform but if you'd like to send a PR to update the code it would be appreciated!

[nunotexbsd]

@alexcrichton

Nice I will do that. Soon I will have arm64 hardware to further tests instead on relying on freebsd build servers.

Thanks

[afonso360]

We probably also need to update the wasmtime-jit-icache-coherence crate.

I looked into this a little bit in #5323 but it looks like FreeBSD doesn't yet support the membarrier interface. But perhaps there is something else that we can use?

I had a look at what other JIT's are doing:

• V8 emits its own icache maintenance instructions.
• Spidermonkey uses membarriers
• Mono also emits their own icache maintenance instructions

Maybe FreeBSD has some other syscall that we can use? I'm not too familiar with it.

CC: #3310

[sec]

Just to add something, for latest relase (6.0.0), small change is need to make it compile:

} else if #[cfg(all(target_os = "freebsd", target_arch = "aarch64"))] {
    let cx = &*(cx as *const libc::mcontext_t);
    (
        cx.mc_gpregs.gp_elr as *const u8,
        cx.mc_gpregs.gp_x[29] as usize,
    )

around https://github.com/bytecodealliance/wasmtime/blob/main/crates/runtime/src/traphandlers/unix.rs#L220

This make it compile, but running tests is not possible becuase of denoland/rusty_v8#1094

[nunotexbsd]

@sec
Thanks for testing it. I still waiting for arm64 hardware so I can't test it.
I will apply your patch and try to find someone that build and run test for me.

https://github.com/lapce/lapce , FreeBSD port, will be tested on arm64.aarch64

[afonso360]

👋 Hey, I've tested the above patch in FreeBSD 14.0-CURRENT

freebsd@generic:~ % uname -a
FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n259967-11b5b9e8a520: Sat Jan  7 18:30:37 UTC 2023     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64

cargo build and cargo run work.

freebsd@generic:~/git/wasmtime % cargo run ./examples/gcd.wat --invoke gcd 8 8
    Finished dev [unoptimized + debuginfo] target(s) in 2.21s
     Running `target/debug/wasmtime ./examples/gcd.wat --invoke gcd 8 8`
warning: using `--invoke` with a function that takes arguments is experimental and may break in the future
warning: using `--invoke` with a function that returns values is experimental and may break in the future
8

I was able to run cargo test, but the tests fail with illegal instruction

     Running unittests src/lib.rs (target/debug/deps/wasmtime_cli-f9e37a0a7433af84)

running 2 tests
test commands::compile::test::test_aarch64_flags_compile ... ok
test commands::compile::test::test_successful_compile ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.57s

     Running unittests src/bin/wasmtime.rs (target/debug/deps/wasmtime-27a7f646c50db093)

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running tests/all/main.rs (target/debug/deps/all-619db9ec6a9e4647)

running 819 tests
test async_functions::async_with_pooling_stacks ... ok
test async_functions::cancel_during_run ... ok
test async_functions::async_host_func_with_pooling_stacks ... ok
test async_functions::fuel_eventually_finishes ... ok
error: test failed, to rerun pass `--test all`

Caused by:
  process didn't exit successfully: `/home/freebsd/git/wasmtime/target/debug/deps/all-619db9ec6a9e4647` (signal: 4, SIGILL: illegal instruction)

I would guess this is due to the missing icache code. The symptoms are fairly similar to what was happening in #4997 for Windows AArch64.

---

Edit: The cranelift test suite passes, which I wouldn't have expected 🤔 Maybe something else is going on here.

[nunotexbsd]

@afonso360
Thanks for do further tests.
I'm working on https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=268929 and if it succeeds, then I can start testing other ports that depends on this crate.

[nunotexbsd]

Just to add something, for latest relase (6.0.0), small change is need to make it compile:

} else if #[cfg(all(target_os = "freebsd", target_arch = "aarch64"))] {
    let cx = &*(cx as *const libc::mcontext_t);
    (
        cx.mc_gpregs.gp_elr as *const u8,
        cx.mc_gpregs.gp_x[29] as usize,
    )

around https://github.com/bytecodealliance/wasmtime/blob/main/crates/runtime/src/traphandlers/unix.rs#L220

This make it compile, but running tests is not possible becuase of denoland/rusty_v8#1094

Hello @sec,

I have results that it builds on aarch64 and I committed your patch at https://cgit.freebsd.org/ports/commit/?id=75a1d429216aa977d1bf6b1f4a9c5eb4eb474ac8

Should a upstream change be made with this patch?

( Build fails on armv7 FreeBSD 13.1:

error[E0308]: mismatched types                                   
   --> /wrkdirs/usr/ports/editors/lapce/work/lapce-0.2.5/cargo-crates/rustix-0.35.10/src/backend/libc/net/syscalls.rs:565:21
    |                                                            
565 |                     tv_sec,                       
    |                     ^^^^^^ expected `i64`, found `i32`
    |                                                                                                                              help: you can convert an `i32` to an `i64`      
    |                                                            
565 |                     tv_sec: tv_sec.into(),    
    |                     +++++++       +++++++     
                                                                 
[num-traits 0.2.15] cargo:rustc-cfg=has_leading_trailing_ones
[num-traits 0.2.15] cargo:rustc-cfg=has_int_assignop_ref
For more information about this error, try `rustc --explain E0308`.
error: could not compile `rustix` due to previous error

This is due to time_t not being a long on armv7. )

[sunfishcode]

I have a fix for that armv7 FreeBSD bug here; I'll do a point release of rustix and update Wasmtime.

[afonso360]

Looked into this a bit more, this branch also does the icache clearing. However I'm still getting the same error.

I'll try to track this down when I get a chance.