Mach ports continued + support aarch64-apple unwinding #2723
Conversation
github-actions
bot
added
cranelift
Subscribe to Label Actioncc @peterhuene
This issue or pull request has been labeled: "cranelift", "cranelift:area:aarch64", "wasmtime:api"
Thus the following users have been cc'd because of the following labels:
To subscribe or unsubscribe from this label, edit the |
bnjbvr
force-pushed
the
mach-ports-continued
branch
from
7bbfb62
to
9813c41
Compare
bnjbvr
force-pushed
the
mach-ports-continued
branch
2 times, most recently
from
2013ea2
to
741dc81
Compare
|
I'm hitting this in CI and I've got no clue how to get around it: |
There was a problem hiding this comment.
👍
I wonder if the clobbering of fp is causing the issues with the system unwinder perhaps? (just a random shot in the dark)
As for the CI issues, it seems like it's probably related to the image that we're running in (whatever is the default on GitHub Actions). If apt-get update -y doesn't work we're unlikely to be the only ones experiencing it, so I'd imagine it'll be fixed in a day or so.
There was a problem hiding this comment.
Thanks @bnjbvr! 👍 for the unwinding-info / return-pointer-auth stuff in particular. I'm a little lost on the stack frame manipulation / stub insertion (a figure or summary might help?) but the general gist seems fine; I'll defer to others for the +1 on that though :-)
bnjbvr
force-pushed
the
mach-ports-continued
branch
from
741dc81
to
1351f35
Compare
|
Thanks for the reviews! This will require a review of fitzgen/mach#64, which I can incorporate in this PR, if it's simpler -- would be nice to have the code be in the right places, though :) |
bnjbvr
force-pushed
the
mach-ports-continued
branch
from
1351f35
to
ef3c1df
Compare
This commit moves macOS to using mach ports instead of signals for handling traps. The motivation for this is listed in bytecodealliance#2456, namely that once mach ports are used in a process that means traditional UNIX signal handlers won't get used. This means that if Wasmtime is integrated with Breakpad, for example, then Wasmtime's trap handler never fires and traps don't work. The `traphandlers` module is refactored as part of this commit to split the platform-specific bits into their own files (it was growing quite a lot for one inline `cfg_if!`). The `unix.rs` and `windows.rs` files remain the same as they were before with a few minor tweaks for some refactored interfaces. The `macos.rs` file is brand new and lifts almost its entire implementation from SpiderMonkey, adapted for Wasmtime though. The main gotcha with mach ports is that a separate thread is what services the exception. Some unsafe magic allows this separate thread to read non-`Send` and temporary state from other threads, but is hoped to be safe in this context. The unfortunate downside is that calling wasm on macOS now involves taking a global lock and modifying a global hash map twice-per-call. I'm not entirely sure how to get out of this cost for now, but hopefully for any embeddings on macOS it's not the end of the world. Closes bytecodealliance#2456
Aarch64 post ARMv8.3 has a feature called pointer authentication, designed to fight ROP/JOP attacks: some pointers may be signed using new instructions, adding payloads to the high (previously unused) bits of the pointers. More on this here: https://lwn.net/Articles/718888/ Unwinders on aarch64 need to know if some pointers contained on the call frame contain an authentication code or not, to be able to properly authenticate them or use them directly. Since native code may have enabled it by default (as is the case on the Mac M1), and the default is that this configuration value is inherited, we need to explicitly disable it, for the only kind of supported pointers (return addresses). To do so, we set the value of a non-existing dwarf pseudo register (34) to 0, as documented in https://github.com/ARM-software/abi-aa/blob/master/aadwarf64/aadwarf64.rst#note-8. This is done at the function granularity, in the spirit of Cranelift compilation model. Alternatively, a single directive could be generated in the CIE, generating less information per module.
bnjbvr
force-pushed
the
mach-ports-continued
branch
from
ef3c1df
to
8a4e93c
Compare
|
Now not depending on the Mach PR anymore; should make the verify CI task happy! |
bnjbvr
force-pushed
the
mach-ports-continued
branch
from
8a4e93c
to
8fa9859
Compare
|
To confirm, am I correct in understanding that this PR now serves two purposes and is reading for landing:
Does that sound right? If so seems reasonable to me to land! |
|
Yes, this is entirely correct! MacOS CI passing suggests that the backtraces are still properly filled (i.e. contain all the frames) on darwin x86_64. One note for readers: it's possible to get all the frames in backtrace by using a custom build of libunwind:
This is sufficient to get all the frames appearing on backtraces on mac aarch64. |
|
Ok! Let's go ahead and land this to get to a more working state along those two points, and we can figure out later how to best handle the unwinding intricacies on macOS AArch64 |
This is #2632 rebased and continued. Many thanks to @alexcrichton for providing a lot of help with the CFI directives in the custom assembly stub; this really helped simplify the design of frame unwinding.
This is sufficient to make mach ports work on Mac M1, with the slight exception that the system's libunwind on this machine doesn't correctly interpret the CFI directives generated by the assembly stub. As a consequence, the stack trace information for wasm call frames is missing. Retrieving the source of another libunwind implementation, in Apple's LLVM fork or from the opensource.apple.com website, building it and linking it against wasmtime is sufficient to get it working as expected. In addition to this, some experiments showed that the system libunwind's source code doesn't match what's on Apple's repository: this was inferred from looking at an abort error message's line number, and seeing it didn't match the line number for the same message in local sources. As a matter of fact, this is impossible to debug as is. Yet it seemed better to have something that worked, even incompletely. In the future, either we can wait for a new functioning version of OSX libunwind, or we could consider shipping our own, as a small C/C++ dependency.
This is also blocked on fitzgen/mach#64 to be landed and released first.
There's a bit more work to do to fully support Mac M1: I'm seeing WASI failures when trying to run the full test suite, to be handled in a second time.