macOS: exceptions aren't caught when linking C++ to arm64 compiled code

[ng-marcus]

I'm compiling OCaml bindings for a C++ library (https://github.com/grain-lang/binaryen.ml). We previously hit a problem where the macOS compiled version wasn't capturing exceptions from the C++ library and was causing SIGABRT instead, as someone else demonstrates well here (https://github.com/vaivaswatha/OCamlCppExn) which was resolved by using keep_dwarf_unwind at the linking stage (issue #9026)

I'm now building an M1/arm64 native version, which compiles and builds fine but tests are failing because an exception thrown in the library is again aborting the program rather than being caught in the OCaml program. Compiling and running https://github.com/vaivaswatha/OCamlCppExn but with keep_dwarf_unwind defined fails with the same problem as before:

libc++abi: terminating with uncaught exception of type std::runtime_error: This is not caught on macOS, why?

when it should hopefully have worked.

It may also be referenced here: https://git.brettgilio.com/ocaml/commit/482b7feb3743e140fd79c2b1bf929f2b738d42eb.html

Has anyone had success in getting arm64 compiled code to capture c++ exceptions?

This is on 4.12.0

[kit-ty-kate]

I've tried the OCamlCppExn example and i can reproduce the issue on macOS 12.0beta (arm64)

I've also tried a similar pure C++ with C main and was able to get the expected behaviour (no uncaught exception). So I can confirm OCaml might be doing something weird here.

[kit-ty-kate]

I was also able to reproduce on a macOS/x86_64 (10.15)

[xavierleroy]

https://github.com/vaivaswatha/OCamlCppExn works fine on my Mac M1 with macOS 11.3 and OCaml 4.12.0 installed from OPAM. A simpler version of this test (not using Ctypes) works fine too.

An alternative to using keep_dwarf_unwind at the linking stage is to use ocamlopt -cc 'clang++' ... for the linking phase. Then, the final link will be done by a C++ compiler, with appropriate linker options.

[kit-ty-kate]

https://github.com/vaivaswatha/OCamlCppExn works fine on my Mac M1 with macOS 11.3 and OCaml 4.12.0 installed from OPAM.

just in case you got caught by it too, the repository is in the "fixed state". You need to remove the -fkeep_dwarf_unwind from the cpp/config/discover.ml

[xavierleroy]

the repository is in the "fixed state". You need to remove the -fkeep_dwarf_unwind from the cpp/config/discover.ml

I don't understand. We have already established that -fkeep_dwarf_unwind is needed to make the test work on x86-64, and I'm saying it also works on ARM64. Why should I remove -fkeep_dwarf_unwind ?

[phated]

@xavierleroy You are correct, and it seems that the the original at that link works, but isn't quite the issue we've encountered. It seems that the issue occurs on Mac M1 when using dune's (foreign_archives) stanza. I've reduced the problem down to a fork of that example at https://github.com/phated/OCamlCppExn

I've also done some more debugging and it seems that the Mac M1 will only work if the C code inside the archive is generated with the -g flag - which I've done at this branch: https://github.com/phated/OCamlCppExn/tree/add-dash-g

This requirement on having a debug build for the foreign archive didn't exist before we tried compiling on the Mac M1.

[mshinwell]

As far as I can tell, this isn't a bug in the OCaml compiler. The LLVM toolchain on macOS/AArch64 seems to be reticent about producing the necessary DWARF sections for C++ exception unwinding. (We've seen problems before with some unwinding tests in the compiler distribution itself, which are currently disabled pending more information from Apple.)

If adding the -g option for the C code works, that seems like a good thing to do. Alternatively I imagine it will probably work if you put the C++ code outside of extern "C" sections and build the C++ code with a C++ compiler (then using @xavierleroy 's -cc invocation above to link).

[xavierleroy]

I've reduced the problem down to a fork of that example at https://github.com/phated/OCamlCppExn

I can reproduce the problem. But if you link with ocamlopt -cc clang++,everything works fine, whether the C and C++ files are compiled with -g or not.

May I suggest that you try -cc clang++ with your library and application? The good thing is that you don't have to guess the name of the C++ library, nor obscure flags like -keep_dwarf_unwind, the clang++ driver should be doing the right thing.

[phated]

@xavierleroy Thanks for the suggestion! I'm trying to determine where that flag is passed in the dune build system to give it a try right now.

[phated]

Finally figured it out! We needed the -cc clang++ flags as in the (ocamlopt_flags) stanza on the (executable) rule in dune (instead of on the library being built). I made those changes in this commit and the error is caught successfully: phated/OCamlCppExn@260a0c1

🙏 Super appreciate you looking into this and providing that solution!

[marcusroberts]

Thanks everyone! This is working now.

[gasche]

It may be possible/useful to add these guidelines to the Dune documentation somewhere; one way to move towards this would be to open an issue on the Dune bugtracker to explain what you were doing before, what went wrong, and how to fix the Dune file. Users could find it out more easily than here, I think.

[xavierleroy]

Actually, this could be mentioned in the reference manual, when documenting the -cc option to ocamlc and ocamlopt. Something like "when linking object files produced by a C++ compiler such as g++ or clang++, it is recommended to use -cc g++ or -cc clang++".

[mt-caret]

Hello, I've observed similar issues when writing OCaml bindings to some Rust code, specifically when catching Rust panics via std::panic::catch_unwind, and I'm not sure this issue is fully resolved.

Prior to learning about this -cc clang++ workaround all attempts to catch panics resulted in aborts, but after introducing the -cc clang++ option into my library I still see aborts when RUST_BACKTRACE=1 or RUST_BACKTRACE=full.

A minimal reproducible example is here, along with associated backtrace output: https://github.com/mt-caret/ocaml-rust-on-apple-silicon-demo

Mentioning @tizoc of ocaml-interop, @zshipko of ocaml-rs, and @LaurentMazare since this issue could be relevant to their (and their users') libraries.

[gasche]

@mt-caret there seems to be related issues in the Rust+OSX ecosystem, see for example rust-lang/rust#113783 (not exactly the same as it does not seem affected by backtrace being required or not) . One workaround used by some projects affected by that issue is to ask Apple for the "classic linker" using -Wl -ld_classic ( https://github.com/GlareDB/glaredb/pull/1996/files ). Could passing -ccopt -Wl,-ld_classic as additional OCaml flags help with your issue?