Support C++ exceptions for rtld-c18n using otypes.

[dstolfa]

Opening this to mainly look for feedback, this should not be merged, especially not into CheriBSD directly. The reason the PR is here is so that the code can easily be built for testing. This approach is similar to the approach taken by @dpgao and the differences are summarized below.

This PR implements an initial version of DWARF unwinding for Morello with the c18n runtime linker. It implementation uses otypes, and therefore might not be compatible with RISC-V.

This implementation is under #ifdef _LIBUNWIND_SANDBOX_OTYPES, as we will likely want to explore different designs for this.

Since libunwind's unw_step() is a part of the public API, adding a new return code when a compartment boundary is encountered is not feasible as it would break third party consumers. Furthermore, I have tried to be careful about introducing any compile-time ABI changes outside of ones under __CHERI_PURE_CAPABILITY__ so that we don't need a secondary libgcc_s for c18n in CheriBSD. This all seems to work and doesn't seem to break any third party software in my testing, but there are probably still some edge cases to catch.

Because libunwind has to be thread-safe and its main approach to doing that is using a context, the executive stack pointer was placed into the context. However, the pointer never leaves libunwind without being sealed. This is also true for all the frame and callee-saved registers that are not sealed using otypes or sentries. Unfortunately this still means that we might leak sentries in the context, but I do not have a good way of addressing that right now.

We still call into the rtld to fetch and restore the executive stack pointer, but I don't see a way around this using this kind of design. Furthermore, the CHERI-specific defines are mostly there as a hack and should probably not live in libunwind.

[dpgao]

This function does not seem to be used anywhere.

[dpgao]

Should getRegister also support this new regNum?

[dpgao]

It seems that this is the only place where sealer_unwbuf is used in RTLD. Would it be a good idea to move sealer_unwbuf to libunwind and let it perform this cseal?

We can use linkage policy to ensure that only libunwind can call _rtld_unw_getcontext.

[dpgao]

This function does not seem to be used anywhere.

[dstolfa]

After some discussion with @dpgao, a few issues were identified:

• Current _rtld_unw_setcontext doesn't actually unwind the restricted stack pointers correctly. This needs to be done in the RTLD itself. This requires a modification in how rtld-c18n treats the next trusted frame in the return trampoline (work in progress by @dpgao). Alternatively, _rtld_unw_setcontext could be renamed to _rtld_unw_resume and implemented as a tail call in libunwind, however that raises some questions about the ABI of that interface and is probably better avoided in favor of the first solution.
• In order to make the interfaces consistent, all the valid capabilities should always be sealed when leaving libunwind, and unsealed before we restore the register context and resume in the handler.
• I've attempted to move the sealer into libunwind itself, however it fails to build because libunwind is not built with -fPIC and I can't access the sealer symbol from the assembly files. However, with the incoming changes, RTLD itself will expect to get a sealed executive stack pointer and will have to unseal it in order to keep the invariant of capabilities not leaving these interfaces unsealed.

[dstolfa]

Updated the review:

• Correctly unwind the trusted frames in rtld
• Make the use of otypes in libunwind itself optional under _LIBUNWIND_SANDBOX_HARDENED
• Clean the code up and separate out the Morello-specific bits better.
• Support the benchmark ABI.

Presently, there are some questions about what the name of the defines should really be, but this seems okay for now. There are also failing libunwind tests on Morello, but perhaps that is a separate PR as they were failing to begin with. This is still not properly tested outside of Morello and the update is mainly to get some high-level discussion going.

[davidchisnall]

Is there a design doc for this? A few things are not clear to me:

• It looks as the unwinder runs on the stack of the faulting compartment (and then on the stacks that you unwind through). What is the expected information leakage here?
• What is the threat model with respect to corrupting the unwinder's state? This is just a heap allocation, and doesn't appear to be sealed, and is reachable from the stack during unwind.
• The personality functions in the libraries will provide cleanup blocks to run. These will have data-dependent control flow and can corrupt anything reachable from the stack and have their control-flow influenced by anything that an attacker can corrupt.
• I don't see any libcxxrt changes. I would expect __cxa_throw and friends to be modified to seal everything except the thrown object so that the C++ runtime's state can be isolated from the code being unwound.
• In which context does the destructor run?

Presumably the exception object and its type information are trusted. This causes some problems because the throwing compartment is the one that initialises the object and provides the pointer to the destructor. If the object type is provided by another library then it's possible for the throwing library to construct an arbitrary object that the destructor will run on. This is basically a COOP gadget that will run in another compartment's context with the program in a state that cannot be reached by normal control flow. The security implications of this are not obvious to me. It may be no worse than calling destructors on local objects, it may be a compartment escape.

[dstolfa]

Thanks for the feedback!

Is there a design doc for this? A few things are not clear to me:

Not yet, we are still going through the design and trying to figure out what the right way to approach the problem is. I took this design as a starting point, effectively opting to treat libunwind as a sort of a TCB, but it is by no means something I am convinced is "the right way" to do things at the moment. I'm waiting on @dpgao to land a change to make unwinding easier at which point the whole hash table bit should go away since we will no longer need to maintain any state there. After that, I'll write down the design document and go through the design in more scrutiny.

It looks as the unwinder runs on the stack of the faulting compartment (and then on the stacks that you unwind through). What is the expected information leakage here?

My information leakage concerns with this design have to do with the register context, notably callee-saved registers, restricted stack pointers from other compartments and even the executive stack pointer could easily leak out of the libunwind boundary and be accessible on the caller stack. The "hardening" happens by sealing anything that is a pointer and is not sealed already (and thus is unable to seal sentries with an otype), but I am fairly certain there are other security concerns here that we'll have to address.

What is the threat model with respect to corrupting the unwinder's state? This is just a heap allocation, and doesn't appear to be sealed, and is reachable from the stack during unwind.

Which one in particular? If you mean the simple hash map allocated as a part of CompartmentInfo, I believe that one should be sealed here:

#ifdef _LIBUNWIND_SANDBOX_HARDENED
    capability_t sealer = addressSpace.getUnwindSealer();
    if (sealer != addressSpace.to_capability_t(-1))
      stackTable = __builtin_cheri_seal(stackTable, sealer);
#endif

However when #2061 lands, this entire hash table should no longer be necessary as we won't need to read and write to the bottom of the restricted stack anymore. Perhaps you are talking about a different heap allocation and I'm misunderstanding which one?

The personality functions in the libraries will provide cleanup blocks to run. These will have data-dependent control flow and can corrupt anything reachable from the stack and have their control-flow influenced by anything that an attacker can corrupt.

This is very likely to be a problem and needs to be thought about further than I have at this stage. The attacker should not be able to corrupt any of the sealed registers themselves, but I'm not quite sure how to protect the context itself seeing as it lives on the caller stack.

I don't see any libcxxrt changes. I would expect __cxa_throw and friends to be modified to seal everything except the thrown object so that the C++ runtime's state can be isolated from the code being unwound.

Agreed. For some background, we thought that libunwind support was necessary to get the desktop stack running properly for a demo that is due next week (and we still aren't sure that it isn't) so the goal was simply to get it working as fast as possible in an experimental state. I took the view of isolating the changes to libunwind as much as possible, but this design is probably a far cry from the thing we want to eventually have.

In which context does the destructor run?

I'm not sure which destructor you mean here, but if it's the exception destructor I think (@dpgao can confirm this) that it runs in the same compartment that libcxxrt is. I believe that Dapeng was looking at handling function pointers in the near-ish future, at which point it should jump through the c18n rtld and be called in a different one (probably with supporting code on the libcxxrt end...?).

Presumably the exception object and its type information are trusted. This causes some problems because the throwing compartment is the one that initialises the object and provides the pointer to the destructor. If the object type is provided by another library then it's possible for the throwing library to construct an arbitrary object that the destructor will run on. This is basically a COOP gadget that will run in another compartment's context with the program in a state that cannot be reached by normal control flow. The security implications of this are not obvious to me. It may be no worse than calling destructors on local objects, it may be a compartment escape.

Agreed, and I would love to have a more detailed discussion about pretty much all of the above if you are available some time (ideally with at least @dpgao also being present as he knows the rtld bits in detail). Even though this "works" it's just a starting point and likely requires more thought to claim any kind of security. If this does land for the demo, it would be have to be documented as experimental with a disclaimer on security. FWIW, one design point I'm interested in is handling exceptions by having each compartment unwind itself up to the boundary and then re-raising the exception in the next compartment. It's unclear to me at this point how it would integrate with libunwind's public APIs, which is part of the reason why I went for this sort of design as a starting point.

[dstolfa]

Update the code to remove the hash table which is no longer necessary.