Small comment about implementation:
The system will execute all connected handlers every time, regardless of the result (ishandled) of an individual handler in case multiple handlers are executed. So it does not stop the process when an individual handler returns IsHandled = true.
The reason is that, due to DI registration/resolving of IExceptionHandlers, we don't have control over the order in which the IExceptionHandlers are executed.

Fyi: the IsHandled is documented as:

        /// <returns>a boolean value indicating if the <see cref="Exception"/> was handled or not, so it could be thrown again</returns>

Do you agree on that?