[Exception Handling] Fail to catch memory access violation in native while interop with UnrealEngine

[Liangjia0411]

Issue Description

I'm using C# as a scripting language for Unreal Engine with CoreCLR as the virtual machine runtime. I'm leveraging the C# unmanaged function pointer mechanism to call C++ methods.

I've encountered an inconsistency in exception handling:

• When C++ code experiences a memory access violation, the game crashes immediately and UE's exception handling mechanism fails to capture it. No C++ stack trace appears in logs.
• However, when C++ code experiences a divide-by-zero exception, the exception is properly caught by the virtual machine, and the game continues to run without crashing.

Reproduction Steps

1. Set up UE with C# scripting using CoreCLR
2. Create C# code that uses unmanaged function pointers to call into C++ code
3. Implement two test scenarios:
   • A C++ function that causes memory access violation (e.g., dereferencing a null pointer)
   • A C++ function that causes a divide-by-zero exception
4. Call both functions from C#
5. Observe that the memory violation crashes the game without stack trace, while divide-by-zero is handled gracefully

Sample Code

// C# method
public class ExceptionTest
{
  public static unsafe void CrashInEngine_AccessInvalidMemory() 
  {
    Internal_CrashInEngine_AccessInvalidMemory();
  }
  private static unsafe delegate* unmanaged<void> Internal_CrashInEngine_AccessInvalidMemory = default;
    
  public static unsafe void CrashInEngine_DivideByZero() 
  {
    Internal_CrashInEngine_DivideByZero();
  }
  private static unsafe delegate* unmanaged<void> Internal_CrashInEngine_DivideByZero = default;
}

// C++ method
void CrashInEngine_AccessInvalidMemory()
{
    int* ptr = nullptr;
    *ptr = 42; // This causes an access violation
}

void Internal_CrashInEngine_DivideByZero()
{
    int zero = 0;
    int result = 1 / zero; // This causes a divide-by-zero exception
}

Expected Behavior

Ideally, all exceptions in C++ code (including memory access violations) should be handled in one of two ways:

1. Caught by the CoreCLR runtime or UE's exception handling mechanism, allowing the game to continue running without crashing
2. Or, if it must crash, at least output the complete C++ stack trace so developers can diagnose the issue

Current Behavior

Memory access violations in C++ code called via unmanaged function pointers from C# cause the game to immediately crash with no stack trace, while divide-by-zero exceptions are correctly caught.

Environment

• Unreal Engine Version: ue5
• CoreCLR Version: 9.0.5
• Operating System: windows

Additional Information

This inconsistency makes developing and debugging C# code that interfaces with C++ more difficult and risky, as memory access violations cannot be gracefully handled or properly diagnosed.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

[jkotas]

.NET runtime treats divide-by-zero as a regular exception that can be caught and handled. .NET runtime on Windows supports exception handling interop that explains the behavior that you are seeing with divide-by-zero exception thrown from C++ code.

.NET runtime treats access violations as fatal errors. It terminates process immediately when a (dirty) access violation exception reaches .NET runtime code. It is done for security reasons. Dirty access violation should not happen in correct programs and they are symptom of a security bug like a buffer overrun.

We have plans to introduce a low-level callback #101560 to enable custom handling of fatal errors. Once this feature is implemented, you should be able to use this callback to log additional UE details like C++ stacktraces for fatal errors like access violations. (However, it won't allow you to ignore the fatal error and continue running.) cc @VSadov

[AaronRobinsonMSFT]

Ideally, all exceptions in C++ code (including memory access violations) should be handled in one of two ways:

.NET doesn't support C++ exceptions at the interop boundary. It only handles Windows Structured Exceptions and obviously only on Windows. See C++ ABI considerations. MSVC implements C++ exceptions using SEH so it works due to this implementation detail, but has nothing to do with explicit C++ exception support. In fact on non-Windows if an exception is detected, it will fail fast. This is by design.

This inconsistency makes developing and debugging C# code that interfaces with C++ more difficult and risky, as memory access violations cannot be gracefully handled or properly diagnosed.

It isn't possible in .NET Core to handle AccessViolationException because it is a corrupting state exception and we explicitly don't permit it. See here and #69585.

[Liangjia0411]

@jkotas @AaronRobinsonMSFT My requirement is not for .NET to handle these fatal errors. The current issue is that when I call UE's C++ methods from C#, if a Fatal Error occurs in the UE C++ method, UE's exception handling mechanism fails to print the C++ stack trace to the logs.

However, if I call these same methods directly from C++ (rather than from C#), UE's exception handling mechanism successfully prints the C++ stack trace to the logs.

[jkotas]

ExceptionHandling.SetFatalErrorHandler API proposed in #101560 should enable UE to print the C++ stack trace for fatal errors encountered by .NET runtime. It won't allow ignoring these fatal errors and continuing execution of the current process.
 
If you need more flexibility (e.g. be able to ignore access violations in C/C++ code and continue execution of the current process), you need to catch and handle these access violations on .NET/C++ boundary so that they are never encountered by .NET runtime. (I do not think it is a good idea to ignore fatal errors like this.)