Unhandled exception inside Interop+OpenSsl.AlpnServerSelectCallback causing the process crash

[VasiliyNovikov]

Description

Unhandled exception which causes process to crash

System.InvalidOperationException: Handle is not initialized.
Stack trace:
  in System.Void System.ThrowHelper.ThrowInvalidOperationException_HandleIsNotInitialized()
  in System.Runtime.InteropServices.GCHandle System.Runtime.InteropServices.GCHandle.FromIntPtr(System.IntPtr)
  in System.Int32 Interop+OpenSsl.AlpnServerSelectCallback(System.IntPtr, System.Byte**, System.Byte*, System.Byte*, System.UInt32, System.IntPtr)

Reproduction Steps

It reproduces sometimes (but extremely rarely) in ASP.NET Core app running on Ubuntu 18:04 Server after we migrated from .NET 6 to .NET 7
It uses Kestrel + SSL certificate for HTTPS

Expected behavior

Actual behavior

Process crashes

Regression?

Maybe .NET 7.0 regression

Known Workarounds

No response

Configuration

uname -a
Linux tll-netem-1 5.4.0-87-lowlatency #98~18.04.1-Ubuntu SMP PREEMPT Wed Sep 22 13:43:18 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

dotnet --info
.NET SDK:
Version: 7.0.102
Commit: 4bbdd14480

Runtime Environment:
OS Name: ubuntu
OS Version: 18.04
OS Platform: Linux
RID: ubuntu.18.04-x64
Base Path: /usr/share/dotnet/sdk/7.0.102/

Host:
Version: 7.0.2
Architecture: x64
Commit: d037e07

.NET SDKs installed:
5.0.408 [/usr/share/dotnet/sdk]
6.0.405 [/usr/share/dotnet/sdk]
7.0.102 [/usr/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 5.0.17 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.13 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.2 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 5.0.17 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.13 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.2 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
None

Environment variables:
Not set

global.json file:
Not found

Other information

No response

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @dotnet/ncl, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Unhandled exception which causes process to crash

System.InvalidOperationException: Handle is not initialized.
Stack trace:
in System.Void System.ThrowHelper.ThrowInvalidOperationException_HandleIsNotInitialized()
in System.Runtime.InteropServices.GCHandle System.Runtime.InteropServices.GCHandle.FromIntPtr(System.IntPtr)
in System.Int32 Interop+OpenSsl.AlpnServerSelectCallback(System.IntPtr, System.Byte**, System.Byte*, System.Byte*, System.UInt32, System.IntPtr)

Reproduction Steps

It reproduces sometimes (but extremely rarely) in ASP.NET Core app running on Ubuntu 18:04 Server after we migrated from .NET 6 to .NET 7
It uses Kestrel + SSL certificate for HTTPS

Expected behavior

Actual behavior

Process crashes

Regression?

Maybe .NET 7.0 regression

Known Workarounds

No response

Configuration

uname -a
Linux tll-netem-1 5.4.0-87-lowlatency #98~18.04.1-Ubuntu SMP PREEMPT Wed Sep 22 13:43:18 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

dotnet --info
.NET SDK:
Version: 7.0.102
Commit: 4bbdd14480

Runtime Environment:
OS Name: ubuntu
OS Version: 18.04
OS Platform: Linux
RID: ubuntu.18.04-x64
Base Path: /usr/share/dotnet/sdk/7.0.102/

Host:
Version: 7.0.2
Architecture: x64
Commit: d037e07

.NET SDKs installed:
5.0.408 [/usr/share/dotnet/sdk]
6.0.405 [/usr/share/dotnet/sdk]
7.0.102 [/usr/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 5.0.17 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.13 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 7.0.2 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 5.0.17 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.13 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.2 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
None

Environment variables:
Not set

global.json file:
Not found

Other information

No response

Author:	VasiliyNovikov
Assignees:	-
Labels:	area-System.Net.Security, untriaged
Milestone:	-

This issue has been marked needs-author-action and may be missing some important information.

[rzikm]

we should look at it in 8.0.

@VasiliyNovikov crash dumps would be very useful for debugging this. Do you have some available?

[VasiliyNovikov]

Unfortunately no, it was reproduced in prod environment only twice. I'll try to configure dump collection so on next repro I could get one

[janvorli]

@VasiliyNovikov is the issue that the unhandled exception occurs at all or that it aborts the process while you'd expect it to be handled somewhere up the call chain?

[VasiliyNovikov]

It aborts the process. It seems to happen outside of the request context

[janvorli]

The thing is that unhandled exceptions abort the process by their nature. So that's why I was asking if you have expected that exception to get handled somewhere up the call chain.
The important thing to mention is that on Unix, exceptions cannot be propagated from callbacks that are called by native code.