Description
Description
Our Application crashed with a internal NetCore Runtime error
It began after migrate from Net Framework 4.8 to NET 7.
Crash reproduce on NET 7.0.14 / 7.0.15 / 8.0.1
Faulting application name: server.exe, version: 1.0.10.1, time stamp: 0x65410000
Faulting module name: coreclr.dll, version: 8.0.123.58001, time stamp: 0x65684720
Exception code: 0xc0000005
Fault offset: 0x00000000000e87a6
Faulting process id: 0x48f8
Faulting application start time: 0x01da488073edf4b2
Faulting application path: D:\1.1.Server\server.exe
Faulting module path: C:\Program Files\dotnet\shared\Microsoft.NETCore.App\8.0.1\coreclr.dll
Report Id: be1a8f26-efe5-4a91-9f2f-242672b1661e
Faulting package full name:
Faulting package-relative application ID:
Description: The process was terminated due to an internal error in the .NET Runtime at IP 0x00007FFA27E587A6 (0x00007FFA27D70000) with exit code 0x80131506.
Reproduction Steps
It crashes spontaneously: could be once a hour or few days without errors. By stack trace it seem depends on Task.Start
Expected behavior
Work without spontaneously crashes
Actual behavior
Dump analyze via WinDbg
************* Preparing the environment for Debugger Extensions Gallery repositories **************
ExtensionRepository : Implicit
UseExperimentalFeatureForNugetShare : true
AllowNugetExeUpdate : true
AllowNugetMSCredentialProviderInstall : true
AllowParallelInitializationOfLocalRepositories : true
-- Configuring repositories
----> Repository : LocalInstalled, Enabled: true
----> Repository : UserExtensions, Enabled: true
Preparing the environment for Debugger Extensions Gallery repositories completed, duration 0.000 seconds
************* Waiting for Debugger Extensions Gallery to Initialize **************
Waiting for Debugger Extensions Gallery to Initialize completed, duration 0.016 seconds
----> Repository : UserExtensions, Enabled: true, Packages count: 0
----> Repository : LocalInstalled, Enabled: true, Packages count: 36
Microsoft (R) Windows Debugger Version 10.0.25921.1001 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.
Loading Dump File [H:\server.exe.18680.dmp]
User Mini Dump File with Full Memory: Only application data is available
************* Path validation summary **************
Response Time (ms) Location
Deferred srv*
Symbol search path is: srv*
Executable search path is:
Windows 10 Version 20348 MP (16 procs) Free x64
Product: Server, suite: TerminalServer SingleUserTS
Edition build lab: 20348.1.amd64fre.fe_release.210507-1500
Debug session time: Tue Jan 16 15:58:19.000 2024 (UTC + 2:00)
System Uptime: 10 days 8:20:29.450
Process Uptime: 0 days 0:25:50.000
................................................................
................................................................
.....................
Loading unloaded module list
.
This dump file has an exception of interest stored in it.
The stored exception information can be accessed via .ecxr.
(48f8.4508): Access violation - code c0000005 (first/second chance not available)
For analysis of this file, run !analyze -v
coreclr!CLREventBase::Set [inlined in coreclr!ObjectNative::PulseAll+0x126]:
00007ffa27e587a6 488b09 mov rcx,qword ptr [rcx] ds:000096ac
68d2d80a=????????????????
0:904> !analyze -v
-
*
-
Exception Analysis *
-
*
Failed to request MethodData, not in JIT code range
KEY_VALUES_STRING: 1
Key : AV.Fault
Value: Read
Key : Analysis.CPU.mSec
Value: 2983
Key : Analysis.Elapsed.mSec
Value: 17261
Key : Analysis.IO.Other.Mb
Value: 15
Key : Analysis.IO.Read.Mb
Value: 1
Key : Analysis.IO.Write.Mb
Value: 20
Key : Analysis.Init.CPU.mSec
Value: 13218
Key : Analysis.Init.Elapsed.mSec
Value: 1129306
Key : Analysis.Memory.CommitPeak.Mb
Value: 243
Key : CLR.Engine
Value: CORECLR
Key : CLR.Version
Value: 8.0.123.58001
Key : Failure.Bucket
Value: INVALID_POINTER_READ_c0000005_coreclr.dll!ObjectNative::PulseAll
Key : Failure.Hash
Value: {be7e1d5f-f864-f771-a72a-7be333bc8c49}
Key : Failure.Source.FileLine
Value: 340
Key : Failure.Source.FilePath
Value: D:\a\_work\1\s\src\coreclr\classlibnative\bcltype\objectnative.cpp
Key : Failure.Source.SourceServerCommand
Value: raw.githubusercontent.com/dotnet/runtime/bf5e279d9239bfef5bb1b8d6212f1b971c434606/src/coreclr/classlibnative/bcltype/objectnative.cpp
Key : Timeline.OS.Boot.DeltaSec
Value: 894029
Key : Timeline.Process.Start.DeltaSec
Value: 1550
Key : WER.OS.Branch
Value: fe_release
Key : WER.OS.Version
Value: 10.0.20348.1
Key : WER.Process.Version
Value: 1.0.10.1
FILE_IN_CAB: server.exe.18680.dmp
NTGLOBALFLAG: 0
PROCESS_BAM_CURRENT_THROTTLED: 0
PROCESS_BAM_PREVIOUS_THROTTLED: 0
APPLICATION_VERIFIER_FLAGS: 0
CONTEXT: (.ecxr)
rax=000000ab838fef10 rbx=000000ab829ff200 rcx=000096ac68d2d80a
rdx=000000ab838fef30 rsi=000000ab829fea70 rdi=000000ab829fed10
rip=00007ffa27e587a6 rsp=000000ab829ff2d0 rbp=000000ab829ff4a0
r8=0000000000000054 r9=0000000000000056 r10=0000000000000007
r11=0000000000000246 r12=0000000000000000 r13=0000000000000000
r14=0000027c7107f878 r15=0000000000000000
iopl=0 nv up ei pl nz na pe nc
cs=0033 ss=002b ds=002b es=002b fs=0053 gs=002b efl=00010200
coreclr!CLREventBase::Set [inlined in coreclr!ObjectNative::PulseAll+0x126]:
00007ffa27e587a6 488b09 mov rcx,qword ptr [rcx] ds:000096ac
68d2d80a=????????????????
Resetting default scope
EXCEPTION_RECORD: (.exr -1)
ExceptionAddress: 00007ffa27e587a6 (coreclr!CLREventBase::Set)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000001
NumberParameters: 2
Parameter[0]: 0000000000000000
Parameter[1]: ffffffffffffffff
Attempt to read from address ffffffffffffffff
PROCESS_NAME: server.exe
READ_ADDRESS: ffffffffffffffff
ERROR_CODE: (NTSTATUS) 0xc0000005 - 0x%p 0x%p. %s.
EXCEPTION_CODE_STR: c0000005
EXCEPTION_PARAMETER1: 0000000000000000
EXCEPTION_PARAMETER2: ffffffffffffffff
IP_ON_HEAP: 00007ff9c9423f8a
The fault address in not in any loaded module, please check your build's rebase
log at \bin\build_logs\timebuild\ntrebase.log for module which may
contain the address if it were loaded.
FRAME_ONE_INVALID: 1
STACK_TEXT:
000000ab829ff2d0 00007ff9
c9423f8a : 0000023c675483a0 0000023c
675483a0 0000023c608a3118 0000023c
67548388 : coreclr!ObjectNative::PulseAll+0x126
000000ab829ff430 00007ff9
ca5b0048 : 0000023c67548388 00007ff9
00000000 000000ab829ff4a0 0000027c
6f03e3d0 : System_Private_CoreLib!System.Threading.ManualResetEventSlim.Set+0x8a
000000ab829ff4b0 00007ff9
ca224447 : 0000023c67548210 00000000
00000000 000000ab829ff540 00000000
00000000 : System_Private_CoreLib!System.Threading.Tasks.Task.RunContinuations+0x128
000000ab829ff550 00007ff9
ca177076 : 0000027c72496170 00000000
00000000 0000027c72496170 00007ffa
56a87563 : System_Private_CoreLib!System.Threading.Tasks.Task.FinishSlow+0x37
000000ab829ff5b0 00007ffa
27eca333 : 0000023c67548210 0000023c
675495d8 0000023c5c822590 00000000
00000000 : System_Private_CoreLib!System.Threading.Tasks.Task.ExecuteWithThreadLocal+0x116
000000ab829ff660 00007ffa
27d84cb4 : 0000000000000008 00000000
00000130 000000ab829ff770 00007ffa
00000000 : coreclr!CallDescrWorkerInternal+0x83
000000ab829ff6a0 00007ffa
27eb74f3 : 0000023c5c81a250 00000000
00000000 0000027c6f03df7c 00000000
00000000 : coreclr!DispatchCallSimple+0x60
000000ab829ff730 00007ffa
27e589dd : 0000027c6f03df70 00007ffa
27e59308 0000027c00004508 0000023c
5c820008 : coreclr!ThreadNative::KickOffThread_Worker+0x63
000000ab829ff790 00007ffa
27e588f3 : 000000ab829ff808 0000023c
5c853fe0 0000000000000000 00000000
00000000 : coreclr!ManagedThreadBase_DispatchMiddle+0x85
000000ab829ff870 00007ffa
27e58a8e : 0000027c00000001 ffffffff
ffffffff 0000023c5c822590 0000023c
5c822590 : coreclr!ManagedThreadBase_DispatchOuter+0xab
000000ab829ff910 00007ffa
56ee4de0 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : coreclr!ThreadNative::KickOffThread+0x7e
000000ab829ff970 00007ffa
5741ed9b : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : kernel32!BaseThreadInitThunk+0x10
000000ab829ff9a0 00000000
00000000 : 0000000000000000 00000000
00000000 0000000000000000 00000000
00000000 : ntdll!RtlUserThreadStart+0x2b
FAULTING_SOURCE_LINE: D:\a_work\1\s\src\coreclr\classlibnative\bcltype\objectnative.cpp
FAULTING_SOURCE_FILE: D:\a_work\1\s\src\coreclr\classlibnative\bcltype\objectnative.cpp
FAULTING_SOURCE_LINE_NUMBER: 340
FAULTING_SOURCE_SRV_COMMAND: https://raw.githubusercontent.com/dotnet/runtime/bf5e279d9239bfef5bb1b8d6212f1b971c434606/src/coreclr/classlibnative/bcltype/objectnative.cpp
FAULTING_SOURCE_CODE:
No source found for 'D:\a_work\1\s\src\coreclr\vm\synch.cpp'
SYMBOL_NAME: coreclr!ObjectNative::PulseAll+126
MODULE_NAME: coreclr
IMAGE_NAME: coreclr.dll
STACK_COMMAND: ~904s; .ecxr ; kb
FAILURE_BUCKET_ID: INVALID_POINTER_READ_c0000005_coreclr.dll!ObjectNative::PulseAll
OS_VERSION: 10.0.20348.1
BUILDLAB_STR: fe_release
OSPLATFORM_TYPE: x64
OSNAME: Windows 10
IMAGE_VERSION: 8.0.123.58001
FAILURE_ID_HASH: {be7e1d5f-f864-f771-a72a-7be333bc8c49}
Followup: MachineOwner
Regression?
We dont have crash on Framework 4.5 / 4.8, but it reproduced on NET 7 / 8
Known Workarounds
Service MySQL56
Configuration
NET 8.0.1, VM with Windows Server 2022 x64, 48 GB RAM
Other information
No response