Application Hang and Stops responding to API calls #71320
Description
Description
We are facing issue intermittently, application hangs and stop responding to api calls. After creating memory dump and analyzing it in DebugDiag and Windbg we are getting following information.
In dotnet.DMP GC is running in this process. The Thread that triggered the GC is [45]
The following threads in dotnet.DMP are waiting for .net garbage collection to finish. Thread 45 triggered the garbage collection.The gargage collector thread wont start doing its work till the time the threads which have pre-emptive GC disabled have finished executing. The following threads have pre-emptive GC disabled [41]
Below is the call stack from Thread 41:
Current frame: ntdll!NtSetEvent + 0xa
Child-SP RetAddr Caller, Callee
00000000131AE6E0 000007fefcd0132c KERNELBASE!SetEvent + 0xc, calling ntdll!NtSetEvent
00000000131AE710 000007fedc7e2328 coreclr!ObjectNative::PulseAll + 0x138 [c:\a\1\s\src\classlibnative\bcltype\objectnative.cpp:330], calling 00000000004d004d
00000000131AE750 000007fedaf42778 (MethodDesc 000007feda8d07b8 + 0x68 DomainNeutralILStubClass.IL_STUB_PInvoke())
00000000131AE800 000007fedaeea67f (MethodDesc 000007feda972e58 + 0x5f System.Threading.ThreadPoolWorkQueue.EnsureThreadRequested()), calling 000007fedaaaf320 (stub for System.Threading.ThreadPool.RequestWorkerThread())
00000000131AE820 000007fedaeaf186 (MethodDesc 000007feda96bd98 + 0x66 System.Threading.Tasks.Task..ctor(System.Action`1<System.Object>, System.Object, System.Threading.CancellationToken, System.Threading.Tasks.TaskCreationOptions)), calling (MethodDesc 000007feda96bda8 + 0 System.Threading.Tasks.Task..ctor(System.Delegate, System.Object, System.Threading.Tasks.Task, System.Threading.CancellationToken, System.Threading.Tasks.TaskCreationOptions, System.Threading.Tasks.InternalTaskOptions, System.Threading.Tasks.TaskScheduler))
00000000131AE838 000007fedc7e226d coreclr!ObjectNative::PulseAll + 0x7d [c:\a\1\s\src\classlibnative\bcltype\objectnative.cpp:325], calling coreclr!LazyMachStateCaptureState [C:\a\1\s\src\vm\amd64\getstate.asm:62]
00000000131AE870 000007fedaee68e6 (MethodDesc 000007feda972018 + 0x66 System.Threading.ManualResetEventSlim.Set(Boolean)), calling (MethodDesc 000007feda953840 + 0 System.Threading.Monitor.PulseAll(System.Object))
00000000131AE8A0 000007fef06a4430 (MethodDesc 000007fe805cd988 + 0x100 System.Threading.Tasks.TaskReplicator+Replica.Execute()), calling (MethodDesc 000007feda96bf10 + 0 System.Threading.Tasks.Task.Start(System.Threading.Tasks.TaskScheduler))
00000000131AE8B8 000007fedc830d51 coreclr!ThreadNative::GetThreadState + 0x91 [c:\a\1\s\src\vm\comsynchronizable.cpp:902], calling coreclr!LazyMachStateCaptureState [C:\a\1\s\src\vm\amd64\getstate.asm:62]
00000000131AE8E0 000007fedaeb274f (MethodDesc 000007feda96c428 + 0x1df System.Threading.Tasks.Task.RunContinuations(System.Object)), calling (MethodDesc 000007feda96c438 + 0 System.Threading.Tasks.Task.RunOrQueueCompletionAction(System.Threading.Tasks.ITaskCompletionAction, Boolean))
00000000131AE900 000007fef06a4549 (MethodDesc 000007fe805ce518 + 0x19 System.Threading.Tasks.TaskReplicator+Replica+<>c.<Execute>b__7_0(System.Object)), calling (MethodDesc 000007fe805cd988 + 0 System.Threading.Tasks.TaskReplicator+Replica.Execute())
00000000131AE930 000007fedadfd469 (MethodDesc 000007feda952f90 + 0x89 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object))
00000000131AE960 000007fedaeb0d13 (MethodDesc 000007feda96c200 + 0x153 System.Threading.Tasks.Task.FinishStageTwo()), calling coreclr!JIT_GetSharedNonGCStaticBase_SingleAppDomain [C:\a\1\s\src\vm\amd64\JitHelpers_SingleAppDomain.asm:26]
00000000131AE9B0 000007fedaeb147e (MethodDesc 000007feda96c2a0 + 0x1be System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)), calling (MethodDesc 000007feda96c200 + 0 System.Threading.Tasks.Task.FinishStageTwo())
00000000131AE9D0 000007fedc7fcf08 coreclr!JIT_GetSharedGCThreadStaticBase + 0xa8 [c:\a\1\s\src\vm\jithelpers.cpp:1928], calling coreclr!__security_check_cookie [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\gs\amd64\amdsecgs.asm:45]
00000000131AEA00 000007fedaeea643 (MethodDesc 000007feda972e58 + 0x23 System.Threading.ThreadPoolWorkQueue.EnsureThreadRequested()), calling coreclr!JIT_GetSharedNonGCStaticBase_SingleAppDomain [C:\a\1\s\src\vm\amd64\JitHelpers_SingleAppDomain.asm:26]
00000000131AEA50 000007fedaeeaa5c (MethodDesc 000007feda972ea8 + 0x19c System.Threading.ThreadPoolWorkQueue.Dispatch()), calling 000007fe7cd17360
00000000131AEAF0 000007fedc863033 coreclr!CallDescrWorkerInternal + 0x83 [C:\a\1\s\src\vm\amd64\CallDescrWorkerAMD64.asm:101]
00000000131AEAF8 000007fedc752594 coreclr!MethodDescCallSite::CallTargetWorker + 0x94 [c:\a\1\s\src\vm\callhelpers.cpp:470], calling coreclr!__chkstk [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\chkstk.asm:67]
00000000131AEB30 000007fedc752773 coreclr!MethodDescCallSite::CallTargetWorker + 0x273 [c:\a\1\s\src\vm\callhelpers.cpp:628], calling coreclr!CallDescrWorkerInternal [C:\a\1\s\src\vm\amd64\CallDescrWorkerAMD64.asm:38]
00000000131AEBA0 000007fedc752594 coreclr!MethodDescCallSite::CallTargetWorker + 0x94 [c:\a\1\s\src\vm\callhelpers.cpp:470], calling coreclr!__chkstk [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\chkstk.asm:67]
00000000131AEBD0 000007fedc859caa coreclr!MetaSig::GetReturnTypeNormalized + 0x26 [c:\a\1\s\src\vm\siginfo.cpp:5225], calling coreclr!SigPointer::PeekElemTypeNormalized [c:\a\1\s\src\vm\siginfo.cpp:2384]
00000000131AEC70 000007fedc752400 coreclr!QueueUserWorkItemManagedCallback + 0x90 [c:\a\1\s\src\vm\comthreadpool.cpp:451], calling coreclr!MethodDescCallSite::CallTargetWorker [c:\a\1\s\src\vm\callhelpers.cpp:333]
00000000131AED60 000007fedc7522b0 coreclr!ManagedThreadBase_DispatchMiddle + 0xe4 [c:\a\1\s\src\vm\threads.cpp:8901], calling coreclr!QueueUserWorkItemManagedCallback [c:\a\1\s\src\vm\comthreadpool.cpp:435]
00000000131AED80 000007fedc7d75c9 coreclr!EEHeapAllocInProcessHeap + 0x29 [c:\a\1\s\src\vm\hosting.cpp:463], calling ntdll!RtlAllocateHeap
00000000131AEE80 000007fedc752137 coreclr!ManagedThreadBase_DispatchOuter + 0xaf [c:\a\1\s\src\vm\threads.cpp:9140], calling coreclr!ManagedThreadBase_DispatchMiddle [c:\a\1\s\src\vm\threads.cpp:8855]
00000000131AEEA0 000007fedc752137 coreclr!ManagedThreadBase_DispatchOuter + 0xaf [c:\a\1\s\src\vm\threads.cpp:9140], calling coreclr!ManagedThreadBase_DispatchMiddle [c:\a\1\s\src\vm\threads.cpp:8855]
00000000131AEF40 000007fedc752033 coreclr!ManagedPerAppDomainTPCount::DispatchWorkItem + 0xe3 [c:\a\1\s\src\vm\threadpoolrequest.cpp:757], calling coreclr!ManagedThreadBase_DispatchOuter [c:\a\1\s\src\vm\threads.cpp:9092]
00000000131AEF70 000007fedc792b18 coreclr!Thread::ResetManagedThreadObjectInCoopMode + 0x2c [c:\a\1\s\src\vm\threads.cpp:10295], calling coreclr!ThreadBaseObject::ResetManagedThreadCulture [c:\a\1\s\src\vm\comsynchronizable.cpp:1368]
00000000131AEFB0 0000000076d59871 kernel32!GetQueuedCompletionStatusStub + 0x11, calling 0000000000410012
00000000131AEFF0 000007fedc7f2241 coreclr!CLRLifoSemaphore::WaitForSignal + 0x2d [c:\a\1\s\src\vm\synch.cpp:671], calling kernel32!GetQueuedCompletionStatusStub
00000000131AF050 000007fedc7f406a coreclr!CLRLifoSemaphore::Wait + 0x12a [c:\a\1\s\src\vm\synch.cpp:915], calling coreclr!CLRLifoSemaphore::WaitForSignal [c:\a\1\s\src\vm\synch.cpp:636]
00000000131AF060 000007fedc751ec7 coreclr!PerAppDomainTPCountList::GetAppDomainIndexForThreadpoolDispatch + 0x11f [c:\a\1\s\src\vm\threadpoolrequest.cpp:302], calling coreclr!_guard_dispatch_icall_nop [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\guard_dispatch.asm:57]
00000000131AF0A0 000007fedc7517b3 coreclr!ThreadpoolMgr::WorkerThreadStart + 0x623 [c:\a\1\s\src\vm\win32threadpool.cpp:2024], calling coreclr!ManagedPerAppDomainTPCount::DispatchWorkItem [c:\a\1\s\src\vm\threadpoolrequest.cpp:701]
00000000131AF210 0000000076d7142a kernel32!HeapFree + 0xa, calling ntdll!RtlFreeHeap
00000000131AF240 000007fedc7d6929 coreclr!EEHeapFreeInProcessHeap + 0x29 [c:\a\1\s\src\vm\hosting.cpp:532], calling kernel32!HeapFree
00000000131AF270 000007fedc7d68ef coreclr!operator delete + 0x2f [c:\a\1\s\src\utilcode\clrhost_nodependencies.cpp:495], calling coreclr!EEHeapFreeInProcessHeap [c:\a\1\s\src\vm\hosting.cpp:507]
00000000131AF2B0 000007fedc78abba coreclr!Thread::intermediateThreadProc + 0x8a [c:\a\1\s\src\vm\threads.cpp:2255], calling coreclr!_guard_dispatch_icall_nop [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\guard_dispatch.asm:57]
00000000131AF4E0 0000000076f97d4e ntdll!RtlAllocateHeap + 0x17e, calling ntdll!memset
00000000131AF520 000007fefcaa17ae crypt32!Ordinal1001 + 0x52a, calling crypt32!Ordinal1001 + 0x560
00000000131AF580 0000000076f97d4e ntdll!RtlAllocateHeap + 0x17e, calling ntdll!memset
00000000131AF5F0 000007fedd6be8c2 Microsoft_DiaSymReader_Native_amd64!_vcrt_FlsSetValue + 0x4a
00000000131AF620 000007fedd6bce41 Microsoft_DiaSymReader_Native_amd64!_vcrt_getptd_noexit + 0xa5, calling kernel32!SetLastError
00000000131AF650 000007fedd6bcaad Microsoft_DiaSymReader_Native_amd64!_vcrt_thread_attach + 0x9, calling Microsoft_DiaSymReader_Native_amd64!_vcrt_getptd_noexit
00000000131AF690 000007feeca937ee VCRUNTIME140_1!__vcrt_FlsSetValue + 0x3e [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\winapi_downlevel.cpp:293], calling VCRUNTIME140_1!_guard_dispatch_icall_nop [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\misc\amd64\guard_dispatch.asm:53]
00000000131AF6B0 000007fedd6bb08d Microsoft_DiaSymReader_Native_amd64!dllmain_crt_dispatch + 0x2d, calling Microsoft_DiaSymReader_Native_amd64!_scrt_dllmain_crt_thread_attach
00000000131AF6C0 000007feeca934b9 VCRUNTIME140_1!__vcrt_getptd_noexit + 0xad [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\per_thread_data.cpp:141], calling kernel32!SetLastError
00000000131AF6E0 0000000076fb41a1 ntdll!LdrpCallTlsInitializers + 0x41, calling ntdll!RtlReleaseSRWLockShared
00000000131AF6F0 000007feeca932bd VCRUNTIME140_1!__vcrt_thread_attach + 0x9 [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\initialization.cpp:104], calling VCRUNTIME140_1!__vcrt_getptd_noexit [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\per_thread_data.cpp:92]
00000000131AF710 000007fef98c11da mscoree!CRT_INIT + 0x1b8, calling mscoree!DllMain
00000000131AF720 000007feeca9415a VCRUNTIME140_1!DllMainDispatch + 0x2a [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\dll\vcruntime_dllmain.cpp:58], calling VCRUNTIME140_1!__vcrt_thread_attach [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\initialization.cpp:100]
00000000131AF750 0000000076fa743f ntdll!LdrpInitializeThread + 0x22f, calling ntdll!RtlLeaveCriticalSection
00000000131AF810 0000000076fa736e ntdll!LdrpInitializeThread + 0x15e, calling ntdll!RtlActivateActivationContextUnsafeFast
00000000131AF818 0000000076fa73aa ntdll!LdrpInitializeThread + 0x19a, calling ntdll!RtlDeactivateActivationContextUnsafeFast
00000000131AF850 0000000076fb2874 ntdll!LdrpInitialize + 0xa4, calling ntdll!ZwTestAlert
00000000131AF8C0 0000000076fb28f8 ntdll!LdrInitializeThunk + 0x18, calling ntdll!ZwContinue
00000000131AFD30 000007fedc78ab97 coreclr!Thread::intermediateThreadProc + 0x67 [c:\a\1\s\src\vm\threads.cpp:2250], calling coreclr!__chkstk [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\chkstk.asm:67]
00000000131AFD70 0000000076d655fd kernel32!BaseThreadInitThunk + 0xd
00000000131AFDA0 0000000076fc294d ntdll!RtlUserThreadStart + 0x1d
Call stack from thread 45:
`
Current frame: ntdll!ZwWaitForSingleObject + 0xa
Child-SP RetAddr Caller, Callee
0000000016A6D2F0 000007fefcd010dc KERNELBASE!WaitForSingleObjectEx + 0x79, calling ntdll!ZwWaitForSingleObject
0000000016A6D390 000007fedc7e3b98 coreclr!CLREventWaitHelper + 0x20 [c:\a\1\s\src\vm\synch.cpp:411], calling kernel32!WaitForSingleObjectExImplementation
0000000016A6D3F0 000007fedc816968 coreclr!SVR::gc_heap::wait_for_gc_done + 0xa8 [c:\a\1\s\src\gc\gc.cpp:10226], calling coreclr!CLREventBase::WaitEx [c:\a\1\s\src\vm\synch.cpp:431]
0000000016A6D420 000007fedc81682a coreclr!SVR::GCHeap::GarbageCollectGeneration + 0x17a [c:\a\1\s\src\gc\gc.cpp:35074], calling coreclr!SVR::gc_heap::wait_for_gc_done [c:\a\1\s\src\gc\gc.cpp:10220]
0000000016A6D470 000007fedc78ccd4 coreclr!SVR::gc_heap::try_allocate_more_space + 0x1a4 [c:\a\1\s\src\gc\gc.cpp:13199], calling coreclr!SVR::GCHeap::GarbageCollectGeneration [c:\a\1\s\src\gc\gc.cpp:34958]
0000000016A6D4D0 0000000076f97d4e ntdll!RtlAllocateHeap + 0x17e, calling ntdll!memset
0000000016A6D530 000007fedc78d480 coreclr!SVR::gc_heap::a_fit_segment_end_p + 0x100 [c:\a\1\s\src\gc\gc.cpp:12174], calling coreclr!SVR::gc_heap::adjust_limit_clr [c:\a\1\s\src\gc\gc.cpp:11364]
0000000016A6D540 000007fedc74d38e coreclr!RuntimeTypeHandle::CreateInstance + 0x34e [c:\a\1\s\src\vm\reflectioninvocation.cpp:554], calling coreclr!__security_check_cookie [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\gs\amd64\amdsecgs.asm:45]
0000000016A6D5A0 000007fedc82e527 coreclr!SVR::gc_heap::allocate_large_object + 0x177 [c:\a\1\s\src\gc\gc.cpp:30636], calling coreclr!SVR::gc_heap::try_allocate_more_space [c:\a\1\s\src\gc\gc.cpp:13119]
0000000016A6D650 000007fedc78cb1a coreclr!SVR::GCHeap::Alloc + 0x16a [c:\a\1\s\src\gc\gc.cpp:34427], calling coreclr!SVR::gc_heap::allocate_large_object [c:\a\1\s\src\gc\gc.cpp:30587]
0000000016A6D690 000007fedc75032f coreclr!FastAllocatePrimitiveArray + 0x197 [c:\a\1\s\src\vm\gchelpers.cpp:819], calling coreclr!_guard_dispatch_icall_nop [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\guard_dispatch.asm:57]
0000000016A6D6E0 000007fedc7d41bc coreclr!SafeHandle::DangerousRelease + 0x1ec [c:\a\1\s\src\vm\safehandle.cpp:422], calling coreclr!__security_check_cookie [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\gs\amd64\amdsecgs.asm:45]
0000000016A6D760 000007fedc74ff19 coreclr!JIT_NewArr1 + 0x209 [c:\a\1\s\src\vm\jithelpers.cpp:3200], calling coreclr!FastAllocatePrimitiveArray [c:\a\1\s\src\vm\gchelpers.cpp:741]
0000000016A6D790 000007fedadf5c33 (MethodDesc 000007feda9514d0 + 0xf3 System.MemoryExtensions.ToUpperInvariant(System.ReadOnlySpan`1<Char>, System.Span`1<Char>)), calling (MethodDesc 000007feda95c2a0 + 0 System.Globalization.TextInfo.ChangeCase(System.ReadOnlySpan`1<Char>, System.Span`1<Char>, Boolean))
0000000016A6D7D0 000007fedc7d44e4 coreclr!SafeHandle::DangerousAddRef + 0x194 [c:\a\1\s\src\vm\safehandle.cpp:408], calling coreclr!__security_check_cookie [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\gs\amd64\amdsecgs.asm:45]
0000000016A6D800 000007fedaddfd36 (MethodDesc 000007feda945dc8 + 0x166 System.Globalization.CompareInfo.GetIgnoreCaseHash(System.String)), calling (MethodDesc 000007feda950030 + 0 System.Marvin.ComputeHash32(Byte ByRef, Int32, UInt64))
0000000016A6D8D8 000007fedc74fd97 coreclr!JIT_NewArr1 + 0x87 [c:\a\1\s\src\vm\jithelpers.cpp:3136], calling coreclr!LazyMachStateCaptureState [C:\a\1\s\src\vm\amd64\getstate.asm:62]
0000000016A6D8F0 000007fedc82abc4 coreclr!JIT_GetSharedGCStaticBaseDynamicClass + 0x54 [c:\a\1\s\src\vm\jithelpers.cpp:1631], calling coreclr!__security_check_cookie [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\gs\amd64\amdsecgs.asm:45]
0000000016A6D920 000007fedadc7dd4 (MethodDesc 000007feda942320 + 0x34 System.Text.Encoding.GetBytes(System.String)), calling coreclr!JIT_NewArr1VC_MP_InlineGetThread [C:\a\1\s\src\vm\amd64\JitHelpers_InlineGetThread.asm:167]
0000000016A6D930 000007fedae7b3e7 (MethodDesc 000007feda9655d8 + 0x47 System.IO.MemoryStream..ctor(Int32)), calling coreclr!JIT_WriteBarrier [C:\a\1\s\src\vm\amd64\JitHelpers_Fast.asm:466]
0000000016A6D970 000007fe7df2bc17 (MethodDesc 000007fe7e0c6788 + 0xf7 Product.UntrustedProvider.Services.HttpClientProvider+<>c__DisplayClass46_0+<<PostAsync>b__0>d.MoveNext()), calling (MethodDesc 000007feda942320 + 0 System.Text.Encoding.GetBytes(System.String))
0000000016A6D9A0 000007fefcd013c6 KERNELBASE!BaseSetLastNTError + 0x16, calling ntdll!RtlSetLastWin32Error
0000000016A6D9D0 000007fedc82984a coreclr!StubHelpers::SetLastError + 0xa [c:\a\1\s\src\vm\stubhelpers.cpp:1075], calling kernel32!GetLastErrorStub
0000000016A6D9E8 000007fedc7d43e3 coreclr!SafeHandle::DangerousAddRef + 0x93 [c:\a\1\s\src\vm\safehandle.cpp:398], calling coreclr!LazyMachStateCaptureState [C:\a\1\s\src\vm\amd64\getstate.asm:62]
0000000016A6DA70 000007fe7df2ba09 (MethodDesc 000007fe7e596bc8 + 0x39 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.UntrustedProvider.Services.HttpClientProvider+<>c__DisplayClass46_0+<<PostAsync>b__0>d, Product.UntrustedProvider.Services]](<<PostAsync>b__0>d ByRef)), calling (MethodDesc 000007fe7e0c6788 + 0 Product.UntrustedProvider.Services.HttpClientProvider+<>c__DisplayClass46_0+<<PostAsync>b__0>d.MoveNext())
0000000016A6DAE0 000007fe7df2b988 (MethodDesc 000007fe7e0a2348 + 0x38 Product.UntrustedProvider.Services.HttpClientProvider+<>c__DisplayClass46_0.<PostAsync>b__0()), calling (MethodDesc 000007fe7e596bc8 + 0 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.UntrustedProvider.Services.HttpClientProvider+<>c__DisplayClass46_0+<<PostAsync>b__0>d, Product.UntrustedProvider.Services]](<<PostAsync>b__0>d ByRef))
0000000016A6DB50 000007fe7de866b6 (MethodDesc 000007fe7e2c65f8 + 0x16 System.Security.Principal.WindowsIdentity+<>c__DisplayClass67_0`1[[System.__Canon, System.Private.CoreLib]].<RunImpersonated>b__0())
0000000016A6DB80 000007fedadfd469 (MethodDesc 000007feda952f90 + 0x89 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object))
0000000016A6DBA0 000007fe7de8292d (MethodDesc 000007fe7ddf9530 + 0x6d System.Security.Principal.WindowsIdentity.GetCurrentToken(System.Security.Principal.TokenAccessLevels, Boolean, Boolean ByRef, Int32 ByRef)), calling (MethodDesc 000007fe7ddf9540 + 0 System.Security.Principal.WindowsIdentity.GetCurrentProcessToken(System.Security.Principal.TokenAccessLevels, Int32 ByRef))
0000000016A6DC00 000007fe7de8276f (MethodDesc 000007fe7ddf94e0 + 0xef System.Security.Principal.WindowsIdentity.RunImpersonatedInternal(Microsoft.Win32.SafeHandles.SafeAccessTokenHandle, System.Action)), calling (MethodDesc 000007feda952f90 + 0 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object))
0000000016A6DC50 000007fe7de864c9 (MethodDesc 000007fe7e2c6398 + 0x89 System.Security.Principal.WindowsIdentity.RunImpersonated[[System.__Canon, System.Private.CoreLib]](Microsoft.Win32.SafeHandles.SafeAccessTokenHandle, System.Func`1<System.__Canon>)), calling (MethodDesc 000007fe7ddf94e0 + 0 System.Security.Principal.WindowsIdentity.RunImpersonatedInternal(Microsoft.Win32.SafeHandles.SafeAccessTokenHandle, System.Action))
0000000016A6DCA0 000007fe7de86294 (MethodDesc 000007fe7e2a49b0 + 0x2e4 Product.UntrustedProvider.Services.HttpClientProvider.GetData[[System.__Canon, System.Private.CoreLib]](System.Func`1<System.Threading.Tasks.Task`1<System.__Canon>>)), calling (MethodDesc 000007fe7e2c6398 + 0 System.Security.Principal.WindowsIdentity.RunImpersonated[[System.__Canon, System.Private.CoreLib]](Microsoft.Win32.SafeHandles.SafeAccessTokenHandle, System.Func`1<System.__Canon>))
0000000016A6DD50 000007fe7df2b66c (MethodDesc 000007fe7e0a24f0 + 0x28c Product.UntrustedProvider.Services.HttpClientProvider+<PostAsync>d__46.MoveNext()), calling (MethodDesc 000007fe7e2a49b0 + 0 Product.UntrustedProvider.Services.HttpClientProvider.GetData[[System.__Canon, System.Private.CoreLib]](System.Func`1<System.Threading.Tasks.Task`1<System.__Canon>>))
0000000016A6DE10 000007fe7df2b2c9 (MethodDesc 000007fe7e596988 + 0x39 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.UntrustedProvider.Services.HttpClientProvider+<PostAsync>d__46, Product.UntrustedProvider.Services]](<PostAsync>d__46 ByRef)), calling (MethodDesc 000007fe7e0a24f0 + 0 Product.UntrustedProvider.Services.HttpClientProvider+<PostAsync>d__46.MoveNext())
0000000016A6DE30 000007fedc78cb1a coreclr!SVR::GCHeap::Alloc + 0x16a [c:\a\1\s\src\gc\gc.cpp:34427], calling coreclr!SVR::gc_heap::allocate_large_object [c:\a\1\s\src\gc\gc.cpp:30587]
0000000016A6DE80 000007fe7df2b24a (MethodDesc 000007fe7d04fc98 + 0x9a Product.UntrustedProvider.Services.HttpClientProvider.PostAsync(System.String, System.String, Boolean, System.String, System.String, System.Nullable`1<System.TimeSpan>, Boolean, Boolean)), calling (MethodDesc 000007fe7e596988 + 0 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.UntrustedProvider.Services.HttpClientProvider+<PostAsync>d__46, Product.UntrustedProvider.Services]](<PostAsync>d__46 ByRef))
0000000016A6DF20 000007fe7df2afa9 (MethodDesc 000007fe7e0a2198 + 0xb9 Product.UntrustedProvider.Services.HttpClientProvider+<PostAsync>d__45.MoveNext()), calling (MethodDesc 000007fe7d04fc98 + 0 Product.UntrustedProvider.Services.HttpClientProvider.PostAsync(System.String, System.String, Boolean, System.String, System.String, System.Nullable`1<System.TimeSpan>, Boolean, Boolean))
0000000016A6DF30 000007fedae8061a (MethodDesc 000007feda965f60 + 0x5a System.IO.Path.GetPathRoot(System.ReadOnlySpan`1<Char>)), calling (MethodDesc 000007feda966140 + 0 System.IO.PathInternal.GetRootLength(System.ReadOnlySpan`1<Char>))
0000000016A6DFE0 000007fe7df2add9 (MethodDesc 000007fe7e596748 + 0x39 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.UntrustedProvider.Services.HttpClientProvider+<PostAsync>d__45, Product.UntrustedProvider.Services]](<PostAsync>d__45 ByRef)), calling (MethodDesc 000007fe7e0a2198 + 0 Product.UntrustedProvider.Services.HttpClientProvider+<PostAsync>d__45.MoveNext())
0000000016A6DFF0 000007fedae02fba (MethodDesc 000007feda954158 + 0x13a System.Collections.Generic.Dictionary`2[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]].FindEntry(System.__Canon))
0000000016A6E000 000007fedae01f54 (MethodDesc 000007feda954070 + 0xc4 System.Collections.Generic.Dictionary`2[[System.__Canon, System.Private.CoreLib],[System.__Canon, System.Private.CoreLib]]..ctor(Int32, System.Collections.Generic.IEqualityComparer`1<System.__Canon>)), calling coreclr!JIT_WriteBarrier [C:\a\1\s\src\vm\amd64\JitHelpers_Fast.asm:466]
0000000016A6E050 000007fe7df2ad57 (MethodDesc 000007fe7d04fc90 + 0x87 Product.UntrustedProvider.Services.HttpClientProvider.PostAsync(System.String, System.String, System.String, System.String, System.Nullable`1<System.TimeSpan>, Boolean, Boolean)), calling (MethodDesc 000007fe7e596748 + 0 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.UntrustedProvider.Services.HttpClientProvider+<PostAsync>d__45, Product.UntrustedProvider.Services]](<PostAsync>d__45 ByRef))
0000000016A6E0E0 000007fe7e7c6f7f (MethodDesc 000007fe7e0a72c0 + 0x1ff Product.UntrustedProvider.Services.ProductUntrustedProviderService+<SaveModel>d__19.MoveNext()), calling 000007fe7cc7b060
0000000016A6E100 000007fee7e62d51 (MethodDesc 000007fe7d33b4f8 + 0x111 System.Net.Sockets.SocketAsyncEventArgs.ProcessIOCPResultWithSingleBufferHandle(System.Net.Sockets.SocketError, Int32, System.Threading.NativeOverlapped*)), calling (MethodDesc 000007feda9ad7d8 + 0 System.Memory`1[[System.Byte, System.Private.CoreLib]].Pin())
0000000016A6E1F0 000007fe7e7c6c49 (MethodDesc 000007fe7eb480f8 + 0x39 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.UntrustedProvider.Services.ProductUntrustedProviderService+<SaveModel>d__19, Product.UntrustedProvider.Services]](<SaveModel>d__19 ByRef)), calling (MethodDesc 000007fe7e0a72c0 + 0 Product.UntrustedProvider.Services.ProductUntrustedProviderService+<SaveModel>d__19.MoveNext())
0000000016A6E210 000007fee7e59333 (MethodDesc 000007fe7dd91200 + 0xc3 System.Net.Sockets.Socket.ReceiveAsync(System.Net.Sockets.SocketAsyncEventArgs)), calling (MethodDesc 000007feda9715d0 + 0 System.Diagnostics.Tracing.EventSource.IsEnabled())
0000000016A6E220 000007fe7dc9cfea (MethodDesc 000007fe7dd9eb48 + 0x2a Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.Internal.SocketAwaitable.OnCompleted(System.Action)), calling 000007fedc82c630 (stub for System.Threading.Interlocked.CompareExchange(System.Object ByRef, System.Object, System.Object))
0000000016A6E260 000007fe7e7c6bcd (MethodDesc 000007fe7e0a1360 + 0x3d Product.UntrustedProvider.Services.ProductUntrustedProviderService.SaveModel(Product.Api.Model.v1_0_0.Loading.ModelDto)), calling (MethodDesc 000007fe7eb480f8 + 0 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.UntrustedProvider.Services.ProductUntrustedProviderService+<SaveModel>d__19, Product.UntrustedProvider.Services]](<SaveModel>d__19 ByRef))
0000000016A6E2D0 000007fe7e7c69be (MethodDesc 000007fe7e0879d8 + 0x5e Product.DataProvider.ModelDataProvider+<SaveModel>d__34.MoveNext()), calling 000007fe7cc7f700
0000000016A6E2E0 000007fedc83434b coreclr!ClrFlsIncrementValue + 0x1f [c:\a\1\s\src\inc\clrhost.h:99], calling coreclr!_guard_dispatch_icall_nop [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\guard_dispatch.asm:57]
0000000016A6E350 000007fe7e7c6849 (MethodDesc 000007fe7eb47278 + 0x39 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.DataProvider.ModelDataProvider+<SaveModel>d__34, Product.DataProvider]](<SaveModel>d__34 ByRef)), calling (MethodDesc 000007fe7e0879d8 + 0 Product.DataProvider.ModelDataProvider+<SaveModel>d__34.MoveNext())
0000000016A6E360 000007fefcd01710 KERNELBASE!PostQueuedCompletionStatus + 0x20, calling ntdll!NtSetIoCompletion
0000000016A6E3C0 000007fe7e7c649d (MethodDesc 000007fe7e082fe0 + 0x3d Product.DataProvider.ModelDataProvider.SaveModel(Product.Api.Model.v1_0_0.Loading.ModelDto, Product.Core.Model.User)), calling (MethodDesc 000007fe7eb47278 + 0 System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[Product.DataProvider.ModelDataProvider+<SaveModel>d__34, Product.DataProvider]](<SaveModel>d__34 ByRef))
0000000016A6E3D0 000007fedc790217 coreclr!ThreadpoolMgr::MaybeAddWorkingWorker + 0xe3 [c:\a\1\s\src\vm\win32threadpool.cpp:1047], calling coreclr!CLRLifoSemaphore::Release [c:\a\1\s\src\vm\synch.cpp:923]
0000000016A6E430 000007fe7e7c642a (MethodDesc 000007fe7e04ad38 + 0x2a Product.Api.Services.LegalEntityService+<>c__DisplayClass70_0.<SaveModel>b__1()), calling 000007fe7cc7f6e0
0000000016A6E460 000007fedadfd469 (MethodDesc 000007feda952f90 + 0x89 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object))
0000000016A6E480 000007fedaf42778 (MethodDesc 000007feda8d07b8 + 0x68 DomainNeutralILStubClass.IL_STUB_PInvoke())
0000000016A6E4E0 000007fedaeb1457 (MethodDesc 000007feda96c2a0 + 0x197 System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)), calling (MethodDesc 000007feda952f90 + 0 System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object))
0000000016A6E500 000007fedc7fcf08 coreclr!JIT_GetSharedGCThreadStaticBase + 0xa8 [c:\a\1\s\src\vm\jithelpers.cpp:1928], calling coreclr!__security_check_cookie [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\gs\amd64\amdsecgs.asm:45]
0000000016A6E530 000007fedaeea67f (MethodDesc 000007feda972e58 + 0x5f System.Threading.ThreadPoolWorkQueue.EnsureThreadRequested()), calling 000007fedaaaf320 (stub for System.Threading.ThreadPool.RequestWorkerThread())
0000000016A6E580 000007fedaeeaa5c (MethodDesc 000007feda972ea8 + 0x19c System.Threading.ThreadPoolWorkQueue.Dispatch()), calling 000007fe7cd17360
0000000016A6E620 000007fedc863033 coreclr!CallDescrWorkerInternal + 0x83 [C:\a\1\s\src\vm\amd64\CallDescrWorkerAMD64.asm:101]
0000000016A6E628 000007fedc752594 coreclr!MethodDescCallSite::CallTargetWorker + 0x94 [c:\a\1\s\src\vm\callhelpers.cpp:470], calling coreclr!__chkstk [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\chkstk.asm:67]
0000000016A6E660 000007fedc752773 coreclr!MethodDescCallSite::CallTargetWorker + 0x273 [c:\a\1\s\src\vm\callhelpers.cpp:628], calling coreclr!CallDescrWorkerInternal [C:\a\1\s\src\vm\amd64\CallDescrWorkerAMD64.asm:38]
0000000016A6E6D0 000007fedc752594 coreclr!MethodDescCallSite::CallTargetWorker + 0x94 [c:\a\1\s\src\vm\callhelpers.cpp:470], calling coreclr!__chkstk [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\chkstk.asm:67]
0000000016A6E700 000007fedc859caa coreclr!MetaSig::GetReturnTypeNormalized + 0x26 [c:\a\1\s\src\vm\siginfo.cpp:5225], calling coreclr!SigPointer::PeekElemTypeNormalized [c:\a\1\s\src\vm\siginfo.cpp:2384]
0000000016A6E7A0 000007fedc752400 coreclr!QueueUserWorkItemManagedCallback + 0x90 [c:\a\1\s\src\vm\comthreadpool.cpp:451], calling coreclr!MethodDescCallSite::CallTargetWorker [c:\a\1\s\src\vm\callhelpers.cpp:333]
0000000016A6E890 000007fedc7522b0 coreclr!ManagedThreadBase_DispatchMiddle + 0xe4 [c:\a\1\s\src\vm\threads.cpp:8901], calling coreclr!QueueUserWorkItemManagedCallback [c:\a\1\s\src\vm\comthreadpool.cpp:435]
0000000016A6E8B0 000007fedc7d75c9 coreclr!EEHeapAllocInProcessHeap + 0x29 [c:\a\1\s\src\vm\hosting.cpp:463], calling ntdll!RtlAllocateHeap
0000000016A6E9B0 000007fedc752137 coreclr!ManagedThreadBase_DispatchOuter + 0xaf [c:\a\1\s\src\vm\threads.cpp:9140], calling coreclr!ManagedThreadBase_DispatchMiddle [c:\a\1\s\src\vm\threads.cpp:8855]
0000000016A6E9D0 000007fedc752137 coreclr!ManagedThreadBase_DispatchOuter + 0xaf [c:\a\1\s\src\vm\threads.cpp:9140], calling coreclr!ManagedThreadBase_DispatchMiddle [c:\a\1\s\src\vm\threads.cpp:8855]
0000000016A6EA70 000007fedc752033 coreclr!ManagedPerAppDomainTPCount::DispatchWorkItem + 0xe3 [c:\a\1\s\src\vm\threadpoolrequest.cpp:757], calling coreclr!ManagedThreadBase_DispatchOuter [c:\a\1\s\src\vm\threads.cpp:9092]
0000000016A6EAA0 000007fedc792b18 coreclr!Thread::ResetManagedThreadObjectInCoopMode + 0x2c [c:\a\1\s\src\vm\threads.cpp:10295], calling coreclr!ThreadBaseObject::ResetManagedThreadCulture [c:\a\1\s\src\vm\comsynchronizable.cpp:1368]
0000000016A6EAE0 0000000076d59871 kernel32!GetQueuedCompletionStatusStub + 0x11, calling 0000000000410012
0000000016A6EB20 000007fedc7f2241 coreclr!CLRLifoSemaphore::WaitForSignal + 0x2d [c:\a\1\s\src\vm\synch.cpp:671], calling kernel32!GetQueuedCompletionStatusStub
0000000016A6EB80 000007fedc7f406a coreclr!CLRLifoSemaphore::Wait + 0x12a [c:\a\1\s\src\vm\synch.cpp:915], calling coreclr!CLRLifoSemaphore::WaitForSignal [c:\a\1\s\src\vm\synch.cpp:636]
0000000016A6EB90 000007fedc751e50 coreclr!PerAppDomainTPCountList::GetAppDomainIndexForThreadpoolDispatch + 0xa8 [c:\a\1\s\src\vm\threadpoolrequest.cpp:316], calling coreclr!ManagedPerAppDomainTPCount::TakeActiveRequest [c:\a\1\s\src\vm\threadpoolrequest.cpp:646]
0000000016A6EBD0 000007fedc7517b3 coreclr!ThreadpoolMgr::WorkerThreadStart + 0x623 [c:\a\1\s\src\vm\win32threadpool.cpp:2024], calling coreclr!ManagedPerAppDomainTPCount::DispatchWorkItem [c:\a\1\s\src\vm\threadpoolrequest.cpp:701]
0000000016A6ED40 0000000076d7142a kernel32!HeapFree + 0xa, calling ntdll!RtlFreeHeap
0000000016A6ED70 000007fedc7d6929 coreclr!EEHeapFreeInProcessHeap + 0x29 [c:\a\1\s\src\vm\hosting.cpp:532], calling kernel32!HeapFree
0000000016A6EDA0 000007fedc7d68ef coreclr!operator delete + 0x2f [c:\a\1\s\src\utilcode\clrhost_nodependencies.cpp:495], calling coreclr!EEHeapFreeInProcessHeap [c:\a\1\s\src\vm\hosting.cpp:507]
0000000016A6EDE0 000007fedc78abba coreclr!Thread::intermediateThreadProc + 0x8a [c:\a\1\s\src\vm\threads.cpp:2255], calling coreclr!_guard_dispatch_icall_nop [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\guard_dispatch.asm:57]
0000000016A6EF90 0000000076f97d4e ntdll!RtlAllocateHeap + 0x17e, calling ntdll!memset
0000000016A6EFD0 000007fefcaa17ae crypt32!Ordinal1001 + 0x52a, calling crypt32!Ordinal1001 + 0x560
0000000016A6F030 0000000076f97d4e ntdll!RtlAllocateHeap + 0x17e, calling ntdll!memset
0000000016A6F0A0 000007fedd6be8c2 Microsoft_DiaSymReader_Native_amd64!_vcrt_FlsSetValue + 0x4a
0000000016A6F0D0 000007fedd6bce41 Microsoft_DiaSymReader_Native_amd64!_vcrt_getptd_noexit + 0xa5, calling kernel32!SetLastError
0000000016A6F100 000007fedd6bcaad Microsoft_DiaSymReader_Native_amd64!_vcrt_thread_attach + 0x9, calling Microsoft_DiaSymReader_Native_amd64!_vcrt_getptd_noexit
0000000016A6F140 000007feeca937ee VCRUNTIME140_1!__vcrt_FlsSetValue + 0x3e [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\winapi_downlevel.cpp:293], calling VCRUNTIME140_1!_guard_dispatch_icall_nop [D:\a\_work\1\s\src\vctools\crt\vcstartup\src\misc\amd64\guard_dispatch.asm:53]
0000000016A6F160 000007fedd6bb08d Microsoft_DiaSymReader_Native_amd64!dllmain_crt_dispatch + 0x2d, calling Microsoft_DiaSymReader_Native_amd64!_scrt_dllmain_crt_thread_attach
0000000016A6F170 000007feeca934b9 VCRUNTIME140_1!__vcrt_getptd_noexit + 0xad [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\per_thread_data.cpp:141], calling kernel32!SetLastError
0000000016A6F190 0000000076fb41a1 ntdll!LdrpCallTlsInitializers + 0x41, calling ntdll!RtlReleaseSRWLockShared
0000000016A6F1A0 000007feeca932bd VCRUNTIME140_1!__vcrt_thread_attach + 0x9 [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\initialization.cpp:104], calling VCRUNTIME140_1!__vcrt_getptd_noexit [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\per_thread_data.cpp:92]
0000000016A6F1C0 000007fef98c11da mscoree!CRT_INIT + 0x1b8, calling mscoree!DllMain
0000000016A6F1D0 000007feeca9415a VCRUNTIME140_1!DllMainDispatch + 0x2a [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\dll\vcruntime_dllmain.cpp:58], calling VCRUNTIME140_1!__vcrt_thread_attach [D:\a\_work\1\s\src\vctools\crt\vcruntime\src\internal\initialization.cpp:100]
0000000016A6F200 0000000076fa743f ntdll!LdrpInitializeThread + 0x22f, calling ntdll!RtlLeaveCriticalSection
0000000016A6F2C0 0000000076fa736e ntdll!LdrpInitializeThread + 0x15e, calling ntdll!RtlActivateActivationContextUnsafeFast
0000000016A6F2C8 0000000076fa73aa ntdll!LdrpInitializeThread + 0x19a, calling ntdll!RtlDeactivateActivationContextUnsafeFast
0000000016A6F300 0000000076fb2874 ntdll!LdrpInitialize + 0xa4, calling ntdll!ZwTestAlert
0000000016A6F370 0000000076fb28f8 ntdll!LdrInitializeThunk + 0x18, calling ntdll!ZwContinue
0000000016A6F7E0 000007fedc78ab97 coreclr!Thread::intermediateThreadProc + 0x67 [c:\a\1\s\src\vm\threads.cpp:2250], calling coreclr!__chkstk [d:\agent\_work\2\s\src\vctools\crt\vcstartup\src\misc\amd64\chkstk.asm:67]
0000000016A6F820 0000000076d655fd kernel32!BaseThreadInitThunk + 0xd
0000000016A6F850 0000000076fc294d ntdll!RtlUserThreadStart + 0x1d'''
`
Please let me know if anything needed. Any help is much appreciated. Thanks
Reproduction Steps
Intermittent issue
Expected behavior
Process Should Not Hang
Actual behavior
Process Hangs
Regression?
No response
Known Workarounds
No response
Configuration
No response
Other information
No response