Skip to content

Assert failure: (g_fEEShutDown&ShutDown_Finalize1) && GetThread() == FinalizerThread::GetFinalizerThread() #106242

New issue
New issue
Closed
#116885
Closed
Assert failure: (g_fEEShutDown&ShutDown_Finalize1) && GetThread() == FinalizerThread::GetFinalizerThread()#106242
#116885
Assignees
VSadov
Labels
Known Build ErrorUse this to report build issues in the .NET Helix tabarch-arm64area-GC-coreclrblocking-clean-ci-optionalBlocking optional rolling runsos-mac-os-xmacOS aka OSX
Milestone
10.0.0
@v-wenyuxu

Description

@v-wenyuxu
v-wenyuxu
opened on Aug 12, 2024 · edited by build-analysis

Failed in: runtime-coreclr jitstress-isas-arm 20240810.1

Failed tests:

coreclr osx arm64 Checked jitstress_isas_incompletehwintrinsic @ OSX.1200.ARM64.Open
    - GC/API/GC/GetTotalAllocatedBytes/GetTotalAllocatedBytes.sh

Error message:

 
Assert failure(PID 40598 [0x00009e96], Thread: 2901095 [0x2c4467]): (g_fEEShutDown&ShutDown_Finalize1) && GetThread() == FinalizerThread::GetFinalizerThread()
    File: /Users/runner/work/1/s/src/coreclr/vm/appdomain.cpp:3670
    Image: /private/tmp/helix/working/A8B4092F/p/corerun

waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0
/private/tmp/helix/working/A8B4092F/w/A7A2096B/e/GC/API/GC/GetTotalAllocatedBytes/GetTotalAllocatedBytes.sh: line 439: 40598 Abort trap: 6           (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"

Return code:      1
Raw output file:      /tmp/helix/working/A8B4092F/w/A7A2096B/uploads/Reports/GC.API/GC/GetTotalAllocatedBytes/GetTotalAllocatedBytes.output.txt
Raw output:
BEGIN EXECUTION
/tmp/helix/working/A8B4092F/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true GetTotalAllocatedBytes.dll ''
[createdump] Gathering state for process 40598 
[createdump] Crashing thread 2c4467 signal 6 (0006)
[createdump] Writing crash report to file /cores/coredump.40598.dmp.crashreport.json
[createdump] Crash report successfully written
[createdump] Writing minidump with heap to file /cores/coredump.40598.dmp
[createdump] Written 489799848 bytes (29895 pages) to core file
[createdump] Target process is alive
[createdump] Dump successfully written in 1319ms
Expected: 100
Actual: 134
END EXECUTION - FAILED
Test failed. Trying to see if dump file was created in /cores since 8/10/2024 1:23:56 PM
Processing /cores/coredump.40598.dmp.crashreport.json
Printing stacktrace from '/cores/coredump.40598.dmp.crashreport.json'
Invoking llvm-symbolizer --pretty-print
Errors while running llvm-symbolizer --pretty-print
System.ComponentModel.Win32Exception (2): An error occurred trying to start process 'llvm-symbolizer' with working directory '/private/tmp/helix/working/A8B4092F/w/A7A2096B/e'. No such file or directory
   at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at CoreclrTestLib.CoreclrTestWrapperLib.TryPrintStackTraceFromCrashReport(String crashReportJsonFile, TextWriter outputWriter)
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=/tmp/helix/working/A8B4092F/p
> /private/tmp/helix/working/A8B4092F/w/A7A2096B/e/GC/API/GC/GetTotalAllocatedBytes/GetTotalAllocatedBytes.sh

Stack trace:

   at GC_API._GC_GetTotalAllocatedBytes_GetTotalAllocatedBytes_._GC_GetTotalAllocatedBytes_GetTotalAllocatedBytes_sh()

   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "(g_fEEShutDown&ShutDown_Finalize1) && GetThread() == FinalizerThread::GetFinalizerThread()",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=907494&view=results
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.
Validation performed at: 1/7/2025 2:15:36 PM UTC

Known issue validation

Build: 🔎
Result validation: ⚠️ Provided build not found. Provide a valid build in the "Build: 🔎" line.
Validation performed at: 1/7/2025 2:17:27 PM UTC

Report

Build Definition Test Pull Request
1073102 dotnet/runtime tracing/runtimeeventsource/nativeruntimeeventsource/nativeruntimeeventsource.cmd #116834
1050720 dotnet/runtime tracing/runtimeeventsource/nativeruntimeeventsource/nativeruntimeeventsource.cmd #115977

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 1 2

Activity

v-wenyuxu
added
arch-arm64
os-mac-os-xmacOS aka OSX
JitStressCLR JIT issues involving JIT internal stress modes
blocking-clean-ci-optionalBlocking optional rolling runs
on Aug 12, 2024
ghost added
area-GC-coreclr
on Aug 12, 2024
dotnet-policy-service
added
untriagedNew issue has not been triaged by the area owner
on Aug 12, 2024
dotnet-policy-service

dotnet-policy-service commented on Aug 12, 2024

@dotnet-policy-service
dotnet-policy-servicebot
on Aug 12, 2024
Contributor

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

v-wenyuxu

v-wenyuxu commented on Aug 19, 2024

@v-wenyuxu
v-wenyuxu
on Aug 19, 2024
Author

Failed in: runtime-coreclr jitstressregs 20240818.1

Failed tests:

coreclr osx arm64 Checked jitstressregs0x2000 @ OSX.1200.ARM64.Open
    - profiler/handles/handles/handles.sh

Error message:

 
Assert failure(PID 1564 [0x0000061c], Thread: 18210 [0x4722]): (g_fEEShutDown&ShutDown_Finalize1) && GetThread() == FinalizerThread::GetFinalizerThread()
    File: /Users/runner/work/1/s/src/coreclr/vm/appdomain.cpp:3661
    Image: /private/tmp/helix/working/B24F0A0D/p/corerun

waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0
Unhandled exception. System.Exception: Profiler tests are expected to contain the text 'PROFILER TEST PASSES' in the console output of the profilee app to indicate a passing test. Usually it is printed from the Shutdown() method of the profiler implementation. This text was not found in the output above. Profilee returned exit code 134.
   at Profiler.Tests.ProfilerTestRunner.FailFastWithMessage(String error)
   at Profiler.Tests.ProfilerTestRunner.Run(String profileePath, String testName, Guid profilerClsid, String profileeArguments, ProfileeOptions profileeOptions, Dictionary`2 envVars, String reverseServerName, Boolean loadAsNotification, Int32 notificationCopies)
   at Profiler.Tests.HandlesTests.Main(String[] args)
waitpid() returned successfully (wstatus 00000000) WEXITSTATUS 0 WTERMSIG 0
/private/tmp/helix/working/B24F0A0D/w/A66808DF/e/profiler/handles/handles/handles.sh: line 446:  1563 Abort trap: 6           (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"

Return code:      1
Raw output file:      /tmp/helix/working/B24F0A0D/w/A66808DF/uploads/Reports/profiler.handles/handles/handles.output.txt
Raw output:
BEGIN EXECUTION
/tmp/helix/working/B24F0A0D/p/corerun -p System.Reflection.Metadata.MetadataUpdater.IsSupported=false -p System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true handles.dll ''
Profiler path: /private/tmp/helix/working/B24F0A0D/w/A66808DF/e/profiler/handles/handles/libProfiler.dylib
Profilee STDOUT: Profiler.dll!DllGetClassObject
Profilee STDOUT: Profiler.dll!Profiler::Initialize
Profilee STDOUT: Handles started. Control-C to exit
Profilee STDOUT: Worker thread started
Profilee STDOUT: Allocating Weak
Profilee STDOUT: Allocating Strong
Profilee STDOUT: Allocating Pinned
Profilee STDOUT: weak = 1-W
Profilee STDOUT: strong = 2-S
Profilee STDOUT: weak = 3-P
Profilee STDOUT: Collection #1
Profilee STDOUT: HandlesProfiler::ObjectAllocated: weak handle created.
Profilee STDOUT: HandlesProfiler::ObjectAllocated: strong handle created.
Profilee STDOUT: HandlesProfiler::ObjectAllocated: pinned handle created.
Profilee STDOUT: HandlesProfiler::CheckIfAlive(weak): object not alive as expected.
Profilee STDOUT: HandlesProfiler::CheckIfAlive(strong): object alive as expected (Profiler.Tests.TestClassForStrongHandle)
Profilee STDOUT: HandlesProfiler::CheckIfAlive(pinned): object alive as expected (Profiler.Tests.TestClassForPinnedHandle)
Profilee STDOUT: Collection #2
Profilee STDOUT: HandlesProfiler::GarbageCollectionFinished(#1): pinned handle object address did not changed as expected.
Profilee STDOUT: HandlesProfiler::CheckIfAlive(strong): object alive as expected (Profiler.Tests.TestClassForStrongHandle)
Profilee STDOUT: HandlesProfiler::CheckIfAlive(pinned): object alive as expected (Profiler.Tests.TestClassForPinnedHandle)
Profilee STDOUT: Collection #3
Profilee STDOUT: Test Passed
Profilee STDOUT: HandlesProfiler::GarbageCollectionFinished(#2): strong handle destroyed.
Profilee STDOUT: HandlesProfiler::GarbageCollectionFinished(#2): pinned handle destroyed.
Profilee STDOUT: HandlesProfiler::GarbageCollectionFinished(#3): Checking handles:
Profilee STDOUT: HandlesProfiler::CheckIfAlive(strong): object not alive as expected.
Profilee STDOUT: HandlesProfiler::CheckIfAlive(pinned): object not alive as expected.
Profilee STDOUT: [createdump] Gathering state for process 1564 
Profilee STDOUT: [createdump] Crashing thread 4722 signal 6 (0006)
Profilee STDOUT: [createdump] Writing crash report to file /cores/coredump.1564.dmp.crashreport.json
Profilee STDOUT: [createdump] Crash report successfully written
Profilee STDOUT: [crea

Stack trace:

   at profiler_handles._handles_handles_._handles_handles_sh()

   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
AndyAyersMS

AndyAyersMS commented on Aug 23, 2024

@AndyAyersMS
AndyAyersMS
on Aug 23, 2024
Member

@VSadov this looks like some kind of assert during finalization, can you take a look?

VSadov

VSadov commented on Aug 23, 2024

@VSadov
VSadov
on Aug 23, 2024
Member

Seems like something with RaiseShutdownEvents sequence.
I wonder what part of this assert did not actually hold. Only finalizer thread ever calls this function and the quit flag is set after setting up global ShutDown_Finalize1.

Perhaps due to some race theGetThread() == FinalizerThread::GetFinalizerThread() is not holding - i.e. something with identity of the finalizer thread was not right.

VSadov
self-assigned this
on Aug 23, 2024
VSadov

VSadov commented on Aug 23, 2024

@VSadov
VSadov
on Aug 23, 2024
Member

the failures that were reported are in completely different tests, but both times on osx-arm64.

mangod9
removed
untriagedNew issue has not been triaged by the area owner
on Sep 4, 2024
mangod9
added this to the 10.0.0 milestone on Sep 4, 2024
jakobbotsch

jakobbotsch commented on Jan 7, 2025

@jakobbotsch
jakobbotsch
on Jan 7, 2025
Member

Also happened in https://dev.azure.com/dnceng-public/public/_build/results?buildId=907494&view=results in tracing/runtimeeventsource/nativeruntimeeventsource/nativeruntimeeventsource.cmd, also on osx-arm64. Does not seem jitstress related.

jakobbotsch
added
Known Build ErrorUse this to report build issues in the .NET Helix tab
and removed
JitStressCLR JIT issues involving JIT internal stress modes
on Jan 7, 2025
jakobbotsch
changed the title [-]Test failure: GC/API/GC/GetTotalAllocatedBytes/GetTotalAllocatedBytes.sh[/-] [+]Assert failure: `(g_fEEShutDown&ShutDown_Finalize1) && GetThread() == FinalizerThread::GetFinalizerThread()`[/+] on Jan 7, 2025
jkotas
added a commit that references this issue on Jun 20, 2025

Change g_fEEShutDown to Volatile

0115a8f
jkotas
mentioned this on Jun 20, 2025
jkotas
closed this as completedin #116885on Jun 21, 2025
stephentoub
added a commit that references this issue on Jun 21, 2025

Change g_fEEShutDown to be Volatile (dotnet#116885)

f0168ee
github-actions
locked and limited conversation to collaborators on Jul 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Known Build ErrorUse this to report build issues in the .NET Helix tabarch-arm64area-GC-coreclrblocking-clean-ci-optionalBlocking optional rolling runsos-mac-os-xmacOS aka OSX

Type

No type

Projects

No projects

Milestone

Relationships

None yet

    Development

    Participants

    @jakobbotsch@VSadov@AndyAyersMS@mangod9@v-wenyuxu

    Issue actions
      Assert failure: `(g_fEEShutDown&ShutDown_Finalize1) && GetThread() == FinalizerThread::GetFinalizerThread()` · Issue #106242 · dotnet/runtime