[ARM32] [GC] Incorrect stack unwinding

[alpencolt]

Using script on debug version from #8391:

$ cat repro.sh
#!/bin/sh
ulimit -c unlimited
try=0
while true
do
  try=$((try+1))
  clr-debug/corerun tests-release/JIT/Methodical/cctor/misc/threads1_cs_r/threads1_cs_r.exe >/dev/null 2>&1
  if [ $? = 100 ]
  then
    echo -n .
  else
    echo
    echo Failed on a try \#$try
    exit
  fi
done

cause assertion:

#0  __libc_do_syscall () at ../sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:47
dotnet/coreclr#1  0xb6ced648 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
dotnet/coreclr#2  0xb6cee34a in __GI_abort () at abort.c:89
dotnet/coreclr#3  0xb69e8df0 in PROCAbort () at /home/alexander/src/coreclr/src/pal/src/thread/process.cpp:3046
dotnet/coreclr#4  0xb69e5af4 in PROCEndProcess (hProcess=0xffffff01, uExitCode=123456789, bTerminateUnconditionally=1) at /home/alexander/src/coreclr/src/pal/src/thread/process.cpp:1394
dotnet/coreclr#5  0xb69e5c32 in TerminateProcess (hProcess=0xffffff01, uExitCode=123456789) at /home/alexander/src/coreclr/src/pal/src/thread/process.cpp:1310
dotnet/coreclr#6  0xb63a282a in TerminateOnAssert () at /home/alexander/src/coreclr/src/utilcode/debug.cpp:183
dotnet/coreclr#7  0xb63a3062 in _DbgBreakCheck (szFile=0xb6a84008 "/home/alexander/src/coreclr/src/vm/object.cpp", iLine=1733, szExpr=0xb6a8436a "!CREATE_CHECK_STRING(pMT && pMT->Validate())", fConstrained=0)
    at /home/alexander/src/coreclr/src/utilcode/debug.cpp:436
dotnet/coreclr#8  0xb63a3440 in _DbgBreakCheckNoThrow (szFile=0xb6a84008 "/home/alexander/src/coreclr/src/vm/object.cpp", iLine=1733, szExpr=0xb6a8436a "!CREATE_CHECK_STRING(pMT && pMT->Validate())", fConstrained=0)
    at /home/alexander/src/coreclr/src/utilcode/debug.cpp:548
dotnet/coreclr#9  0xb63a379a in DbgAssertDialog (szFile=0xb6a84008 "/home/alexander/src/coreclr/src/vm/object.cpp", iLine=1733, szExpr=0xb6a8436a "!CREATE_CHECK_STRING(pMT && pMT->Validate())") at /home/alexander/src/coreclr/src/utilcode/debug.cpp:735
dotnet/coreclr#10 0xb645345e in Object::ValidateInner (this=0xad1fdd78, bDeep=1, bVerifyNextHeader=1, bVerifySyncBlock=1) at /home/alexander/src/coreclr/src/vm/object.cpp:1733
dotnet/coreclr#11 0xb6452980 in Object::Validate (this=0xad1fdd78, bDeep=1, bVerifyNextHeader=1, bVerifySyncBlock=1) at /home/alexander/src/coreclr/src/vm/object.cpp:1709
dotnet/coreclr#12 0xb6613eba in GcInfoDecoder::ReportStackSlotToGC (this=0xad1f8f5c, spOffset=-16, spBase=GC_FRAMEREG_REL, gcFlags=0, pRD=0xad1f9858, flags=2, pCallBack=0xb652db29 <GcEnumObject(void*, OBJECTREF*, unsigned int)>, hCallBack=0xad1f9e80)
    at /home/alexander/src/coreclr/src/vm/gcinfodecoder.cpp:1821
dotnet/coreclr#13 0xb661466c in GcInfoDecoder::ReportSlotToGC (this=0xad1f8f5c, slotDecoder=..., slotIndex=5, pRD=0xad1f9858, reportScratchSlots=false, inputFlags=2, pCallBack=0xb652db29 <GcEnumObject(void*, OBJECTREF*, unsigned int)>, 
    hCallBack=0xad1f9e80) at /home/alexander/src/coreclr/src/inc/gcinfodecoder.h:665
dotnet/coreclr#14 0xb6612b46 in GcInfoDecoder::EnumerateLiveSlots (this=0xad1f8f5c, pRD=0xad1f9858, reportScratchSlots=false, inputFlags=2, pCallBack=0xb652db29 <GcEnumObject(void*, OBJECTREF*, unsigned int)>, hCallBack=0xad1f9e80)
    at /home/alexander/src/coreclr/src/vm/gcinfodecoder.cpp:934
dotnet/coreclr#15 0xb63d6d40 in EECodeManager::EnumGcRefs (this=0x62008, pRD=0xad1f9858, pCodeInfo=0xad1f9710, flags=2, pCallBack=0xb652db29 <GcEnumObject(void*, OBJECTREF*, unsigned int)>, hCallBack=0xad1f9e80, relOffsetOverride=4294967295)
    at /home/alexander/src/coreclr/src/vm/eetwain.cpp:5062
dotnet/coreclr#16 0xb652e080 in GcStackCrawlCallBack (pCF=0xad1f94f0, pData=0xad1f9e80) at /home/alexander/src/coreclr/src/vm/gcenv.ee.common.cpp:280
dotnet/coreclr#17 0xb64797c8 in Thread::MakeStackwalkerCallback (this=0xe5998, pCF=0xad1f94f0, pCallback=0xb652dca5 <GcStackCrawlCallBack(CrawlFrame*, void*)>, pData=0xad1f9e80, uFramesProcessed=54)
    at /home/alexander/src/coreclr/src/vm/stackwalk.cpp:864
dotnet/coreclr#18 0xb647996a in Thread::StackWalkFramesEx (this=0xe5998, pRD=0xad1f9858, pCallback=0xb652dca5 <GcStackCrawlCallBack(CrawlFrame*, void*)>, pData=0xad1f9e80, flags=34048, pStartFrame=0x0)
    at /home/alexander/src/coreclr/src/vm/stackwalk.cpp:945
dotnet/coreclr#19 0xb647a1ee in Thread::StackWalkFrames (this=0xe5998, pCallback=0xb652dca5 <GcStackCrawlCallBack(CrawlFrame*, void*)>, pData=0xad1f9e80, flags=34048, pStartFrame=0x0) at /home/alexander/src/coreclr/src/vm/stackwalk.cpp:1028
dotnet/coreclr#20 0xb6792d74 in ScanStackRoots (pThread=0xe5998, fn=0xb667f071 <WKS::GCHeap::Promote(Object**, ScanContext*, unsigned int)>, sc=0xad1f9fe0) at /home/alexander/src/coreclr/src/vm/gcenv.ee.cpp:149
dotnet/coreclr#21 0xb6792af4 in GCToEEInterface::GcScanRoots (fn=0xb667f071 <WKS::GCHeap::Promote(Object**, ScanContext*, unsigned int)>, condemned=0, max_gen=2, sc=0xad1f9fe0) at /home/alexander/src/coreclr/src/vm/gcenv.ee.cpp:178
dotnet/coreclr#22 0xb67d3426 in GCScan::GcScanRoots (fn=0xb667f071 <WKS::GCHeap::Promote(Object**, ScanContext*, unsigned int)>, condemned=0, max_gen=2, sc=0xad1f9fe0) at /home/alexander/src/coreclr/src/gc/gcscan.cpp:155
dotnet/coreclr#23 0xb6673586 in WKS::gc_heap::mark_phase (condemned_gen_number=0, mark_only_p=0) at /home/alexander/src/coreclr/src/gc/gc.cpp:19609
dotnet/coreclr#24 0xb66713dc in WKS::gc_heap::gc1 () at /home/alexander/src/coreclr/src/gc/gc.cpp:15394
dotnet/runtime#3858 0xb6679b5a in WKS::gc_heap::garbage_collect (n=0) at /home/alexander/src/coreclr/src/gc/gc.cpp:16971
dotnet/runtime#3859 0xb666d8a8 in WKS::GCHeap::GarbageCollectGeneration (this=0x56200, gen=0, reason=reason_alloc_soh) at /home/alexander/src/coreclr/src/gc/gc.cpp:35098
dotnet/coreclr#27 0xb666ec38 in WKS::gc_heap::try_allocate_more_space (acontext=0xe59d8, size=12, gen_number=0) at /home/alexander/src/coreclr/src/gc/gc.cpp:13198
dotnet/runtime#3860 0xb666ed50 in WKS::gc_heap::allocate_more_space (acontext=0xe59d8, size=12, alloc_generation_number=0) at /home/alexander/src/coreclr/src/gc/gc.cpp:13492
dotnet/runtime#3861 0xb66974b8 in WKS::gc_heap::allocate (jsize=12, acontext=0xe59d8) at /home/alexander/src/coreclr/src/gc/gc.cpp:13523
dotnet/runtime#3862 0xb668f47c in WKS::GCHeap::Alloc (this=0x56200, context=0xe59d8, size=12, flags=0) at /home/alexander/src/coreclr/src/gc/gc.cpp:34440
dotnet/runtime#3863 0xb6530602 in Alloc (size=12, bFinalize=0, bContainsPointers=0) at /home/alexander/src/coreclr/src/vm/gchelpers.cpp:241
dotnet/coreclr#32 0xb652ff10 in AllocateObject (pMT=0xb465cb98) at /home/alexander/src/coreclr/src/vm/gchelpers.cpp:1171
dotnet/runtime#3864 0xb65492be in JIT_New (typeHnd_=0xb465cb98) at /home/alexander/src/coreclr/src/vm/jithelpers.cpp:2818
dotnet/runtime#3865 0xadd61a8c in ?? ()

Checked version failed with assertion in System.Threading.ExecutionContext.Restore(). Debug version failed as well but on printing stack trace occur error above.
The reason of it is that GetGCSafeMethodTable() in Object::ValidateInner() returns 0. It looks that GC tries to crawl over incorrect stack frame.

There is closed #8892 with the same error and similar bug in CoreRT dotnet/corert#4626. So it look something's not right in this process.

cc @dotnet/arm32-contrib @jkotas

[jkotas]

Any bug in GC info will tend to crash with this call stack. It is where the similarity with other mentioned bugs ends. The root cause is most likely going to be different.

[BruceForstall]

@alpencolt Have you had any luck investigating this? Have you tried running with COMPlus_GCStress=4 or COMPlus_GCStress=C? That typically makes a bad JIT-generated GC info problem repeatable.

[alpencolt]

@BruceForstall I'm working on CoreRT related issues right now, so there is no progress.
Test crash with the same assertion with both COMPlus_GCStress=4and COMPlus_GCStress=C options almost immediately on first try.
I haven't checked on latest master.

[BruceForstall]

I can't reproduce this. It looks similar to issues that have recently been fixed. So I'm going to close the issue.