GCStress + JitStress test failures

[BruceForstall]

The following fail with Windows x64 Checked and:

COMPlus_TieredCompilation=0
COMPlus_GCStress=0xC
COMPlus_JitStress=1

Interop\BestFitMapping\BestFitMapping\BestFitMapping.cmd
Interop\COM\NETClients\Events\NETClientEvents\NETClientEvents.cmd
Interop\COM\NETClients\IDispatch\NETClientIDispatch\NETClientIDispatch.cmd
Interop\COM\NETClients\Licensing\NETClientLicense\NETClientLicense.cmd
Interop\COM\NETClients\Primitives\NETClientPrimitives\NETClientPrimitives.cmd
Interop\COM\NETClients\Primitives\NETClientPrimitivesInALC\NETClientPrimitivesInALC.cmd
Interop\COM\Reflection\Reflection\Reflection.cmd
Interop\IJW\FixupCallsHostWhenLoaded\FixupCallsHostWhenLoaded\FixupCallsHostWhenLoaded.cmd
Interop\IJW\NativeCallingManaged\NativeCallingManaged\NativeCallingManaged.cmd
Interop\IJW\NativeVarargs\NativeVarargsTest\NativeVarargsTest.cmd
Interop\NativeLibrary\NativeLibraryTests\NativeLibraryTests.cmd
Interop\PInvoke\Array\MarshalArrayAsParam\AsDefault\AsDefaultTest\AsDefaultTest.cmd
Interop\PInvoke\Array\MarshalArrayAsParam\AsLPArray\AsLPArrayTest\AsLPArrayTest.cmd
Interop\PInvoke\SizeParamIndex\PInvoke\PassingByRef\PassingByRefTest\PassingByRefTest.cmd
Interop\StringMarshalling\UTF8\UTF8Test\UTF8Test.cmd
Interop\StructPacking\StructPacking\StructPacking.cmd
JIT\HardwareIntrinsics\General\Vector128_1\Vector128_1_ro\Vector128_1_ro.cmd
JIT\HardwareIntrinsics\General\Vector128_1\Vector128_1_r\Vector128_1_r.cmd
JIT\HardwareIntrinsics\General\Vector256_1\Vector256_1_ro\Vector256_1_ro.cmd
JIT\Regression\Dev11\External\dev11_239804\ShowLocallocAlignment\ShowLocallocAlignment.cmd
CoreMangLib\system\runtime\interopservices\marshal\MarshalSizeOf1_PSC\MarshalSizeOf1_PSC.cmd
Regressions\coreclr\0584\Test584\Test584.cmd
Regressions\coreclr\0570\Test570\Test570.cmd

There are various failure modes (classic GCStress failures), e.g.:

Assert failure(PID 25956 [0x00006564], Thread: 10512 [0x2910]): SanityCheck()

CORECLR! MethodTable::Validate + 0x3F (0x00007ffa`5e9eb52b)
CORECLR! Object::ValidateInner + 0xF1 (0x00007ffa`5e92fe31)
CORECLR! Object::Validate + 0x149 (0x00007ffa`5e92fcf9)
CORECLR! WKS::GCHeap::Promote + 0x9F (0x00007ffa`5ed90bff)
CORECLR! PromoteCarefully + 0xAB (0x00007ffa`5e95b84b)
CORECLR! TransitionFrame::PromoteCallerStackUsingGCRefMap + 0x132 (0x00007ffa`5e8a6412)
CORECLR! ExternalMethodFrame::GcScanRoots + 0xE0 (0x00007ffa`5e8a3400)
CORECLR! GcStackCrawlCallBack + 0x37A (0x00007ffa`5ec9038a)
CORECLR! Thread::MakeStackwalkerCallback + 0x52 (0x00007ffa`5e6c9d82)
CORECLR! Thread::StackWalkFramesEx + 0x17A (0x00007ffa`5e6cbe82)
    File: F:\gh\runtime\src\coreclr\src\vm\methodtable.cpp Line: 9901
    Image: F:\gh\runtime\artifacts\tests\coreclr\Windows_NT.x64.Checked\Tests\Core_Root\CoreRun.exe

Assert failure(PID 24136 [0x00005e48], Thread: 26188 [0x664c]): !CREATE_CHECK_STRING(pMT && pMT->Validate())

CORECLR! Object::ValidateInner + 0x10D (0x00007ffa`5e92fe4d)
CORECLR! Object::Validate + 0x149 (0x00007ffa`5e92fcf9)
CORECLR! GcInfoDecoder::ReportRegisterToGC + 0x12A (0x00007ffa`5ed4938e)
CORECLR! GcInfoDecoder::ReportSlotToGC + 0xCD (0x00007ffa`5ed495b9)
CORECLR! GcInfoDecoder::EnumerateLiveSlots + 0xCEE (0x00007ffa`5ed487be)
CORECLR! EECodeManager::EnumGcRefs + 0x39A (0x00007ffa`5e9a27aa)
CORECLR! GcStackCrawlCallBack + 0x2CD (0x00007ffa`5ec902dd)
CORECLR! Thread::MakeStackwalkerCallback + 0x52 (0x00007ffa`5e6c9d82)
CORECLR! Thread::StackWalkFramesEx + 0x17A (0x00007ffa`5e6cbe82)
CORECLR! Thread::StackWalkFrames + 0x194 (0x00007ffa`5e6cbc68)
    File: F:\gh\runtime\src\coreclr\src\vm\object.cpp Line: 557
    Image: F:\gh\runtime\artifacts\tests\coreclr\Windows_NT.x64.Checked\Tests\Core_Root\CoreRun.exe

Assert failure(PID 27972 [0x00006d44], Thread: 27564 [0x6bac]): (GetComponentSize() <= 2) || IsArray()

CORECLR! MethodTable::SanityCheck + 0x49 (0x00007ffa`5e9e9a59)
CORECLR! MethodTable::Validate + 0x17 (0x00007ffa`5e9eb503)
CORECLR! Object::ValidateInner + 0xF1 (0x00007ffa`5e92fe31)
CORECLR! Object::Validate + 0x149 (0x00007ffa`5e92fcf9)
CORECLR! GcInfoDecoder::ReportRegisterToGC + 0x12A (0x00007ffa`5ed4938e)
CORECLR! GcInfoDecoder::ReportSlotToGC + 0xCD (0x00007ffa`5ed495b9)
CORECLR! GcInfoDecoder::EnumerateLiveSlots + 0xCEE (0x00007ffa`5ed487be)
CORECLR! EECodeManager::EnumGcRefs + 0x39A (0x00007ffa`5e9a27aa)
CORECLR! GcStackCrawlCallBack + 0x2CD (0x00007ffa`5ec902dd)
CORECLR! Thread::MakeStackwalkerCallback + 0x52 (0x00007ffa`5e6c9d82)
    File: F:\gh\runtime\src\coreclr\src\vm\methodtable.cpp Line: 7916
    Image: F:\gh\runtime\artifacts\tests\coreclr\Windows_NT.x64.Checked\Tests\Core_Root\CoreRun.exe

[BruceForstall]

@dotnet/jit-contrib

[BruceForstall]

Windows x86 and Linux x64 have similar-looking failures.

AzDO "runtime-coreclr gcstress-extra" results:
https://dev.azure.com/dnceng/public/_build/results?buildId=490095&view=ms.vss-test-web.build-test-results-tab

[AndyAyersMS]

I'll investigate.

[AndyAyersMS]

Have been looking at the ShowLocallocAlignment failure, and think I have some pieces of the puzzle.

Via jitstress method isolation the failing method is System.Runtime.InteropServices.OSPlatform:get_Windows
via stress mode isolation (with inlining held constant, see #1983) this fails with STRESS_GENERIC_VARN and STRESS_PROFILER_CALLBACKS.

Here VARN forces a minopts fallback, and so we end up with a fully interruptible method, and the method returns a GC ref, which must be kept live across the profiler leave callback. There is code in the jit to do just this.

However there's also a nop after that callback (added by emitOutputPreEpilogNOP) and the return value liveness seems to end prematurely at this nop instead of reaching the epilog. Under GcStress a GC is triggered at the nop and during that GC, we don't update RAX.

G_M47661_IG03:
00004F 488B45F0             mov      rax, gword ptr [rbp-10H]   -RAX
000053 48B9303CB606FD7F0000 mov      rcx, 0x7FFD06B63C30        +RAX
00005D 488D5510             lea      rdx, [rbp+10H]
000061 FF1539D7FA5F         call     [CORINFO_HELP_PROF_FCN_LEAVE]  
000067 90                   nop                                       // GC Here, RAX dead

G_M47661_IG04:
000068 488D6500             lea      rsp, [rbp]   -RAX
00006C 5D                   pop      rbp
00006D C3                   ret


Stack slot id for offset -16 (0xfffffff0) (frame) (untracked) = 0.
Register slot id for reg rax = 1.
Set state of slot 1 at instr offset 0x45 to Live.
Set state of slot 1 at instr offset 0x4f to Dead.
Set state of slot 1 at instr offset 0x53 to Live.
Set state of slot 1 at instr offset 0x67 to Dead.
Defining interruptible range: [0x24, 0x68).

Not updating RAX leads to asserts later since the return value from the method is now bogus. Seems like the fix is to extend the gc liveness all the way to the epilog.

Odd that this has not shown up in past testing, since the leave callback will seemingly always inspire a nop like this. I suppose it would require gcstress; normally we'd be very unlikely to GC there.

[AndyAyersMS]

Ah, think I figured the rest out -- the code that tries to extend the return value lifetime doesn't trigger here. The method is returning a struct-wrapped object ref, and so the jit needs to be careful to check the "native return type" for GCness, not the actual return type. Might explain the rareness of this bug, we need the right return type in addition to minopts and the profiler exit hook.

[AndyAyersMS]

Prospective fix works for that test, trying others...

[AndyAyersMS]

Seems to fix all the examples above except the HW intrinsics tests. Will put up a PR.