Test failure : JIT/Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.sh #48892

VincentBu · 2021-03-01T02:14:55Z

Run: runtime-coreclr jitstress2-jitstressregs 20210228.1

Failed test:

CoreCLR Linux arm Checked jitstress2_jitstressregs3 @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7-bfcd90a-20200121150440
 
-JIT/Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.sh

Error message:

/root/helix/work/workitem/JIT/Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.sh: line 388: 1648 Segmentation fault (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"

Return code: 1
Raw output file: /root/helix/work/workitem/JIT/Methodical/Reports/JIT.Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun 10w5d_cs_ro.dll ''
Func_0_6_5_6_3: != true
Func_0_6_5_4_6: <= true
Func_0_6_5_4_5: > true
Func_0_6_5_4_3: == false
Func_0_6_4_6_2: != true
Func_0_6_4_5_4: < true
Func_0_6_4_5_1: < true
Func_0_6_4_4: <= true
Func_0_6_3_6_5: == false
Func_0_6_3_4_4: <= true
Func_0_6_3_4_3: < true
Func_0_6_3_3: < true
Func_0_6_3_2_3: >= true
Func_0_6_3_1_5: < true
Func_0_6_3_1: <= false
Func_0_6_3: != true
Func_0_6_2_5: != true
Func_0_6_2_1: > false
Func_0_6_1_6_4: < false
Func_0_6_1_6_3: > false
Func_0_6_1_5_4: > true
Func_0_6_1_3: == false
Func_0_5_6_6_6: <= true
Func_0_5_6_6_1: > true
Func_0_5_6_4_6: <= false
Func_0_5_6_4_2: >= false
Func_0_5_6_3_6: < false
Func_0_5_6_3_1: == false
Func_0_5_6_3: <= true
Func_0_5_6_2_1: >= false
Func_0_5_6_1_3: > true
Func_0_5_6_1: <= false
Func_0_5_5_6: != true
Func_0_5_5_5_6: != true
Func_0_5_5_5_4: == false
Func_0_5_5_5_2: == false
Func_0_5_5_4_5: < true
Func_0_5_5_4: > true
Func_0_5_5_3_4: != true
Func_0_5_5_3_2: <= true
Func_0_5_5_3_1: > true
Func_0_5_5_1: < false
Func_0_5_4_5: >= false
Func_0_5_4_1_3: <= false
Func_0_5_4_1_2: == false
Func_0_5_4: != true
Func_0_5_3_6: != true
Func_0_5_3_5_3: < true
Func_0_5_3_4_2: == false
Func_0_5_3_4_1: >= true
Func_0_5_3_4: <= true
Func_0_5_3_1_2: < true
Func_0_5_3: >= false
Func_0_5_2_6_6: != true
Func_0_5_2_6_3: != true
Func_0_5_2_5_6: != true
Func_0_5_2_5_4: == false
Func_0_5_2_5_3: <= true
Func_0_5_2_5_2: != true
Func_0_5_2_4_5: > false
Func_0_5_2_4_2: <= true
Func_0_5_2_2_5: > true
Func_0_5_2_1_6: > false
Func_0_5_2_1_1: != true
Func_0_5_1: != true
Func_0_4_6_6_5: <= true
Func_0_4_6_6: == false
Func_0_4_6_5_6: <= true
Func_0_4_6_5_5: != true
Func_0_4_6_5_2: <= true
Func_0_4_6_5: == false
Func_0_4_6_3_6: >= true
Func_0_4_6_3_3: == false
Func_0_4_6_3_2: <= false
Func_0_4_6_2: == false
Func_0_4_5_6: >= false
Func_0_4_5_5: < true
Func_0_4_5_3_5: == false
Func_0_4_5_2_6: > false
Func_0_4_5_2_4: >= true
Func_0_4_5_2_1: <= false
Func_0_4_5_2: >= true
Func_0_4_5_1_5: == false
Func_0_4_5_1_2: > true
Func_0_4_4_5: != true
Func_0_4_4_2_6: < true
Func_0_4_4_2_5: != false
Func_0_4_4_2_3: <= true
Func_0_4_4_2: == false
Func_0_4_4_1_4: >= false
Func_0_4_3_6_6: <= true
Func_0_4_3_6_5: <= true
Func_0_4_3_6_2: >= true
Func_0_4_3_6_1: == true
Func_0_4_3_4: >= true
Func_0_4_3_2_2: == false
Func_0_4_3_1_3: > true
Func_0_4_3_1_1: != true
Func_0_4_2_5_4: <= true
Func_0_4_2_5_1: != true
Func_0_4_1_4: >= true
Func_0_4_1_3_1: >= true
Func_0_4_1_2_3: < false
Func_0_4_1_1_1: >= true
Func_0_3_6_6_5: >= true
Func_0_3_6_6_1: < true
Func_0_3_6_5_5: > false
Func_0_3_6_4_1: < false
Func_0_3_6_3_4: != true
Func_0_3_5_6_6: == true
Func_0_3_5_6_5: <= true
Func_0_3_5_6_2: != true
Func_0_3_5_4: == false
Func_0_3_5_3: > false
Func_0_3_5_2_6: != true
Func_0_3_5_2_5: <= true
Func_0_3_5_2_4: > false
Func_0_3_5_2_2: == false
Func_0_3_5_1_1: != true
Func_0_3_4_1_1: < true
Func_0_3_4_1: < true
Func_0_3_3_6_4: <= false
Func_0_3_3_6_1: <= true
Func_0_3_3_5_6: != true
Func_0_3_3_5_1: != true
Func_0_3_3_4_6: <= true
Func_0_3_3_4_5: <= true
Func_0_3_3_4_2: < true
Func_0_3_2_6_6: <= true
Func_0_3_2_6_3: >= false
Func_0_3_2_6_1: >= true
Func_0_3_2_5_6: < true
Func_0_3_2_5_3: >= true
Func_0_3_2_5_1: > true
Func_0_3_2_4_3: > true
Func_0_3_2_3_3: > true
Func_0_3_2_3_2: <= false
Func_0_3_2_1_5: != true
Func_0_3_1_6_6: > true
Func_0_3_1_6_4: >= false
Func_0_3_1_4_4: <= true
Func_0_3_1_4_3: == true
Func_0_3_1_4_2: == false
Func_0_3_1_4_1: <= true
Func_0_3_1_3_5: == false
Func_0

Stack trace
   at JIT_Methodical._fp_exgen_10w5d_cs_ro_10w5d_cs_ro_._fp_exgen_10w5d_cs_ro_10w5d_cs_ro_sh()

The text was updated successfully, but these errors were encountered:

BruceForstall · 2021-03-01T05:30:22Z

@dotnet/jit-contrib related to #44419?

BruceForstall · 2021-03-05T01:07:56Z

This is still failing: https://dev.azure.com/dnceng/public/_build/results?buildId=1023241&view=ms.vss-test-web.build-test-results-tab&runId=31831088&resultId=103648&paneView=debug

JulieLeeMSFT · 2021-03-11T18:24:31Z

@briansull PTAL

BruceForstall · 2021-03-15T21:59:11Z

This is failing every day for Linux arm32 (jitstress=2, jitstressregs=3). @briansull Could you please disable this test for Linux arm32 until it can be investigated?

BruceForstall · 2021-03-19T16:41:34Z

@briansull ping

VincentBu · 2021-03-22T02:44:25Z

Failed again in runtime-coreclr jitstress2-jitstressregs 20210321.1

Failed test:

CoreCLR Linux arm Checked jitstress2_jitstressregs3 @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7-bfcd90a-20200121150440
 -JIT/Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.sh

Error message:

/root/helix/work/workitem/JIT/Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.sh: line 375: 2861 Segmentation fault (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"

Return code: 1
Raw output file: /root/helix/work/workitem/JIT/Methodical/Reports/JIT.Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun 10w5d_cs_ro.dll ''
Func_0_6_5_6_3: != true
Func_0_6_5_4_6: <= true
Func_0_6_5_4_5: > true
Func_0_6_5_4_3: == false
Func_0_6_4_6_2: != true
Func_0_6_4_5_4: < true
Func_0_6_4_5_1: < true
Func_0_6_4_4: <= true
Func_0_6_3_6_5: == false
Func_0_6_3_4_4: <= true
Func_0_6_3_4_3: < true
Func_0_6_3_3: < true
Func_0_6_3_2_3: >= true
Func_0_6_3_1_5: < true
Func_0_6_3_1: <= false
Func_0_6_3: != true
Func_0_6_2_5: != true
Func_0_6_2_1: > false
Func_0_6_1_6_4: < false
Func_0_6_1_6_3: > false
Func_0_6_1_5_4: > true
Func_0_6_1_3: == false
Func_0_5_6_6_6: <= true
Func_0_5_6_6_1: > true
Func_0_5_6_4_6: <= false
Func_0_5_6_4_2: >= false
Func_0_5_6_3_6: < false
Func_0_5_6_3_1: == false
Func_0_5_6_3: <= true
Func_0_5_6_2_1: >= false
Func_0_5_6_1_3: > true
Func_0_5_6_1: <= false
Func_0_5_5_6: != true
Func_0_5_5_5_6: != true
Func_0_5_5_5_4: == false
Func_0_5_5_5_2: == false
Func_0_5_5_4_5: < true
Func_0_5_5_4: > true
Func_0_5_5_3_4: != true
Func_0_5_5_3_2: <= true
Func_0_5_5_3_1: > true
Func_0_5_5_1: < false
Func_0_5_4_5: >= false
Func_0_5_4_1_3: <= false
Func_0_5_4_1_2: == false
Func_0_5_4: != true
Func_0_5_3_6: != true
Func_0_5_3_5_3: < true
Func_0_5_3_4_2: == false
Func_0_5_3_4_1: >= true
Func_0_5_3_4: <= true
Func_0_5_3_1_2: < true
Func_0_5_3: >= false
Func_0_5_2_6_6: != true
Func_0_5_2_6_3: != true
Func_0_5_2_5_6: != true
Func_0_5_2_5_4: == false
Func_0_5_2_5_3: <= true
Func_0_5_2_5_2: != true
Func_0_5_2_4_5: > false
Func_0_5_2_4_2: <= true
Func_0_5_2_2_5: > true
Func_0_5_2_1_6: > false
Func_0_5_2_1_1: != true
Func_0_5_1: != true
Func_0_4_6_6_5: <= true
Func_0_4_6_6: == false
Func_0_4_6_5_6: <= true
Func_0_4_6_5_5: != true
Func_0_4_6_5_2: <= true
Func_0_4_6_5: == false
Func_0_4_6_3_6: >= true
Func_0_4_6_3_3: == false
Func_0_4_6_3_2: <= false
Func_0_4_6_2: == false
Func_0_4_5_6: >= false
Func_0_4_5_5: < true
Func_0_4_5_3_5: == false
Func_0_4_5_2_6: > false
Func_0_4_5_2_4: >= true
Func_0_4_5_2_1: <= false
Func_0_4_5_2: >= true
Func_0_4_5_1_5: == false
Func_0_4_5_1_2: > true
Func_0_4_4_5: != true
Func_0_4_4_2_6: < true
Func_0_4_4_2_5: != false
Func_0_4_4_2_3: <= true
Func_0_4_4_2: == false
Func_0_4_4_1_4: >= false
Func_0_4_3_6_6: <= true
Func_0_4_3_6_5: <= true
Func_0_4_3_6_2: >= true
Func_0_4_3_6_1: == true
Func_0_4_3_4: >= true
Func_0_4_3_2_2: == false
Func_0_4_3_1_3: > true
Func_0_4_3_1_1: != true
Func_0_4_2_5_4: <= true
Func_0_4_2_5_1: != true
Func_0_4_1_4: >= true
Func_0_4_1_3_1: >= true
Func_0_4_1_2_3: < false
Func_0_4_1_1_1: >= true
Func_0_3_6_6_5: >= true
Func_0_3_6_6_1: < true
Func_0_3_6_5_5: > false
Func_0_3_6_4_1: < false
Func_0_3_6_3_4: != true
Func_0_3_5_6_6: == true
Func_0_3_5_6_5: <= true
Func_0_3_5_6_2: != true
Func_0_3_5_4: == false
Func_0_3_5_3: > false
Func_0_3_5_2_6: != true
Func_0_3_5_2_5: <= true
Func_0_3_5_2_4: > false
Func_0_3_5_2_2: == false
Func_0_3_5_1_1: != true
Func_0_3_4_1_1: < true
Func_0_3_4_1: < true
Func_0_3_3_6_4: <= false
Func_0_3_3_6_1: <= true
Func_0_3_3_5_6: != true
Func_0_3_3_5_1: != true
Func_0_3_3_4_6: <= true
Func_0_3_3_4_5: <= true
Func_0_3_3_4_2: < true
Func_0_3_2_6_6: <= true
Func_0_3_2_6_3: >= false
Func_0_3_2_6_1: >= true
Func_0_3_2_5_6: < true
Func_0_3_2_5_3: >= true
Func_0_3_2_5_1: > true
Func_0_3_2_4_3: > true
Func_0_3_2_3_3: > true
Func_0_3_2_3_2: <= false
Func_0_3_2_1_5: != true
Func_0_3_1_6_6: > true
Func_0_3_1_6_4: >= false
Func_0_3_1_4_4: <= true
Func_0_3_1_4_3: == true
Func_0_3_1_4_2: == false
Func_0_3_1_4_1: <= true
Func_0_3_1_3_5: == false
Func_0

Stack trace
   at JIT_Methodical._fp_exgen_10w5d_cs_ro_10w5d_cs_ro_._fp_exgen_10w5d_cs_ro_10w5d_cs_ro_sh()

Currently failing in jitstress2_jitstressregs3 configuration. Issue: dotnet#48892

Currently failing in jitstress2_jitstressregs3 configuration. Issue: #48892

AndyAyersMS · 2021-03-30T18:39:58Z

Repros at e830f10. The jit is crashing during LSRA while jitting testout1.Func_0_2_3_4().

Last bit of the jit log shows:

RESOLVING EDGES
fgFindInsertPoint(regionIndex=0, putInTryRegion=true, startBlk=BB01, endBlk=BB00, nearBlk=BB12, jumpBlk=BB00, runRarely=false)
fgNewBBinRegion(jumpKind=6, tryIndex=0, hndIndex=0, putInFilter=false, runRarely=false, insertAtEnd=false): inserting after BB15
New Basic Block BB32 [0051] created.
Splitting edge from BB12 to BB15; adding BB32
Setting edge weights for BB12 -> BB32 to [0 .. 3.402823e+38]
   BB32 bottom: move V44 from STK to f16 (Critical)
fgFindInsertPoint(regionIndex=0, putInTryRegion=true, startBlk=BB01, endBlk=BB00, nearBlk=BB28, jumpBlk=BB00, runRarely=true)
fgNewBBinRegion(jumpKind=6, tryIndex=0, hndIndex=0, putInFilter=false, runRarely=true, insertAtEnd=false): inserting after BB29
New Basic Block BB33 [0052] created.
Splitting edge from BB28 to BB15; adding BB33
Setting edge weights for BB28 -> BB33 to [0 .. 3.402823e+38]
   BB33 bottom: move V44 from f2 to f16 (Critical)
   BB21 bottom: move V44 from STK to f16 (Join)
   BB22 top: move V44 from f2 to STK (Split)
   BB22 bottom: move V44 from STK to f16 (Join)
   BB29 bottom: move V51 from f16 to f0 (Split)
   BB29 bottom: move V44 from f2 to STK (Split)
   BB29 bottom: move V45 from f17 to f2 (Split)

Seems like interval is invalid here:

    insertMove(block, insertionPoint, interval->varNum, fromReg, toReg);

Stack backtrace:

Process 245 stopped
* thread #1, name = 'corerun', stop reason = signal SIGSEGV: invalid address (fault address: 0x23)
    frame #0: 0xf087e818 libclrjit.so`LinearScan::addResolution(this=0x00eded54, block=0x00ebbcc8, insertionPoint=0x00000000, interval=0x00000003, toReg=REG_F17, fromReg=204) at lsra.cpp:8229
(lldb) thread backtrace
* thread #1, name = 'corerun', stop reason = signal SIGSEGV: invalid address (fault address: 0x23)
  * frame #0: 0xf087e818 libclrjit.so`LinearScan::addResolution(this=0x00eded54, block=0x00ebbcc8, insertionPoint=0x00000000, interval=0x00000003, toReg=REG_F17, fromReg=204) at lsra.cpp:8229
    frame #1: 0xf088016e libclrjit.so`LinearScan::resolveEdge(this=<unavailable>, fromBlock=<unavailable>, toBlock=<unavailable>, resolveType=<unavailable>, liveSet=0xfffecc3c) at lsra.cpp:9031
    frame #2: 0xf087bf60 libclrjit.so`LinearScan::resolveEdges(this=<unavailable>) at lsra.cpp:8631
    frame #3: 0xf08718b4 libclrjit.so`LinearScan::resolveRegisters(this=0x00eded54) at lsra.cpp:7702
    frame #4: 0xf086cc74 libclrjit.so`LinearScan::doLinearScan(this=0x00eded54) at lsra.cpp:1280
    frame #5: 0xf079b1cc libclrjit.so`ActionPhase<Compiler::compCompile(void**, unsigned int*, JitFlags*)::$_10>::DoPhase() [inlined] Compiler::compCompile(this=<unavailable>)::$_10::operator()() const at compiler.cpp:5172
    frame #6: 0xf079b1c2 libclrjit.so`ActionPhase<Compiler::compCompile(void**, unsigned int*, JitFlags*)::$_10>::DoPhase(this=<unavailable>) at phase.h:64
    frame #7: 0xf08cb3f6 libclrjit.so`Phase::Run(this=0xfffecd98) at phase.cpp:61
    frame #8: 0xf0795a7a libclrjit.so`Compiler::compCompile(void**, unsigned int*, JitFlags*) [inlined] void DoPhase<Compiler::compCompile(void**, unsigned int*, JitFlags*)::$_10>(_compiler=<unavailable>, _phase=<unavailable>, _action=<unavailable>)::$_10) at phase.h:78
    frame #9: 0xf0795a5e libclrjit.so`Compiler::compCompile(this=0x00fba1dc, methodCodePtr=<unavailable>, methodCodeSize=<unavailable>, compileFlags=<unavailable>) at compiler.cpp:5173

Will see if this repros using an altjit.

AndyAyersMS · 2021-03-30T19:16:53Z

Yes, repros with

set COMPlus_AltJitName=clrjit_unix_arm_x86.dll
set COMPlus_AltJit=Func_0_2_3_4
set COMPlus_JitStress=2
set COMPlus_JitStressRegs=3
set COMPlus_TieredCompilation=0

on checked windows x86:

Assert failure(PID 158136 [0x000269b8], Thread: 159408 [0x26eb0]): Assertion failed 'interval != nullptr' in 'testout1:Func_0_2_3_4():double' during 'LSRA allocate' (IL size 362)

    File: C:\repos\runtime2\src\coreclr\jit\lsra.cpp Line: 9030
    Image: C:\repos\runtime2\artifacts\tests\coreclr\windows.x86.Checked\Tests\Core_Root\corerun.exe

@kunalspathak do you want me to keep digging in, or do you want to take this one over?

kunalspathak · 2021-03-30T19:18:42Z

Thanks @AndyAyersMS for getting minimal repro. I can look into it.

kunalspathak · 2021-04-07T00:22:11Z

I was able to repro the issue on e830f10 but it doesn't repro on main. I will investigate further.

VincentBu added arch-arm32 os-linux Linux OS (any supported distro) JitStress CLR JIT issues involving JIT internal stress modes labels Mar 1, 2021

dotnet-issue-labeler bot added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI untriaged New issue has not been triaged by the area owner labels Mar 1, 2021

BruceForstall added this to the 6.0.0 milestone Mar 1, 2021

BruceForstall removed the untriaged New issue has not been triaged by the area owner label Mar 1, 2021

JulieLeeMSFT assigned briansull Mar 11, 2021

JulieLeeMSFT added this to Needs Triage in .NET Core CodeGen via automation Mar 11, 2021

JulieLeeMSFT moved this from Needs Triage to P1 in .NET Core CodeGen Mar 11, 2021

BruceForstall added a commit to BruceForstall/runtime that referenced this issue Mar 22, 2021

Disable JIT/Methodical/fp/exgen/10w5d_cs_ro for Linux/arm32

1485aa8

Currently failing in jitstress2_jitstressregs3 configuration. Issue: dotnet#48892

BruceForstall mentioned this issue Mar 22, 2021

Disable JIT/Methodical/fp/exgen/10w5d_cs_ro for Linux/arm32 #50056

Merged

BruceForstall added a commit that referenced this issue Mar 23, 2021

Disable JIT/Methodical/fp/exgen/10w5d_cs_ro for Linux/arm32 (#50056)

8722cda

Currently failing in jitstress2_jitstressregs3 configuration. Issue: #48892

JulieLeeMSFT added the Priority:0 Work that we can't release without label Mar 29, 2021

JulieLeeMSFT assigned AndyAyersMS and unassigned briansull Mar 29, 2021

kunalspathak assigned kunalspathak and unassigned AndyAyersMS Mar 30, 2021

JulieLeeMSFT added the preview label Apr 1, 2021

kunalspathak mentioned this issue Apr 7, 2021

Fix resolution code for scenarios where values are copied from stack #50890

Merged

ghost added the in-pr There is an active PR which will close this issue when it is merged label Apr 7, 2021

kunalspathak closed this as completed in #50890 Apr 14, 2021

.NET Core CodeGen automation moved this from P1 to Done Apr 14, 2021

ghost locked as resolved and limited conversation to collaborators May 14, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test failure : JIT/Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.sh #48892

Test failure : JIT/Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.sh #48892

VincentBu commented Mar 1, 2021

BruceForstall commented Mar 1, 2021

BruceForstall commented Mar 5, 2021

JulieLeeMSFT commented Mar 11, 2021

BruceForstall commented Mar 15, 2021

BruceForstall commented Mar 19, 2021

VincentBu commented Mar 22, 2021

AndyAyersMS commented Mar 30, 2021

AndyAyersMS commented Mar 30, 2021

kunalspathak commented Mar 30, 2021 •

edited

Loading

kunalspathak commented Apr 7, 2021

Test failure : JIT/Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.sh #48892

Test failure : JIT/Methodical/fp/exgen/10w5d_cs_ro/10w5d_cs_ro.sh #48892

Comments

VincentBu commented Mar 1, 2021

BruceForstall commented Mar 1, 2021

BruceForstall commented Mar 5, 2021

JulieLeeMSFT commented Mar 11, 2021

BruceForstall commented Mar 15, 2021

BruceForstall commented Mar 19, 2021

VincentBu commented Mar 22, 2021

AndyAyersMS commented Mar 30, 2021

AndyAyersMS commented Mar 30, 2021

kunalspathak commented Mar 30, 2021 • edited Loading

kunalspathak commented Apr 7, 2021

kunalspathak commented Mar 30, 2021 •

edited

Loading