AVX512 miscompile during type legalization of v4i1 = setcc v4i64, v4i64

[llvmbot]

Bugzilla Link	34177
Resolution	FIXED
Resolved on	Aug 21, 2017 16:28
Version	trunk
OS	Windows NT
Blocks	#33196
Reporter	LLVM Bugzilla Contributor
CC	@zmodem,@RKSimon,@ZviRackover

Extended Description

$ cat /tmp/tst.ll

target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

define void @​test() local_unnamed_addr {
%1 = icmp eq <4 x i64> <i64 0, i64 1, i64 2, i64 3>, undef
%2 = select <4 x i1> %1, <4 x x86_fp80> <x86_fp80 0xK3FFF8000000000000000, x86_fp80 0xK3FFF8000000000000000, x86_fp80 0xK3FFF8000000000000000, x86_fp80 0xK3FFF8000000000000000>, <4 x x86_fp80> zeroinitializer
%3 = fadd <4 x x86_fp80> undef, %2
%4 = shufflevector <4 x x86_fp80> %3, <4 x x86_fp80> undef, <8 x i32> <i32 0, i32 4, i32 1, i32 5, i32 2, i32 6, i32 3, i32 7>
store <8 x x86_fp80> %4, <8 x x86_fp80>* undef, align 16
unreachable
}

This was minimized from a miscompile in the Eigen test-suite (test/matrix_power.cpp). (similar to #32696 )

llc -mcpu=haswell does not crash.

llc -mcpu=skx hits an assertion:

ScalarizeVectorOperand Op #​0: t47: v1i1 = setcc t44, undef:v1i64, seteq:ch

Do not know how to scalarize this operator's operand!
UNREACHABLE executed at ../lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:464!
#​0 0x00007f04b9064f89 llvm::sys::PrintStackTrace(llvm::raw_ostream&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/Support/Unix/Signals.inc:398:11
#​1 0x00007f04b9065139 PrintStackTraceSignalHandler(void*) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/Support/Unix/Signals.inc:462:1
#​2 0x00007f04b90637a3 llvm::sys::RunSignalHandlers() /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/Support/Signals.cpp:0:5
#​3 0x00007f04b9065494 SignalHandler(int) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/Support/Unix/Signals.inc:252:1
#​4 0x00007f04b80c3100 __restore_rt (/lib64/libpthread.so.0+0xf100)
#​5 0x00007f04b75075f7 __GI_raise (/lib64/libc.so.6+0x355f7)
#​6 0x00007f04b7508ce8 __GI_abort (/lib64/libc.so.6+0x36ce8)
#​7 0x00007f04b8f82a60 LLVMInstallFatalErrorHandler /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/Support/ErrorHandling.cpp:204:0
#​8 0x00007f04b9508945 llvm::DAGTypeLegalizer::ScalarizeVectorOperand(llvm::SDNode*, unsigned int) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp:466:36
#​9 0x00007f04b94e8519 llvm::DAGTypeLegalizer::run() /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/CodeGen/SelectionDAG/LegalizeTypes.cpp:314:26
#​10 0x00007f04b94edb6e llvm::SelectionDAG::LegalizeTypes() /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/CodeGen/SelectionDAG/LegalizeTypes.cpp:1171:34
#​11 0x00007f04b9648b14 llvm::SelectionDAGISel::CodeGenAndEmitDAG() /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:759:23
#​12 0x00007f04b9648210 llvm::SelectionDAGISel::SelectBasicBlock(llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Instruction, true, false, void>, false, true>, llvm::ilist_iterator<llvm::ilist_detail::node_options<llvm::Instruction, true, false, void>, false, true>, bool&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:666:1
#​13 0x00007f04b9647f15 llvm::SelectionDAGISel::SelectAllBasicBlocks(llvm::Function const&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:0:7
#​14 0x00007f04b96451f9 llvm::SelectionDAGISel::runOnMachineFunction(llvm::MachineFunction&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp:467:22
#​15 0x00007f04be19ab7b (anonymous namespace)::X86DAGToDAGISel::runOnMachineFunction(llvm::MachineFunction&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/Target/X86/X86ISelDAGToDAG.cpp:177:25
#​16 0x00007f04bb77d801 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/CodeGen/MachineFunctionPass.cpp:62:8
#​17 0x00007f04bade8ddf llvm::FPPassManager::runOnFunction(llvm::Function&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/IR/LegacyPassManager.cpp:1514:27
#​18 0x00007f04bade90f5 llvm::FPPassManager::runOnModule(llvm::Module&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/IR/LegacyPassManager.cpp:1535:16
#​19 0x00007f04bade988a (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/IR/LegacyPassManager.cpp:1591:27
#​20 0x00007f04bade93b6 llvm::legacy::PassManagerImpl::run(llvm::Module&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/IR/LegacyPassManager.cpp:1694:16
#​21 0x00007f04bade9da1 llvm::legacy::PassManager::run(llvm::Module&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../lib/IR/LegacyPassManager.cpp:1725:3
#​22 0x000000000041e977 compileModule(char**, llvm::LLVMContext&) /export/iusers/cohenela/git_workspaces/playground/llvm/build/../tools/llc/llc.cpp:567:42
#​23 0x000000000041cedc main /export/iusers/cohenela/git_workspaces/playground/llvm/build/../tools/llc/llc.cpp:342:13
#​24 0x00007f04b74f3b15 __libc_start_main (/lib64/libc.so.6+0x21b15)
#​25 0x000000000041c6d5 _start (./bin/llc+0x41c6d5)
Stack dump:
0. Program arguments: ./bin/llc -mcpu=skx /tmp/tst.ll

1. Running pass 'Function Pass Manager' on module '/tmp/tst.ll'.
2. Running pass 'X86 DAG->DAG Instruction Selection' on function '@test'

Essentially, IIUC what seems to be happening is that a "v4i1 setcc v4i64 v4i64" is being scalarized and broken down until it reaches "v1i1 setcc v1i64 v1i64" - at this point the result of this value is not being scalarized since v1i1 is legal on AVX512 - but the legalizer tries to scalarize the operands where it fails since there is no scalarizeVecOp_SETCC.

[RKSimon]

https://reviews.llvm.org/D36651

[llvmbot]

Thanks! I forgot to add the link ^ to the proposed fix patch - Sorry for that.

[RKSimon]

rL311071 - worth merging into 5.00 or not?

[llvmbot]

I'll check if it passed on 4.0

[llvmbot]

It passed with 4.0, so I guess we should have it merged to 5.0.

Adding Hans, and marking as a blocker.

[zmodem]

It passed with 4.0, so I guess we should have it merged to 5.0.

Adding Hans, and marking as a blocker.

Sounds good to me. Let's have it sit in trunk a little longer, and then I'll merge.

[zmodem]

It passed with 4.0, so I guess we should have it merged to 5.0.

Adding Hans, and marking as a blocker.

Sounds good to me. Let's have it sit in trunk a little longer, and then I'll
merge.

Merged in r311409.