[SelectionDAG] Do not crash on large integers in CheckInteger

[uweigand]

The CheckInteger routine called from TableGen-generated selection logic uses getSExtValue - which will abort if the underlying APInt does not fit into an int64_t.

This case is now triggered by the SystemZ back-end since i128 is a legal type on certain machines. While we do not have any regular instructions that take 128-bit immediates (like most other platforms), there are patterns in the .td files that recognize an i128 "xor ..., -1" as a "not".

These patterns cause code to be generated that calls the CheckInteger routine on some i128-valued integer, which may trigger the assert.

Fix by using trySExtValue instead.

Fixes #75710

[llvmbot]

@llvm/pr-subscribers-llvm-selectiondag

Author: Ulrich Weigand (uweigand)

Changes

The CheckInteger routine called from TableGen-generated selection logic uses getSExtValue - which will abort if the underlying APInt does not fit into an int64_t.

This case is now triggered by the SystemZ back-end since i128 is a legal type on certain machines. While we do not have any regular instructions that take 128-bit immediates (like most other platforms), there are patterns in the .td files that recognize an i128 "xor ..., -1" as a "not".

These patterns cause code to be generated that calls the CheckInteger routine on some i128-valued integer, which may trigger the assert.

Fix by using trySExtValue instead.

Fixes #75710

---

Full diff: https://github.com/llvm/llvm-project/pull/75787.diff

2 Files Affected:

• (modified) llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp (+1-1)
• (modified) llvm/test/CodeGen/SystemZ/xor-09.ll (+14)

diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp
index a1cf4cbbee1b85..af49ef17a3f2dd 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGISel.cpp
@@ -2786,7 +2786,7 @@ CheckInteger(const unsigned char *MatcherTable, unsigned &MatcherIndex,
   Val = decodeSignRotatedValue(Val);
 
   ConstantSDNode *C = dyn_cast<ConstantSDNode>(N);
-  return C && C->getSExtValue() == Val;
+  return C && C->getAPIntValue().trySExtValue() == Val;
 }
 
 LLVM_ATTRIBUTE_ALWAYS_INLINE static bool
diff --git a/llvm/test/CodeGen/SystemZ/xor-09.ll b/llvm/test/CodeGen/SystemZ/xor-09.ll
index d0287f7fdd77ec..7b7aaa404c00f6 100644
--- a/llvm/test/CodeGen/SystemZ/xor-09.ll
+++ b/llvm/test/CodeGen/SystemZ/xor-09.ll
@@ -15,3 +15,17 @@ define i128 @f1(i128 %a, i128 %b) {
   %res = xor i128 %a, %b
   ret i128 %res
 }
+
+; Verify that xor with a large constant does not crash.
+define i128 @f2(i128 %x) {
+; CHECK-LABEL: f2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    larl %r1, .LCPI1_0
+; CHECK-NEXT:    vl %v0, 0(%r3), 3
+; CHECK-NEXT:    vl %v1, 0(%r1), 3
+; CHECK-NEXT:    vx %v0, %v0, %v1
+; CHECK-NEXT:    vst %v0, 0(%r2), 3
+; CHECK-NEXT:    br %r14
+  %res = xor i128 %x, 17440380254424117642
+  ret i128 %res
+}

[RKSimon]

LGTM