Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[InstCombine] Set disjoint flag when turning Add into Or. #72702

Merged
merged 4 commits into from
Nov 27, 2023

Conversation

topperc
Copy link
Collaborator

@topperc topperc commented Nov 17, 2023

This is stacked on #72583 where the new disjoint flag was introduced`

@llvmbot
Copy link
Collaborator

llvmbot commented Nov 17, 2023

@llvm/pr-subscribers-clang
@llvm/pr-subscribers-llvm-transforms

@llvm/pr-subscribers-llvm-analysis

Author: Craig Topper (topperc)

Changes

This is stacked on #72583 where the new disjoint flag was introduced`


Patch is 202.43 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/72702.diff

55 Files Affected:

  • (modified) llvm/docs/LangRef.rst (+7)
  • (modified) llvm/include/llvm/AsmParser/LLToken.h (+1)
  • (modified) llvm/include/llvm/Bitcode/LLVMBitCodes.h (+4)
  • (modified) llvm/include/llvm/IR/InstrTypes.h (+17-1)
  • (modified) llvm/include/llvm/IR/Instruction.h (+8)
  • (modified) llvm/lib/AsmParser/LLLexer.cpp (+1)
  • (modified) llvm/lib/AsmParser/LLParser.cpp (+7-1)
  • (modified) llvm/lib/Bitcode/Reader/BitcodeReader.cpp (+3)
  • (modified) llvm/lib/Bitcode/Writer/BitcodeWriter.cpp (+3)
  • (modified) llvm/lib/IR/AsmWriter.cpp (+4)
  • (modified) llvm/lib/IR/Instruction.cpp (+23)
  • (modified) llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp (+5-2)
  • (modified) llvm/test/Analysis/ValueTracking/assume.ll (+1-1)
  • (modified) llvm/test/Assembler/flags.ll (+5)
  • (modified) llvm/test/Bitcode/flags.ll (+4)
  • (modified) llvm/test/Transforms/InstCombine/add.ll (+7-7)
  • (modified) llvm/test/Transforms/InstCombine/add2.ll (+3-3)
  • (modified) llvm/test/Transforms/InstCombine/apint-add.ll (+2-2)
  • (modified) llvm/test/Transforms/InstCombine/apint-shift.ll (+2-2)
  • (modified) llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll (+1-1)
  • (modified) llvm/test/Transforms/InstCombine/freeze.ll (+11)
  • (modified) llvm/test/Transforms/InstCombine/masked-merge-add.ll (+17-17)
  • (modified) llvm/test/Transforms/InstCombine/minmax-intrinsics.ll (+1-1)
  • (modified) llvm/test/Transforms/InstCombine/or.ll (+1-1)
  • (modified) llvm/test/Transforms/InstCombine/ptrtoint-nullgep.ll (+1-1)
  • (modified) llvm/test/Transforms/InstCombine/rem.ll (+1-1)
  • (modified) llvm/test/Transforms/LoopUnroll/AArch64/runtime-unroll-generic.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopUnroll/WebAssembly/basic-unrolling.ll (+4-4)
  • (modified) llvm/test/Transforms/LoopUnroll/runtime-multiexit-heuristic.ll (+14-14)
  • (modified) llvm/test/Transforms/LoopUnroll/runtime-unroll-remainder.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopVectorize/AArch64/sve-interleaved-accesses.ll (+4-4)
  • (modified) llvm/test/Transforms/LoopVectorize/SystemZ/addressing.ll (+1-1)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/float-induction-x86.ll (+14-14)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/parallel-loops.ll (+4-4)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/small-size.ll (+6-6)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/x86-interleaved-accesses-masked-group.ll (+2-2)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/x86-interleaved-store-accesses-with-gaps.ll (+1-1)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/x86_fp80-vector-store.ll (+1-1)
  • (modified) llvm/test/Transforms/LoopVectorize/consecutive-ptr-uniforms.ll (+9-9)
  • (modified) llvm/test/Transforms/LoopVectorize/float-induction.ll (+17-17)
  • (modified) llvm/test/Transforms/LoopVectorize/forked-pointers.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopVectorize/induction.ll (+52-52)
  • (modified) llvm/test/Transforms/LoopVectorize/interleaved-accesses-pred-stores.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll (+20-20)
  • (modified) llvm/test/Transforms/LoopVectorize/loop-scalars.ll (+2-2)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-inloop-cond.ll (+12-12)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-inloop-pred.ll (+45-45)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-inloop-uf4.ll (+30-30)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-inloop.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-predselect.ll (+30-30)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction.ll (+4-4)
  • (modified) llvm/test/Transforms/LoopVectorize/scalar_after_vectorization.ll (+1-1)
  • (modified) llvm/test/Transforms/PhaseOrdering/AArch64/peel-multiple-unreachable-exits-for-vectorization.ll (+2-2)
  • (modified) llvm/test/Transforms/PhaseOrdering/X86/SROA-after-final-loop-unrolling-2.ll (+3-3)
  • (modified) llvm/test/Transforms/SLPVectorizer/AArch64/getelementptr.ll (+1-1)
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index bc1eab1e0b7a07f..a4ea477870448fa 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -9981,6 +9981,7 @@ Syntax:
 ::
 
       <result> = or <ty> <op1>, <op2>   ; yields ty:result
+      <result> = or disjoint <ty> <op1>, <op2>   ; yields ty:result
 
 Overview:
 """""""""
@@ -10012,6 +10013,12 @@ The truth table used for the '``or``' instruction is:
 |   1 |   1 |   1 |
 +-----+-----+-----+
 
+``disjoint`` means every bit is known to be zero in at least one of the inputs.
+This allows the Or to be treated as an Add since no carry can occur from any
+bit. If the disjoint keyword is present, the result value of the ``or`` is a
+:ref:`poison value <poisonvalues>` if both inputs have a one in any bit
+position. For vectors, only the element containing the bit is poison.
+
 Example:
 """"""""
 
diff --git a/llvm/include/llvm/AsmParser/LLToken.h b/llvm/include/llvm/AsmParser/LLToken.h
index c9dcd29b31955dc..f4b12938590fe18 100644
--- a/llvm/include/llvm/AsmParser/LLToken.h
+++ b/llvm/include/llvm/AsmParser/LLToken.h
@@ -109,6 +109,7 @@ enum Kind {
   kw_nuw,
   kw_nsw,
   kw_exact,
+  kw_disjoint,
   kw_inbounds,
   kw_nneg,
   kw_inrange,
diff --git a/llvm/include/llvm/Bitcode/LLVMBitCodes.h b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
index 9fa70c0671ef340..99a41fa107d0811 100644
--- a/llvm/include/llvm/Bitcode/LLVMBitCodes.h
+++ b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
@@ -512,6 +512,10 @@ enum PossiblyNonNegInstOptionalFlags { PNNI_NON_NEG = 0 };
 /// PossiblyExactOperator's SubclassOptionalData contents.
 enum PossiblyExactOperatorOptionalFlags { PEO_EXACT = 0 };
 
+/// PossiblyDisjointInstOptionalFlags - Flags for serializing
+/// PossiblyDisjointInst's SubclassOptionalData contents.
+enum PossiblyDisjointInstOptionalFlags { PDI_DISJOINT = 0 };
+
 /// Encoded AtomicOrdering values.
 enum AtomicOrderingCodes {
   ORDERING_NOTATOMIC = 0,
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index fc5e228168a058b..99145ab9acd7fd3 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -415,6 +415,22 @@ struct OperandTraits<BinaryOperator> :
 
 DEFINE_TRANSPARENT_OPERAND_ACCESSORS(BinaryOperator, Value)
 
+/// A or instruction, which can be marked as "disjoint", indicating that the
+/// inputs don't have a 1 in the same bit position. Meaning this instruction
+/// can also be treated as an add.
+class PossiblyDisjointInst : public BinaryOperator {
+public:
+  enum { IsDisjoint = (1 << 0) };
+
+  static bool classof(const Instruction *I) {
+    return I->getOpcode() == Instruction::Or;
+  }
+
+  static bool classof(const Value *V) {
+    return isa<Instruction>(V) && classof(cast<Instruction>(V));
+  }
+};
+
 //===----------------------------------------------------------------------===//
 //                               CastInst Class
 //===----------------------------------------------------------------------===//
@@ -1085,7 +1101,7 @@ class CmpInst : public Instruction {
   }
 };
 
-// FIXME: these are redundant if CmpInst < BinaryOperator
+// FIXME: these are redundant if CmpInst < ninaryOperator
 template <>
 struct OperandTraits<CmpInst> : public FixedNumOperandTraits<CmpInst, 2> {
 };
diff --git a/llvm/include/llvm/IR/Instruction.h b/llvm/include/llvm/IR/Instruction.h
index 58fc32237367d93..ba5fc35d0d408d5 100644
--- a/llvm/include/llvm/IR/Instruction.h
+++ b/llvm/include/llvm/IR/Instruction.h
@@ -448,6 +448,11 @@ class Instruction : public User,
   /// which supports this flag. See LangRef.html for the meaning of this flag.
   void setIsExact(bool b = true);
 
+  /// Set or clear the disjoint flag on this instruction, which must be an
+  /// operator which supports this flag. See LangRef.html for the meaning of
+  /// this flag.
+  void setIsDisjoint(bool b = true);
+
   /// Set or clear the nneg flag on this instruction, which must be a zext
   /// instruction.
   void setNonNeg(bool b = true);
@@ -500,6 +505,9 @@ class Instruction : public User,
   /// Determine whether the exact flag is set.
   bool isExact() const LLVM_READONLY;
 
+  /// Determine whether the disjoint flag is set.
+  bool isDisjoint() const LLVM_READONLY;
+
   /// Set or clear all fast-math-flags on this instruction, which must be an
   /// operator which supports this flag. See LangRef.html for the meaning of
   /// this flag.
diff --git a/llvm/lib/AsmParser/LLLexer.cpp b/llvm/lib/AsmParser/LLLexer.cpp
index da9e9f4a3c9833b..854aa9cca2c5e37 100644
--- a/llvm/lib/AsmParser/LLLexer.cpp
+++ b/llvm/lib/AsmParser/LLLexer.cpp
@@ -564,6 +564,7 @@ lltok::Kind LLLexer::LexIdentifier() {
   KEYWORD(nuw);
   KEYWORD(nsw);
   KEYWORD(exact);
+  KEYWORD(disjoint);
   KEYWORD(inbounds);
   KEYWORD(nneg);
   KEYWORD(inrange);
diff --git a/llvm/lib/AsmParser/LLParser.cpp b/llvm/lib/AsmParser/LLParser.cpp
index f9df70fb6fc0996..0c170d8da9b73eb 100644
--- a/llvm/lib/AsmParser/LLParser.cpp
+++ b/llvm/lib/AsmParser/LLParser.cpp
@@ -6368,8 +6368,14 @@ int LLParser::parseInstruction(Instruction *&Inst, BasicBlock *BB,
   case lltok::kw_srem:
     return parseArithmetic(Inst, PFS, KeywordVal,
                            /*IsFP*/ false);
+  case lltok::kw_or: {
+    bool Disjoint = EatIfPresent(lltok::kw_disjoint);
+    if (parseLogical(Inst, PFS, KeywordVal))
+      return true;
+    if (Disjoint) cast<PossiblyDisjointInst>(Inst)->setIsDisjoint(true);
+    return false;
+  }
   case lltok::kw_and:
-  case lltok::kw_or:
   case lltok::kw_xor:
     return parseLogical(Inst, PFS, KeywordVal);
   case lltok::kw_icmp:
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index 76431e883b8d96d..e5aaa56f575c3ab 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -4870,6 +4870,9 @@ Error BitcodeReader::parseFunctionBody(Function *F) {
                    Opc == Instruction::AShr) {
           if (Record[OpNum] & (1 << bitc::PEO_EXACT))
             cast<BinaryOperator>(I)->setIsExact(true);
+        } else if (Opc == Instruction::Or) {
+          if (Record[OpNum] & (1 << bitc::PDI_DISJOINT))
+            cast<BinaryOperator>(I)->setIsDisjoint(true);
         } else if (isa<FPMathOperator>(I)) {
           FastMathFlags FMF = getDecodedFastMathFlags(Record[OpNum]);
           if (FMF.any())
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index d16b5c7781c2413..135801a5c61c434 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -1541,6 +1541,9 @@ static uint64_t getOptimizationFlags(const Value *V) {
   } else if (const auto *PEO = dyn_cast<PossiblyExactOperator>(V)) {
     if (PEO->isExact())
       Flags |= 1 << bitc::PEO_EXACT;
+  } else if (const auto *PDI = dyn_cast<PossiblyDisjointInst>(V)) {
+    if (PDI->isDisjoint())
+      Flags |= 1 << bitc::PDI_DISJOINT;
   } else if (const auto *FPMO = dyn_cast<FPMathOperator>(V)) {
     if (FPMO->hasAllowReassoc())
       Flags |= bitc::AllowReassoc;
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index 6d66b34423949fb..688f1d7e078eaec 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -1355,6 +1355,10 @@ static void WriteOptimizationInfo(raw_ostream &Out, const User *U) {
                dyn_cast<PossiblyExactOperator>(U)) {
     if (Div->isExact())
       Out << " exact";
+  } else if (const PossiblyDisjointInst *PDI =
+               dyn_cast<PossiblyDisjointInst>(U)) {
+    if (PDI->isDisjoint())
+      Out << " disjoint";
   } else if (const GEPOperator *GEP = dyn_cast<GEPOperator>(U)) {
     if (GEP->isInBounds())
       Out << " inbounds";
diff --git a/llvm/lib/IR/Instruction.cpp b/llvm/lib/IR/Instruction.cpp
index 7449692f05d7bf9..fcf79f6cd8c4615 100644
--- a/llvm/lib/IR/Instruction.cpp
+++ b/llvm/lib/IR/Instruction.cpp
@@ -317,6 +317,12 @@ void Instruction::setIsExact(bool b) {
   cast<PossiblyExactOperator>(this)->setIsExact(b);
 }
 
+void Instruction::setIsDisjoint(bool b) {
+  assert(isa<PossiblyDisjointInst>(this) && "Must be or");
+  SubclassOptionalData = (SubclassOptionalData & ~PossiblyDisjointInst::IsDisjoint) |
+                         (b * PossiblyDisjointInst::IsDisjoint);
+}
+
 void Instruction::setNonNeg(bool b) {
   assert(isa<PossiblyNonNegInst>(this) && "Must be zext");
   SubclassOptionalData = (SubclassOptionalData & ~PossiblyNonNegInst::NonNeg) |
@@ -357,6 +363,10 @@ void Instruction::dropPoisonGeneratingFlags() {
     cast<PossiblyExactOperator>(this)->setIsExact(false);
     break;
 
+  case Instruction::Or:
+    cast<PossiblyDisjointInst>(this)->setIsDisjoint(false);
+    break;
+
   case Instruction::GetElementPtr:
     cast<GetElementPtrInst>(this)->setIsInBounds(false);
     break;
@@ -419,6 +429,11 @@ bool Instruction::isExact() const {
   return cast<PossiblyExactOperator>(this)->isExact();
 }
 
+bool Instruction::isDisjoint() const {
+  assert(isa<PossiblyDisjointInst>(this) && "Must be or");
+  return (SubclassOptionalData & PossiblyDisjointInst::IsDisjoint) != 0;
+}
+
 void Instruction::setFast(bool B) {
   assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
   cast<FPMathOperator>(this)->setFast(B);
@@ -532,6 +547,10 @@ void Instruction::copyIRFlags(const Value *V, bool IncludeWrapFlags) {
     if (isa<PossiblyExactOperator>(this))
       setIsExact(PE->isExact());
 
+  if (auto *PD = dyn_cast<PossiblyDisjointInst>(V))
+    if (isa<PossiblyDisjointInst>(this))
+      setIsDisjoint(PD->isDisjoint());
+
   // Copy the fast-math flags.
   if (auto *FP = dyn_cast<FPMathOperator>(V))
     if (isa<FPMathOperator>(this))
@@ -558,6 +577,10 @@ void Instruction::andIRFlags(const Value *V) {
     if (isa<PossiblyExactOperator>(this))
       setIsExact(isExact() && PE->isExact());
 
+  if (auto *PE = dyn_cast<PossiblyDisjointInst>(V))
+    if (isa<PossiblyDisjointInst>(this))
+      setIsDisjoint(isDisjoint() && PE->isDisjoint());
+
   if (auto *FP = dyn_cast<FPMathOperator>(V)) {
     if (isa<FPMathOperator>(this)) {
       FastMathFlags FM = getFastMathFlags();
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp b/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp
index 318992b55e4f9f8..1536a490d0cd5a1 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp
@@ -1571,8 +1571,11 @@ Instruction *InstCombinerImpl::visitAdd(BinaryOperator &I) {
 
   // A+B --> A|B iff A and B have no bits set in common.
   WithCache<const Value *> LHSCache(LHS), RHSCache(RHS);
-  if (haveNoCommonBitsSet(LHSCache, RHSCache, SQ.getWithInstruction(&I)))
-    return BinaryOperator::CreateOr(LHS, RHS);
+  if (haveNoCommonBitsSet(LHSCache, RHSCache, SQ.getWithInstruction(&I))) {
+    auto *Or = BinaryOperator::CreateOr(LHS, RHS);
+    Or->setIsDisjoint();
+    return Or;
+  }
 
   if (Instruction *Ext = narrowMathIfNoOverflow(I))
     return Ext;
diff --git a/llvm/test/Analysis/ValueTracking/assume.ll b/llvm/test/Analysis/ValueTracking/assume.ll
index cc098e10138321d..4c4b46c41996868 100644
--- a/llvm/test/Analysis/ValueTracking/assume.ll
+++ b/llvm/test/Analysis/ValueTracking/assume.ll
@@ -7,7 +7,7 @@ define i32 @assume_add(i32 %a, i32 %b) {
 ; CHECK-NEXT:    [[LAST_TWO_DIGITS:%.*]] = and i32 [[T1]], 3
 ; CHECK-NEXT:    [[T2:%.*]] = icmp eq i32 [[LAST_TWO_DIGITS]], 0
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[T2]])
-; CHECK-NEXT:    [[T3:%.*]] = or i32 [[T1]], 3
+; CHECK-NEXT:    [[T3:%.*]] = or disjoint i32 [[T1]], 3
 ; CHECK-NEXT:    ret i32 [[T3]]
 ;
   %t1 = add i32 %a, %b
diff --git a/llvm/test/Assembler/flags.ll b/llvm/test/Assembler/flags.ll
index 6ab5e1bfb9c4f46..04bddd02f50c814 100644
--- a/llvm/test/Assembler/flags.ll
+++ b/llvm/test/Assembler/flags.ll
@@ -256,3 +256,8 @@ define i64 @test_zext(i32 %a) {
   ret i64 %res
 }
 
+define i64 @test_or(i64 %a, i64 %b) {
+; CHECK: %res = or disjoint i64 %a, %b
+  %res = or disjoint i64 %a, %b
+  ret i64 %res
+}
diff --git a/llvm/test/Bitcode/flags.ll b/llvm/test/Bitcode/flags.ll
index a6e368b7e76327f..e3fc827d865d7e2 100644
--- a/llvm/test/Bitcode/flags.ll
+++ b/llvm/test/Bitcode/flags.ll
@@ -18,6 +18,8 @@ second:                                           ; preds = %first
   %z = add i32 %a, 0                              ; <i32> [#uses=0]
   %hh = zext nneg i32 %a to i64
   %ll = zext i32 %s to i64
+  %jj = or disjoint i32 %a, 0
+  %oo = or i32 %a, 0
   unreachable
 
 first:                                            ; preds = %entry
@@ -28,5 +30,7 @@ first:                                            ; preds = %entry
   %zz = add i32 %a, 0                             ; <i32> [#uses=0]
   %kk = zext nneg i32 %a to i64
   %rr = zext i32 %ss to i64
+  %mm = or disjoint i32 %a, 0
+  %nn = or i32 %a, 0
   br label %second
 }
diff --git a/llvm/test/Transforms/InstCombine/add.ll b/llvm/test/Transforms/InstCombine/add.ll
index 2ae0181974bf8a6..8d7260cd4473ad4 100644
--- a/llvm/test/Transforms/InstCombine/add.ll
+++ b/llvm/test/Transforms/InstCombine/add.ll
@@ -199,7 +199,7 @@ define i32 @test8(i32 %A, i32 %B) {
 ; CHECK-LABEL: @test8(
 ; CHECK-NEXT:    [[A1:%.*]] = and i32 [[A:%.*]], 7
 ; CHECK-NEXT:    [[B1:%.*]] = and i32 [[B:%.*]], 128
-; CHECK-NEXT:    [[C:%.*]] = or i32 [[A1]], [[B1]]
+; CHECK-NEXT:    [[C:%.*]] = or disjoint i32 [[A1]], [[B1]]
 ; CHECK-NEXT:    ret i32 [[C]]
 ;
   %A1 = and i32 %A, 7
@@ -810,7 +810,7 @@ define i32 @test32(i32 %A) {
 
 define i8 @test33(i8 %A) {
 ; CHECK-LABEL: @test33(
-; CHECK-NEXT:    [[C:%.*]] = or i8 [[A:%.*]], 1
+; CHECK-NEXT:    [[C:%.*]] = or disjoint i8 [[A:%.*]], 1
 ; CHECK-NEXT:    ret i8 [[C]]
 ;
   %B = and i8 %A, -2
@@ -2565,7 +2565,7 @@ define i16 @add_sub_zext_constant(i8 %x) {
 define <vscale x 1 x i32> @add_to_or_scalable(<vscale x 1 x i32> %in) {
 ; CHECK-LABEL: @add_to_or_scalable(
 ; CHECK-NEXT:    [[SHL:%.*]] = shl <vscale x 1 x i32> [[IN:%.*]], shufflevector (<vscale x 1 x i32> insertelement (<vscale x 1 x i32> poison, i32 1, i32 0), <vscale x 1 x i32> poison, <vscale x 1 x i32> zeroinitializer)
-; CHECK-NEXT:    [[ADD:%.*]] = or <vscale x 1 x i32> [[SHL]], shufflevector (<vscale x 1 x i32> insertelement (<vscale x 1 x i32> poison, i32 1, i32 0), <vscale x 1 x i32> poison, <vscale x 1 x i32> zeroinitializer)
+; CHECK-NEXT:    [[ADD:%.*]] = or disjoint <vscale x 1 x i32> [[SHL]], shufflevector (<vscale x 1 x i32> insertelement (<vscale x 1 x i32> poison, i32 1, i32 0), <vscale x 1 x i32> poison, <vscale x 1 x i32> zeroinitializer)
 ; CHECK-NEXT:    ret <vscale x 1 x i32> [[ADD]]
 ;
   %shl = shl <vscale x 1 x i32> %in, shufflevector (<vscale x 1 x i32> insertelement (<vscale x 1 x i32> poison, i32 1, i32 0), <vscale x 1 x i32> poison, <vscale x 1 x i32> zeroinitializer)
@@ -2626,7 +2626,7 @@ define i5 @zext_sext_not(i4 %x) {
 ; CHECK-NEXT:    [[ZX:%.*]] = zext i4 [[X:%.*]] to i5
 ; CHECK-NEXT:    [[NOTX:%.*]] = xor i4 [[X]], -1
 ; CHECK-NEXT:    [[SNOTX:%.*]] = sext i4 [[NOTX]] to i5
-; CHECK-NEXT:    [[R:%.*]] = or i5 [[ZX]], [[SNOTX]]
+; CHECK-NEXT:    [[R:%.*]] = or disjoint i5 [[ZX]], [[SNOTX]]
 ; CHECK-NEXT:    ret i5 [[R]]
 ;
   %zx = zext i4 %x to i5
@@ -2643,7 +2643,7 @@ define i8 @zext_sext_not_commute(i4 %x) {
 ; CHECK-NEXT:    [[NOTX:%.*]] = xor i4 [[X]], -1
 ; CHECK-NEXT:    [[SNOTX:%.*]] = sext i4 [[NOTX]] to i8
 ; CHECK-NEXT:    call void @use(i8 [[SNOTX]])
-; CHECK-NEXT:    [[R:%.*]] = or i8 [[SNOTX]], [[ZX]]
+; CHECK-NEXT:    [[R:%.*]] = or disjoint i8 [[SNOTX]], [[ZX]]
 ; CHECK-NEXT:    ret i8 [[R]]
 ;
   %zx = zext i4 %x to i8
@@ -2660,7 +2660,7 @@ define i9 @sext_zext_not(i4 %x) {
 ; CHECK-NEXT:    [[SX:%.*]] = sext i4 [[X:%.*]] to i9
 ; CHECK-NEXT:    [[NOTX:%.*]] = xor i4 [[X]], -1
 ; CHECK-NEXT:    [[ZNOTX:%.*]] = zext i4 [[NOTX]] to i9
-; CHECK-NEXT:    [[R:%.*]] = or i9 [[SX]], [[ZNOTX]]
+; CHECK-NEXT:    [[R:%.*]] = or disjoint i9 [[SX]], [[ZNOTX]]
 ; CHECK-NEXT:    ret i9 [[R]]
 ;
   %sx = sext i4 %x to i9
@@ -2675,7 +2675,7 @@ define i9 @sext_zext_not_commute(i4 %x) {
 ; CHECK-NEXT:    [[SX:%.*]] = sext i4 [[X:%.*]] to i9
 ; CHECK-NEXT:    [[NOTX:%.*]] = xor i4 [[X]], -1
 ; CHECK-NEXT:    [[ZNOTX:%.*]] = zext i4 [[NOTX]] to i9
-; CHECK-NEXT:    [[R:%.*]] = or i9 [[ZNOTX]], [[SX]]
+; CHECK-NEXT:    [[R:%.*]] = or disjoint i9 [[ZNOTX]], [[SX]]
 ; CHECK-NEXT:    ret i9 [[R]]
 ;
   %sx = sext i4 %x to i9
diff --git a/llvm/test/Transforms/InstCombine/add2.ll b/llvm/test/Transforms/InstCombine/add2.ll
index 9170d291794e070..9ebcdac77179ee2 100644
--- a/llvm/test/Transforms/InstCombine/add2.ll
+++ b/llvm/test/Transforms/InstCombine/add2.ll
@@ -28,7 +28,7 @@ define i32 @test3(i32 %A) {
 ; CHECK-LABEL: @test3(
 ; CHECK-NEXT:    [[B:%.*]] = and i32 [[A:%.*]], 128
 ; CHECK-NEXT:    [[C:%.*]] = lshr i32 [[A]], 30
-; CHECK-NEXT:    [[F:%.*]] = or i32 [[B]], [[C]]
+; CHECK-NEXT:    [[F:%.*]] = or disjoint i32 [[B]], [[C]]
 ; CHECK-NEXT:    ret i32 [[F]]
 ;
   %B = and i32 %A, 128
@@ -330,7 +330,7 @@ define i16 @mul_add_to_mul_9(i16 %a) {
 define i16 @add_cttz(i16 %a) {
 ; CHECK-LABEL: @add_cttz(
 ; CHECK-NEXT:    [[CTTZ:%.*]] = call i16 @llvm.cttz.i16(i16 [[A:%.*]], i1 true), !range [[RNG0:![0-9]+]]
-; CHECK-NEXT:    [[B:%.*]] = or i16 [[CTTZ]], -8
+; CHECK-NEXT:    [[B:%.*]] = or disjoint i16 [[CTTZ]], -8
 ; CHECK-NEXT:    ret i16 [[B]]
 ;
   ; llvm.cttz.i16(..., /*is_zero_undefined=*/true) implies the value returned
@@ -352,7 +352,7 @@ declare i16 @llvm.cttz.i16(i16, i1)
 define i16 @add_cttz_2(i16 %a) {
 ; CHECK-LABEL: @add_cttz_2(
 ; CHECK-NEXT:    [[CTTZ:%.*]] = call i16 @llvm.cttz.i16(i16 [[A:%.*]], i1 true), !range [[RNG1:![0-9]+]]
-; CHECK-NEXT:    [[B:%.*]] = or i16 [[CTTZ]], -16
+; CHECK-NEXT:    [[B:%.*]] = or disjoint i16 [[CTTZ]], -16
 ; CHECK-NEXT:    ret i16 [[B]]
 ;
   ; llvm.cttz.i16(..., /*is_zero_undefined=*/true) implies the value returned
diff --git a/llvm/test/Transforms/InstCombine/apint-add.ll b/llvm/test/Transforms/InstCombine/apint-add.ll
index c125fe7db605a3d..a1305d8ad149266 100644
--- a/llvm/test/Transforms/InstCombine/apint-add.ll
+++ b/llvm/test/Transforms/InstCombine/apint-add.ll
@@ -46,7 +46,7 @@ define <2 x i5> @test3vec(<2 x i5> %x) {
 ;; (x & 0b1111..0) + 1 -> x | 1
 define i49 @test4(i49 %x) {
 ; CHECK-LABEL: @test4(
-; CHECK-NEXT:    [[TMP_4:%.*]] = or i49 [[X:%.*]], 1
+; CHECK-NEXT:    [[TMP_4:%.*]] = or disjoint i49 [[X:%.*]], 1
 ; CHECK-NEXT:    ret i49 [[TMP_4]]
 ;
   %tmp.2 = and i49 %x, 562949953421310
@@ -149,7 +149,7 @@ define i128 @test8(i128 %x) {
 define i77 @test9(i77 %x) {
 ; CHECK-LABEL: @test9(
 ; CHECK-NEXT:    [[TMP_2:%.*]] = and i77 [[X:%.*]], 562949953421310
-; CHECK-NEXT:    [[TMP_4:%.*]] = or i77 [[TMP_2]], 1
+; CHECK-NEXT:    [[TMP_4:%.*]] = or disjoint i77 [[TMP_2]], 1
 ; CHECK-NEXT:    ret i77 [[TMP_4]]
 ;
   %tmp.2 = and i77 %x, 562949953421310
diff --git a/llvm/test/Transforms/InstCombine/apint-shift.ll b/llvm/test/Transforms/InstCombine/apint-shift.ll
index 2d862ff6debd156..377cc9978c5b766 100644
--- a/llvm/test/Transforms/InstCombine/apint-shift.ll
+++ b/llvm/test/Transforms/InstCombine/apint-shift.ll
@@ -481,7 +481,7 @@ define i44 @shl_lshr_eq_amt_multi_use(i44 %A) {
 ; CHECK-LABEL: @shl_lshr_eq_amt_multi_use(
 ; CHECK-NEXT:    [[B:%.*]] = shl i44 [[A:%.*]], 33
 ; CHECK-NEXT:    [[C:%.*]] = and i44 [[A]], 2047
-; CHECK-NEXT:    [[D:%.*]] = or i44 [[B]], [[C]]
+; CHECK-NEXT:    [[D:%.*]] = or disjoint i44 [[B]], [[C]]
 ; CHECK-NEXT:    ret i44 [[D]]
 ;
   %B = shl i44 %A, 33
@@ -496,7 +496,7 @@ define <2 x i44> @shl_lshr_eq_amt_multi_use_splat_vec(<2 x i44> %A) {
 ; CHECK-LABEL: @shl_lshr_eq_amt_multi_use_splat_vec(
 ; CHECK-NEXT:    [[B:%.*]] = shl <2 x i44> [[A:%.*]], <i44 33, i44 33>
 ; CHECK-NEXT:    [[C:%.*]] = and <2 x i44> [[A]], <i44 2047, i44 2047>
-; CHECK-NEXT:    [[D:%.*]] = or <2 x i44> [[B]], [[C]]
+; CHECK-NEXT:    [[D:%.*]] = or disjoint <2 x i44> [[B]], [[C]]
 ; CHECK-NEXT:    ret <2 x i44> [[D]]
 ;
   %B = shl <2 x i44> %A, <i44 33, i44 33>
diff --git a/llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll b/llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll
index 86a57a1b702fa2c..ad2b56f492fb78f 100644
--- a/llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll
+++ b/llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll
@@ -48,7 +48,7 @@ define i8 @add_bitreverse(i8 %a) {
 ; CHECK-LABEL: @add_bitreverse(
 ; CHECK-NEXT:    [[B:%.*]] = and i8 [[A:%.*]], -4
 ; CHECK-NEXT:    [[RE...
[truncated]

@llvmbot
Copy link
Collaborator

llvmbot commented Nov 17, 2023

@llvm/pr-subscribers-llvm-ir

Author: Craig Topper (topperc)

Changes

This is stacked on #72583 where the new disjoint flag was introduced`


Patch is 202.43 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/72702.diff

55 Files Affected:

  • (modified) llvm/docs/LangRef.rst (+7)
  • (modified) llvm/include/llvm/AsmParser/LLToken.h (+1)
  • (modified) llvm/include/llvm/Bitcode/LLVMBitCodes.h (+4)
  • (modified) llvm/include/llvm/IR/InstrTypes.h (+17-1)
  • (modified) llvm/include/llvm/IR/Instruction.h (+8)
  • (modified) llvm/lib/AsmParser/LLLexer.cpp (+1)
  • (modified) llvm/lib/AsmParser/LLParser.cpp (+7-1)
  • (modified) llvm/lib/Bitcode/Reader/BitcodeReader.cpp (+3)
  • (modified) llvm/lib/Bitcode/Writer/BitcodeWriter.cpp (+3)
  • (modified) llvm/lib/IR/AsmWriter.cpp (+4)
  • (modified) llvm/lib/IR/Instruction.cpp (+23)
  • (modified) llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp (+5-2)
  • (modified) llvm/test/Analysis/ValueTracking/assume.ll (+1-1)
  • (modified) llvm/test/Assembler/flags.ll (+5)
  • (modified) llvm/test/Bitcode/flags.ll (+4)
  • (modified) llvm/test/Transforms/InstCombine/add.ll (+7-7)
  • (modified) llvm/test/Transforms/InstCombine/add2.ll (+3-3)
  • (modified) llvm/test/Transforms/InstCombine/apint-add.ll (+2-2)
  • (modified) llvm/test/Transforms/InstCombine/apint-shift.ll (+2-2)
  • (modified) llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll (+1-1)
  • (modified) llvm/test/Transforms/InstCombine/freeze.ll (+11)
  • (modified) llvm/test/Transforms/InstCombine/masked-merge-add.ll (+17-17)
  • (modified) llvm/test/Transforms/InstCombine/minmax-intrinsics.ll (+1-1)
  • (modified) llvm/test/Transforms/InstCombine/or.ll (+1-1)
  • (modified) llvm/test/Transforms/InstCombine/ptrtoint-nullgep.ll (+1-1)
  • (modified) llvm/test/Transforms/InstCombine/rem.ll (+1-1)
  • (modified) llvm/test/Transforms/LoopUnroll/AArch64/runtime-unroll-generic.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopUnroll/WebAssembly/basic-unrolling.ll (+4-4)
  • (modified) llvm/test/Transforms/LoopUnroll/runtime-multiexit-heuristic.ll (+14-14)
  • (modified) llvm/test/Transforms/LoopUnroll/runtime-unroll-remainder.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopVectorize/AArch64/sve-interleaved-accesses.ll (+4-4)
  • (modified) llvm/test/Transforms/LoopVectorize/SystemZ/addressing.ll (+1-1)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/float-induction-x86.ll (+14-14)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/parallel-loops.ll (+4-4)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/small-size.ll (+6-6)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/x86-interleaved-accesses-masked-group.ll (+2-2)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/x86-interleaved-store-accesses-with-gaps.ll (+1-1)
  • (modified) llvm/test/Transforms/LoopVectorize/X86/x86_fp80-vector-store.ll (+1-1)
  • (modified) llvm/test/Transforms/LoopVectorize/consecutive-ptr-uniforms.ll (+9-9)
  • (modified) llvm/test/Transforms/LoopVectorize/float-induction.ll (+17-17)
  • (modified) llvm/test/Transforms/LoopVectorize/forked-pointers.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopVectorize/induction.ll (+52-52)
  • (modified) llvm/test/Transforms/LoopVectorize/interleaved-accesses-pred-stores.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopVectorize/interleaved-accesses.ll (+20-20)
  • (modified) llvm/test/Transforms/LoopVectorize/loop-scalars.ll (+2-2)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-inloop-cond.ll (+12-12)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-inloop-pred.ll (+45-45)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-inloop-uf4.ll (+30-30)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-inloop.ll (+3-3)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction-predselect.ll (+30-30)
  • (modified) llvm/test/Transforms/LoopVectorize/reduction.ll (+4-4)
  • (modified) llvm/test/Transforms/LoopVectorize/scalar_after_vectorization.ll (+1-1)
  • (modified) llvm/test/Transforms/PhaseOrdering/AArch64/peel-multiple-unreachable-exits-for-vectorization.ll (+2-2)
  • (modified) llvm/test/Transforms/PhaseOrdering/X86/SROA-after-final-loop-unrolling-2.ll (+3-3)
  • (modified) llvm/test/Transforms/SLPVectorizer/AArch64/getelementptr.ll (+1-1)
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index bc1eab1e0b7a07f..a4ea477870448fa 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -9981,6 +9981,7 @@ Syntax:
 ::
 
       <result> = or <ty> <op1>, <op2>   ; yields ty:result
+      <result> = or disjoint <ty> <op1>, <op2>   ; yields ty:result
 
 Overview:
 """""""""
@@ -10012,6 +10013,12 @@ The truth table used for the '``or``' instruction is:
 |   1 |   1 |   1 |
 +-----+-----+-----+
 
+``disjoint`` means every bit is known to be zero in at least one of the inputs.
+This allows the Or to be treated as an Add since no carry can occur from any
+bit. If the disjoint keyword is present, the result value of the ``or`` is a
+:ref:`poison value <poisonvalues>` if both inputs have a one in any bit
+position. For vectors, only the element containing the bit is poison.
+
 Example:
 """"""""
 
diff --git a/llvm/include/llvm/AsmParser/LLToken.h b/llvm/include/llvm/AsmParser/LLToken.h
index c9dcd29b31955dc..f4b12938590fe18 100644
--- a/llvm/include/llvm/AsmParser/LLToken.h
+++ b/llvm/include/llvm/AsmParser/LLToken.h
@@ -109,6 +109,7 @@ enum Kind {
   kw_nuw,
   kw_nsw,
   kw_exact,
+  kw_disjoint,
   kw_inbounds,
   kw_nneg,
   kw_inrange,
diff --git a/llvm/include/llvm/Bitcode/LLVMBitCodes.h b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
index 9fa70c0671ef340..99a41fa107d0811 100644
--- a/llvm/include/llvm/Bitcode/LLVMBitCodes.h
+++ b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
@@ -512,6 +512,10 @@ enum PossiblyNonNegInstOptionalFlags { PNNI_NON_NEG = 0 };
 /// PossiblyExactOperator's SubclassOptionalData contents.
 enum PossiblyExactOperatorOptionalFlags { PEO_EXACT = 0 };
 
+/// PossiblyDisjointInstOptionalFlags - Flags for serializing
+/// PossiblyDisjointInst's SubclassOptionalData contents.
+enum PossiblyDisjointInstOptionalFlags { PDI_DISJOINT = 0 };
+
 /// Encoded AtomicOrdering values.
 enum AtomicOrderingCodes {
   ORDERING_NOTATOMIC = 0,
diff --git a/llvm/include/llvm/IR/InstrTypes.h b/llvm/include/llvm/IR/InstrTypes.h
index fc5e228168a058b..99145ab9acd7fd3 100644
--- a/llvm/include/llvm/IR/InstrTypes.h
+++ b/llvm/include/llvm/IR/InstrTypes.h
@@ -415,6 +415,22 @@ struct OperandTraits<BinaryOperator> :
 
 DEFINE_TRANSPARENT_OPERAND_ACCESSORS(BinaryOperator, Value)
 
+/// A or instruction, which can be marked as "disjoint", indicating that the
+/// inputs don't have a 1 in the same bit position. Meaning this instruction
+/// can also be treated as an add.
+class PossiblyDisjointInst : public BinaryOperator {
+public:
+  enum { IsDisjoint = (1 << 0) };
+
+  static bool classof(const Instruction *I) {
+    return I->getOpcode() == Instruction::Or;
+  }
+
+  static bool classof(const Value *V) {
+    return isa<Instruction>(V) && classof(cast<Instruction>(V));
+  }
+};
+
 //===----------------------------------------------------------------------===//
 //                               CastInst Class
 //===----------------------------------------------------------------------===//
@@ -1085,7 +1101,7 @@ class CmpInst : public Instruction {
   }
 };
 
-// FIXME: these are redundant if CmpInst < BinaryOperator
+// FIXME: these are redundant if CmpInst < ninaryOperator
 template <>
 struct OperandTraits<CmpInst> : public FixedNumOperandTraits<CmpInst, 2> {
 };
diff --git a/llvm/include/llvm/IR/Instruction.h b/llvm/include/llvm/IR/Instruction.h
index 58fc32237367d93..ba5fc35d0d408d5 100644
--- a/llvm/include/llvm/IR/Instruction.h
+++ b/llvm/include/llvm/IR/Instruction.h
@@ -448,6 +448,11 @@ class Instruction : public User,
   /// which supports this flag. See LangRef.html for the meaning of this flag.
   void setIsExact(bool b = true);
 
+  /// Set or clear the disjoint flag on this instruction, which must be an
+  /// operator which supports this flag. See LangRef.html for the meaning of
+  /// this flag.
+  void setIsDisjoint(bool b = true);
+
   /// Set or clear the nneg flag on this instruction, which must be a zext
   /// instruction.
   void setNonNeg(bool b = true);
@@ -500,6 +505,9 @@ class Instruction : public User,
   /// Determine whether the exact flag is set.
   bool isExact() const LLVM_READONLY;
 
+  /// Determine whether the disjoint flag is set.
+  bool isDisjoint() const LLVM_READONLY;
+
   /// Set or clear all fast-math-flags on this instruction, which must be an
   /// operator which supports this flag. See LangRef.html for the meaning of
   /// this flag.
diff --git a/llvm/lib/AsmParser/LLLexer.cpp b/llvm/lib/AsmParser/LLLexer.cpp
index da9e9f4a3c9833b..854aa9cca2c5e37 100644
--- a/llvm/lib/AsmParser/LLLexer.cpp
+++ b/llvm/lib/AsmParser/LLLexer.cpp
@@ -564,6 +564,7 @@ lltok::Kind LLLexer::LexIdentifier() {
   KEYWORD(nuw);
   KEYWORD(nsw);
   KEYWORD(exact);
+  KEYWORD(disjoint);
   KEYWORD(inbounds);
   KEYWORD(nneg);
   KEYWORD(inrange);
diff --git a/llvm/lib/AsmParser/LLParser.cpp b/llvm/lib/AsmParser/LLParser.cpp
index f9df70fb6fc0996..0c170d8da9b73eb 100644
--- a/llvm/lib/AsmParser/LLParser.cpp
+++ b/llvm/lib/AsmParser/LLParser.cpp
@@ -6368,8 +6368,14 @@ int LLParser::parseInstruction(Instruction *&Inst, BasicBlock *BB,
   case lltok::kw_srem:
     return parseArithmetic(Inst, PFS, KeywordVal,
                            /*IsFP*/ false);
+  case lltok::kw_or: {
+    bool Disjoint = EatIfPresent(lltok::kw_disjoint);
+    if (parseLogical(Inst, PFS, KeywordVal))
+      return true;
+    if (Disjoint) cast<PossiblyDisjointInst>(Inst)->setIsDisjoint(true);
+    return false;
+  }
   case lltok::kw_and:
-  case lltok::kw_or:
   case lltok::kw_xor:
     return parseLogical(Inst, PFS, KeywordVal);
   case lltok::kw_icmp:
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index 76431e883b8d96d..e5aaa56f575c3ab 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -4870,6 +4870,9 @@ Error BitcodeReader::parseFunctionBody(Function *F) {
                    Opc == Instruction::AShr) {
           if (Record[OpNum] & (1 << bitc::PEO_EXACT))
             cast<BinaryOperator>(I)->setIsExact(true);
+        } else if (Opc == Instruction::Or) {
+          if (Record[OpNum] & (1 << bitc::PDI_DISJOINT))
+            cast<BinaryOperator>(I)->setIsDisjoint(true);
         } else if (isa<FPMathOperator>(I)) {
           FastMathFlags FMF = getDecodedFastMathFlags(Record[OpNum]);
           if (FMF.any())
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index d16b5c7781c2413..135801a5c61c434 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -1541,6 +1541,9 @@ static uint64_t getOptimizationFlags(const Value *V) {
   } else if (const auto *PEO = dyn_cast<PossiblyExactOperator>(V)) {
     if (PEO->isExact())
       Flags |= 1 << bitc::PEO_EXACT;
+  } else if (const auto *PDI = dyn_cast<PossiblyDisjointInst>(V)) {
+    if (PDI->isDisjoint())
+      Flags |= 1 << bitc::PDI_DISJOINT;
   } else if (const auto *FPMO = dyn_cast<FPMathOperator>(V)) {
     if (FPMO->hasAllowReassoc())
       Flags |= bitc::AllowReassoc;
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index 6d66b34423949fb..688f1d7e078eaec 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -1355,6 +1355,10 @@ static void WriteOptimizationInfo(raw_ostream &Out, const User *U) {
                dyn_cast<PossiblyExactOperator>(U)) {
     if (Div->isExact())
       Out << " exact";
+  } else if (const PossiblyDisjointInst *PDI =
+               dyn_cast<PossiblyDisjointInst>(U)) {
+    if (PDI->isDisjoint())
+      Out << " disjoint";
   } else if (const GEPOperator *GEP = dyn_cast<GEPOperator>(U)) {
     if (GEP->isInBounds())
       Out << " inbounds";
diff --git a/llvm/lib/IR/Instruction.cpp b/llvm/lib/IR/Instruction.cpp
index 7449692f05d7bf9..fcf79f6cd8c4615 100644
--- a/llvm/lib/IR/Instruction.cpp
+++ b/llvm/lib/IR/Instruction.cpp
@@ -317,6 +317,12 @@ void Instruction::setIsExact(bool b) {
   cast<PossiblyExactOperator>(this)->setIsExact(b);
 }
 
+void Instruction::setIsDisjoint(bool b) {
+  assert(isa<PossiblyDisjointInst>(this) && "Must be or");
+  SubclassOptionalData = (SubclassOptionalData & ~PossiblyDisjointInst::IsDisjoint) |
+                         (b * PossiblyDisjointInst::IsDisjoint);
+}
+
 void Instruction::setNonNeg(bool b) {
   assert(isa<PossiblyNonNegInst>(this) && "Must be zext");
   SubclassOptionalData = (SubclassOptionalData & ~PossiblyNonNegInst::NonNeg) |
@@ -357,6 +363,10 @@ void Instruction::dropPoisonGeneratingFlags() {
     cast<PossiblyExactOperator>(this)->setIsExact(false);
     break;
 
+  case Instruction::Or:
+    cast<PossiblyDisjointInst>(this)->setIsDisjoint(false);
+    break;
+
   case Instruction::GetElementPtr:
     cast<GetElementPtrInst>(this)->setIsInBounds(false);
     break;
@@ -419,6 +429,11 @@ bool Instruction::isExact() const {
   return cast<PossiblyExactOperator>(this)->isExact();
 }
 
+bool Instruction::isDisjoint() const {
+  assert(isa<PossiblyDisjointInst>(this) && "Must be or");
+  return (SubclassOptionalData & PossiblyDisjointInst::IsDisjoint) != 0;
+}
+
 void Instruction::setFast(bool B) {
   assert(isa<FPMathOperator>(this) && "setting fast-math flag on invalid op");
   cast<FPMathOperator>(this)->setFast(B);
@@ -532,6 +547,10 @@ void Instruction::copyIRFlags(const Value *V, bool IncludeWrapFlags) {
     if (isa<PossiblyExactOperator>(this))
       setIsExact(PE->isExact());
 
+  if (auto *PD = dyn_cast<PossiblyDisjointInst>(V))
+    if (isa<PossiblyDisjointInst>(this))
+      setIsDisjoint(PD->isDisjoint());
+
   // Copy the fast-math flags.
   if (auto *FP = dyn_cast<FPMathOperator>(V))
     if (isa<FPMathOperator>(this))
@@ -558,6 +577,10 @@ void Instruction::andIRFlags(const Value *V) {
     if (isa<PossiblyExactOperator>(this))
       setIsExact(isExact() && PE->isExact());
 
+  if (auto *PE = dyn_cast<PossiblyDisjointInst>(V))
+    if (isa<PossiblyDisjointInst>(this))
+      setIsDisjoint(isDisjoint() && PE->isDisjoint());
+
   if (auto *FP = dyn_cast<FPMathOperator>(V)) {
     if (isa<FPMathOperator>(this)) {
       FastMathFlags FM = getFastMathFlags();
diff --git a/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp b/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp
index 318992b55e4f9f8..1536a490d0cd5a1 100644
--- a/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp
+++ b/llvm/lib/Transforms/InstCombine/InstCombineAddSub.cpp
@@ -1571,8 +1571,11 @@ Instruction *InstCombinerImpl::visitAdd(BinaryOperator &I) {
 
   // A+B --> A|B iff A and B have no bits set in common.
   WithCache<const Value *> LHSCache(LHS), RHSCache(RHS);
-  if (haveNoCommonBitsSet(LHSCache, RHSCache, SQ.getWithInstruction(&I)))
-    return BinaryOperator::CreateOr(LHS, RHS);
+  if (haveNoCommonBitsSet(LHSCache, RHSCache, SQ.getWithInstruction(&I))) {
+    auto *Or = BinaryOperator::CreateOr(LHS, RHS);
+    Or->setIsDisjoint();
+    return Or;
+  }
 
   if (Instruction *Ext = narrowMathIfNoOverflow(I))
     return Ext;
diff --git a/llvm/test/Analysis/ValueTracking/assume.ll b/llvm/test/Analysis/ValueTracking/assume.ll
index cc098e10138321d..4c4b46c41996868 100644
--- a/llvm/test/Analysis/ValueTracking/assume.ll
+++ b/llvm/test/Analysis/ValueTracking/assume.ll
@@ -7,7 +7,7 @@ define i32 @assume_add(i32 %a, i32 %b) {
 ; CHECK-NEXT:    [[LAST_TWO_DIGITS:%.*]] = and i32 [[T1]], 3
 ; CHECK-NEXT:    [[T2:%.*]] = icmp eq i32 [[LAST_TWO_DIGITS]], 0
 ; CHECK-NEXT:    call void @llvm.assume(i1 [[T2]])
-; CHECK-NEXT:    [[T3:%.*]] = or i32 [[T1]], 3
+; CHECK-NEXT:    [[T3:%.*]] = or disjoint i32 [[T1]], 3
 ; CHECK-NEXT:    ret i32 [[T3]]
 ;
   %t1 = add i32 %a, %b
diff --git a/llvm/test/Assembler/flags.ll b/llvm/test/Assembler/flags.ll
index 6ab5e1bfb9c4f46..04bddd02f50c814 100644
--- a/llvm/test/Assembler/flags.ll
+++ b/llvm/test/Assembler/flags.ll
@@ -256,3 +256,8 @@ define i64 @test_zext(i32 %a) {
   ret i64 %res
 }
 
+define i64 @test_or(i64 %a, i64 %b) {
+; CHECK: %res = or disjoint i64 %a, %b
+  %res = or disjoint i64 %a, %b
+  ret i64 %res
+}
diff --git a/llvm/test/Bitcode/flags.ll b/llvm/test/Bitcode/flags.ll
index a6e368b7e76327f..e3fc827d865d7e2 100644
--- a/llvm/test/Bitcode/flags.ll
+++ b/llvm/test/Bitcode/flags.ll
@@ -18,6 +18,8 @@ second:                                           ; preds = %first
   %z = add i32 %a, 0                              ; <i32> [#uses=0]
   %hh = zext nneg i32 %a to i64
   %ll = zext i32 %s to i64
+  %jj = or disjoint i32 %a, 0
+  %oo = or i32 %a, 0
   unreachable
 
 first:                                            ; preds = %entry
@@ -28,5 +30,7 @@ first:                                            ; preds = %entry
   %zz = add i32 %a, 0                             ; <i32> [#uses=0]
   %kk = zext nneg i32 %a to i64
   %rr = zext i32 %ss to i64
+  %mm = or disjoint i32 %a, 0
+  %nn = or i32 %a, 0
   br label %second
 }
diff --git a/llvm/test/Transforms/InstCombine/add.ll b/llvm/test/Transforms/InstCombine/add.ll
index 2ae0181974bf8a6..8d7260cd4473ad4 100644
--- a/llvm/test/Transforms/InstCombine/add.ll
+++ b/llvm/test/Transforms/InstCombine/add.ll
@@ -199,7 +199,7 @@ define i32 @test8(i32 %A, i32 %B) {
 ; CHECK-LABEL: @test8(
 ; CHECK-NEXT:    [[A1:%.*]] = and i32 [[A:%.*]], 7
 ; CHECK-NEXT:    [[B1:%.*]] = and i32 [[B:%.*]], 128
-; CHECK-NEXT:    [[C:%.*]] = or i32 [[A1]], [[B1]]
+; CHECK-NEXT:    [[C:%.*]] = or disjoint i32 [[A1]], [[B1]]
 ; CHECK-NEXT:    ret i32 [[C]]
 ;
   %A1 = and i32 %A, 7
@@ -810,7 +810,7 @@ define i32 @test32(i32 %A) {
 
 define i8 @test33(i8 %A) {
 ; CHECK-LABEL: @test33(
-; CHECK-NEXT:    [[C:%.*]] = or i8 [[A:%.*]], 1
+; CHECK-NEXT:    [[C:%.*]] = or disjoint i8 [[A:%.*]], 1
 ; CHECK-NEXT:    ret i8 [[C]]
 ;
   %B = and i8 %A, -2
@@ -2565,7 +2565,7 @@ define i16 @add_sub_zext_constant(i8 %x) {
 define <vscale x 1 x i32> @add_to_or_scalable(<vscale x 1 x i32> %in) {
 ; CHECK-LABEL: @add_to_or_scalable(
 ; CHECK-NEXT:    [[SHL:%.*]] = shl <vscale x 1 x i32> [[IN:%.*]], shufflevector (<vscale x 1 x i32> insertelement (<vscale x 1 x i32> poison, i32 1, i32 0), <vscale x 1 x i32> poison, <vscale x 1 x i32> zeroinitializer)
-; CHECK-NEXT:    [[ADD:%.*]] = or <vscale x 1 x i32> [[SHL]], shufflevector (<vscale x 1 x i32> insertelement (<vscale x 1 x i32> poison, i32 1, i32 0), <vscale x 1 x i32> poison, <vscale x 1 x i32> zeroinitializer)
+; CHECK-NEXT:    [[ADD:%.*]] = or disjoint <vscale x 1 x i32> [[SHL]], shufflevector (<vscale x 1 x i32> insertelement (<vscale x 1 x i32> poison, i32 1, i32 0), <vscale x 1 x i32> poison, <vscale x 1 x i32> zeroinitializer)
 ; CHECK-NEXT:    ret <vscale x 1 x i32> [[ADD]]
 ;
   %shl = shl <vscale x 1 x i32> %in, shufflevector (<vscale x 1 x i32> insertelement (<vscale x 1 x i32> poison, i32 1, i32 0), <vscale x 1 x i32> poison, <vscale x 1 x i32> zeroinitializer)
@@ -2626,7 +2626,7 @@ define i5 @zext_sext_not(i4 %x) {
 ; CHECK-NEXT:    [[ZX:%.*]] = zext i4 [[X:%.*]] to i5
 ; CHECK-NEXT:    [[NOTX:%.*]] = xor i4 [[X]], -1
 ; CHECK-NEXT:    [[SNOTX:%.*]] = sext i4 [[NOTX]] to i5
-; CHECK-NEXT:    [[R:%.*]] = or i5 [[ZX]], [[SNOTX]]
+; CHECK-NEXT:    [[R:%.*]] = or disjoint i5 [[ZX]], [[SNOTX]]
 ; CHECK-NEXT:    ret i5 [[R]]
 ;
   %zx = zext i4 %x to i5
@@ -2643,7 +2643,7 @@ define i8 @zext_sext_not_commute(i4 %x) {
 ; CHECK-NEXT:    [[NOTX:%.*]] = xor i4 [[X]], -1
 ; CHECK-NEXT:    [[SNOTX:%.*]] = sext i4 [[NOTX]] to i8
 ; CHECK-NEXT:    call void @use(i8 [[SNOTX]])
-; CHECK-NEXT:    [[R:%.*]] = or i8 [[SNOTX]], [[ZX]]
+; CHECK-NEXT:    [[R:%.*]] = or disjoint i8 [[SNOTX]], [[ZX]]
 ; CHECK-NEXT:    ret i8 [[R]]
 ;
   %zx = zext i4 %x to i8
@@ -2660,7 +2660,7 @@ define i9 @sext_zext_not(i4 %x) {
 ; CHECK-NEXT:    [[SX:%.*]] = sext i4 [[X:%.*]] to i9
 ; CHECK-NEXT:    [[NOTX:%.*]] = xor i4 [[X]], -1
 ; CHECK-NEXT:    [[ZNOTX:%.*]] = zext i4 [[NOTX]] to i9
-; CHECK-NEXT:    [[R:%.*]] = or i9 [[SX]], [[ZNOTX]]
+; CHECK-NEXT:    [[R:%.*]] = or disjoint i9 [[SX]], [[ZNOTX]]
 ; CHECK-NEXT:    ret i9 [[R]]
 ;
   %sx = sext i4 %x to i9
@@ -2675,7 +2675,7 @@ define i9 @sext_zext_not_commute(i4 %x) {
 ; CHECK-NEXT:    [[SX:%.*]] = sext i4 [[X:%.*]] to i9
 ; CHECK-NEXT:    [[NOTX:%.*]] = xor i4 [[X]], -1
 ; CHECK-NEXT:    [[ZNOTX:%.*]] = zext i4 [[NOTX]] to i9
-; CHECK-NEXT:    [[R:%.*]] = or i9 [[ZNOTX]], [[SX]]
+; CHECK-NEXT:    [[R:%.*]] = or disjoint i9 [[ZNOTX]], [[SX]]
 ; CHECK-NEXT:    ret i9 [[R]]
 ;
   %sx = sext i4 %x to i9
diff --git a/llvm/test/Transforms/InstCombine/add2.ll b/llvm/test/Transforms/InstCombine/add2.ll
index 9170d291794e070..9ebcdac77179ee2 100644
--- a/llvm/test/Transforms/InstCombine/add2.ll
+++ b/llvm/test/Transforms/InstCombine/add2.ll
@@ -28,7 +28,7 @@ define i32 @test3(i32 %A) {
 ; CHECK-LABEL: @test3(
 ; CHECK-NEXT:    [[B:%.*]] = and i32 [[A:%.*]], 128
 ; CHECK-NEXT:    [[C:%.*]] = lshr i32 [[A]], 30
-; CHECK-NEXT:    [[F:%.*]] = or i32 [[B]], [[C]]
+; CHECK-NEXT:    [[F:%.*]] = or disjoint i32 [[B]], [[C]]
 ; CHECK-NEXT:    ret i32 [[F]]
 ;
   %B = and i32 %A, 128
@@ -330,7 +330,7 @@ define i16 @mul_add_to_mul_9(i16 %a) {
 define i16 @add_cttz(i16 %a) {
 ; CHECK-LABEL: @add_cttz(
 ; CHECK-NEXT:    [[CTTZ:%.*]] = call i16 @llvm.cttz.i16(i16 [[A:%.*]], i1 true), !range [[RNG0:![0-9]+]]
-; CHECK-NEXT:    [[B:%.*]] = or i16 [[CTTZ]], -8
+; CHECK-NEXT:    [[B:%.*]] = or disjoint i16 [[CTTZ]], -8
 ; CHECK-NEXT:    ret i16 [[B]]
 ;
   ; llvm.cttz.i16(..., /*is_zero_undefined=*/true) implies the value returned
@@ -352,7 +352,7 @@ declare i16 @llvm.cttz.i16(i16, i1)
 define i16 @add_cttz_2(i16 %a) {
 ; CHECK-LABEL: @add_cttz_2(
 ; CHECK-NEXT:    [[CTTZ:%.*]] = call i16 @llvm.cttz.i16(i16 [[A:%.*]], i1 true), !range [[RNG1:![0-9]+]]
-; CHECK-NEXT:    [[B:%.*]] = or i16 [[CTTZ]], -16
+; CHECK-NEXT:    [[B:%.*]] = or disjoint i16 [[CTTZ]], -16
 ; CHECK-NEXT:    ret i16 [[B]]
 ;
   ; llvm.cttz.i16(..., /*is_zero_undefined=*/true) implies the value returned
diff --git a/llvm/test/Transforms/InstCombine/apint-add.ll b/llvm/test/Transforms/InstCombine/apint-add.ll
index c125fe7db605a3d..a1305d8ad149266 100644
--- a/llvm/test/Transforms/InstCombine/apint-add.ll
+++ b/llvm/test/Transforms/InstCombine/apint-add.ll
@@ -46,7 +46,7 @@ define <2 x i5> @test3vec(<2 x i5> %x) {
 ;; (x & 0b1111..0) + 1 -> x | 1
 define i49 @test4(i49 %x) {
 ; CHECK-LABEL: @test4(
-; CHECK-NEXT:    [[TMP_4:%.*]] = or i49 [[X:%.*]], 1
+; CHECK-NEXT:    [[TMP_4:%.*]] = or disjoint i49 [[X:%.*]], 1
 ; CHECK-NEXT:    ret i49 [[TMP_4]]
 ;
   %tmp.2 = and i49 %x, 562949953421310
@@ -149,7 +149,7 @@ define i128 @test8(i128 %x) {
 define i77 @test9(i77 %x) {
 ; CHECK-LABEL: @test9(
 ; CHECK-NEXT:    [[TMP_2:%.*]] = and i77 [[X:%.*]], 562949953421310
-; CHECK-NEXT:    [[TMP_4:%.*]] = or i77 [[TMP_2]], 1
+; CHECK-NEXT:    [[TMP_4:%.*]] = or disjoint i77 [[TMP_2]], 1
 ; CHECK-NEXT:    ret i77 [[TMP_4]]
 ;
   %tmp.2 = and i77 %x, 562949953421310
diff --git a/llvm/test/Transforms/InstCombine/apint-shift.ll b/llvm/test/Transforms/InstCombine/apint-shift.ll
index 2d862ff6debd156..377cc9978c5b766 100644
--- a/llvm/test/Transforms/InstCombine/apint-shift.ll
+++ b/llvm/test/Transforms/InstCombine/apint-shift.ll
@@ -481,7 +481,7 @@ define i44 @shl_lshr_eq_amt_multi_use(i44 %A) {
 ; CHECK-LABEL: @shl_lshr_eq_amt_multi_use(
 ; CHECK-NEXT:    [[B:%.*]] = shl i44 [[A:%.*]], 33
 ; CHECK-NEXT:    [[C:%.*]] = and i44 [[A]], 2047
-; CHECK-NEXT:    [[D:%.*]] = or i44 [[B]], [[C]]
+; CHECK-NEXT:    [[D:%.*]] = or disjoint i44 [[B]], [[C]]
 ; CHECK-NEXT:    ret i44 [[D]]
 ;
   %B = shl i44 %A, 33
@@ -496,7 +496,7 @@ define <2 x i44> @shl_lshr_eq_amt_multi_use_splat_vec(<2 x i44> %A) {
 ; CHECK-LABEL: @shl_lshr_eq_amt_multi_use_splat_vec(
 ; CHECK-NEXT:    [[B:%.*]] = shl <2 x i44> [[A:%.*]], <i44 33, i44 33>
 ; CHECK-NEXT:    [[C:%.*]] = and <2 x i44> [[A]], <i44 2047, i44 2047>
-; CHECK-NEXT:    [[D:%.*]] = or <2 x i44> [[B]], [[C]]
+; CHECK-NEXT:    [[D:%.*]] = or disjoint <2 x i44> [[B]], [[C]]
 ; CHECK-NEXT:    ret <2 x i44> [[D]]
 ;
   %B = shl <2 x i44> %A, <i44 33, i44 33>
diff --git a/llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll b/llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll
index 86a57a1b702fa2c..ad2b56f492fb78f 100644
--- a/llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll
+++ b/llvm/test/Transforms/InstCombine/bitreverse-known-bits.ll
@@ -48,7 +48,7 @@ define i8 @add_bitreverse(i8 %a) {
 ; CHECK-LABEL: @add_bitreverse(
 ; CHECK-NEXT:    [[B:%.*]] = and i8 [[A:%.*]], -4
 ; CHECK-NEXT:    [[RE...
[truncated]

Copy link

github-actions bot commented Nov 17, 2023

✅ With the latest revision this PR passed the C/C++ code formatter.

@nikic
Copy link
Contributor

nikic commented Nov 20, 2023

It looks like some clang tests also need to be updated.

auto *Or = BinaryOperator::CreateOr(LHS, RHS);
Or->setIsDisjoint();
return Or;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@llvmbot llvmbot added the clang Clang issues not falling into any other category label Nov 20, 2023
@topperc topperc force-pushed the pr/disjoint-instcombine branch 2 times, most recently from 135b0ad to e333cd8 Compare November 20, 2023 22:09
@nhaehnle
Copy link
Collaborator

Time to rebase this on main, I think...

Copy link
Contributor

@nikic nikic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM assuming CI happy.

@topperc topperc merged commit 03d4a9d into llvm:main Nov 27, 2023
2 of 3 checks passed
@topperc topperc deleted the pr/disjoint-instcombine branch November 27, 2023 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang Clang issues not falling into any other category llvm:analysis llvm:ir llvm:transforms
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants