[BDCE] Handle multi-use `and`/`or`/`xor` on demanded bits #79688

antoniofrighetto · 2024-01-27T10:45:13Z

Simplify multi-use and or ors when these last do not affect the demanded bits being considered.

Fixes: #78596.

Proofs: https://alive2.llvm.org/ce/z/kmVqE-.

llvmbot · 2024-01-27T10:45:40Z

@llvm/pr-subscribers-llvm-transforms

Author: Antonio Frighetto (antoniofrighetto)

Changes

Simplify multi-use select of ands when these last do not affect the demanded bits being considered.

Fixes: #78596.

Proofs: https://alive2.llvm.org/ce/z/BBdrCz.

Full diff: https://github.com/llvm/llvm-project/pull/79688.diff

2 Files Affected:

(modified) llvm/lib/Transforms/Scalar/BDCE.cpp (+27)
(added) llvm/test/Transforms/BDCE/select-multiuse.ll (+122)

diff --git a/llvm/lib/Transforms/Scalar/BDCE.cpp b/llvm/lib/Transforms/Scalar/BDCE.cpp
index 1fa2c75b0f42ac1..178b5a0da469cdd 100644
--- a/llvm/lib/Transforms/Scalar/BDCE.cpp
+++ b/llvm/lib/Transforms/Scalar/BDCE.cpp
@@ -125,6 +125,33 @@ static bool bitTrackingDCE(Function &F, DemandedBits &DB) {
       }
     }
 
+    // Simplify select of ands when the mask does not affect the demanded bits.
+    if (SelectInst *SI = dyn_cast<SelectInst>(&I)) {
+      APInt Demanded = DB.getDemandedBits(SI);
+      if (!Demanded.isAllOnes()) {
+        for (Use &U : I.operands()) {
+          auto *And = dyn_cast<BinaryOperator>(&U);
+          if (!And || !And->hasOneUse() || And->getOpcode() != Instruction::And)
+            continue;
+
+          auto *CV = dyn_cast<ConstantInt>(And->getOperand(1));
+          if (!CV)
+            continue;
+
+          APInt Mask = CV->getValue();
+          if ((Mask | ~Demanded).isAllOnes()) {
+            clearAssumptionsOfUsers(And, DB);
+            U.set(And->getOperand(0));
+            Worklist.push_back(And);
+            Changed = true;
+          }
+        }
+      }
+
+      if (Changed)
+        continue;
+    }
+
     for (Use &U : I.operands()) {
       // DemandedBits only detects dead integer uses.
       if (!U->getType()->isIntOrIntVectorTy())
diff --git a/llvm/test/Transforms/BDCE/select-multiuse.ll b/llvm/test/Transforms/BDCE/select-multiuse.ll
new file mode 100644
index 000000000000000..8ad91cc7458d114
--- /dev/null
+++ b/llvm/test/Transforms/BDCE/select-multiuse.ll
@@ -0,0 +1,122 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 4
+; RUN: opt -S -passes=bdce < %s | FileCheck %s
+
+define void @select(i1 %c, i64 %a, i64 %b) {
+; CHECK-LABEL: define void @select(
+; CHECK-SAME: i1 [[C:%.*]], i64 [[A:%.*]], i64 [[B:%.*]]) {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[S:%.*]] = select i1 [[C]], i64 [[A]], i64 [[B]]
+; CHECK-NEXT:    [[RET1:%.*]] = and i64 [[S]], 8
+; CHECK-NEXT:    [[RET2:%.*]] = and i64 [[S]], 16
+; CHECK-NEXT:    call void @use(i64 [[RET1]])
+; CHECK-NEXT:    call void @use(i64 [[RET2]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  %and1 = and i64 %a, 24                          ; Mask:          0001 1000
+  %and2 = and i64 %b, 25                          ; Mask:          0001 1001
+  %s = select i1 %c, i64 %and1, i64 %and2
+  %ret1 = and i64 %s, 8                           ; Demanded bits: 0000 1000
+  %ret2 = and i64 %s, 16                          ; Demanded bits: 0001 0000
+  call void @use(i64 %ret1)
+  call void @use(i64 %ret2)
+  ret void
+}
+
+define void @select_2(i1 %c, i64 %a, i64 %b) {
+; CHECK-LABEL: define void @select_2(
+; CHECK-SAME: i1 [[C:%.*]], i64 [[A:%.*]], i64 [[B:%.*]]) {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[AND2:%.*]] = and i64 [[B]], 23
+; CHECK-NEXT:    [[S:%.*]] = select i1 [[C]], i64 [[A]], i64 [[AND2]]
+; CHECK-NEXT:    [[RET1:%.*]] = and i64 [[S]], 8
+; CHECK-NEXT:    [[RET2:%.*]] = and i64 [[S]], 16
+; CHECK-NEXT:    call void @use(i64 [[RET1]])
+; CHECK-NEXT:    call void @use(i64 [[RET2]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  %and1 = and i64 %a, 25                          ; Mask:          0001 1001
+  %and2 = and i64 %b, 23                          ; Mask:          0001 0111
+  %s = select i1 %c, i64 %and1, i64 %and2
+  %ret1 = and i64 %s, 8                           ; Demanded bits: 0000 1000
+  %ret2 = and i64 %s, 16                          ; Demanded bits: 0001 0000
+  call void @use(i64 %ret1)
+  call void @use(i64 %ret2)
+  ret void
+}
+
+define void @select_different_demanded(i1 %c, i64 %a, i64 %b) {
+; CHECK-LABEL: define void @select_different_demanded(
+; CHECK-SAME: i1 [[C:%.*]], i64 [[A:%.*]], i64 [[B:%.*]]) {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[AND1:%.*]] = and i64 0, 24
+; CHECK-NEXT:    [[AND2:%.*]] = and i64 [[B]], 25
+; CHECK-NEXT:    [[S:%.*]] = select i1 [[C]], i64 [[AND1]], i64 [[AND2]]
+; CHECK-NEXT:    [[RET1:%.*]] = and i64 [[S]], 3
+; CHECK-NEXT:    [[RET2:%.*]] = and i64 [[S]], 7
+; CHECK-NEXT:    call void @use(i64 [[RET1]])
+; CHECK-NEXT:    call void @use(i64 [[RET2]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  %and1 = and i64 %a, 24                          ; Mask:          0001 1000
+  %and2 = and i64 %b, 25                          ; Mask:          0001 1001
+  %s = select i1 %c, i64 %and1, i64 %and2
+  %ret1 = and i64 %s, 3                           ; Demanded bits: 0000 0011
+  %ret2 = and i64 %s, 7                           ; Demanded bits: 0000 0111
+  call void @use(i64 %ret1)
+  call void @use(i64 %ret2)
+  ret void
+}
+
+define void @and_multiuse(i1 %c, i64 %a, i64 %b) {
+; CHECK-LABEL: define void @and_multiuse(
+; CHECK-SAME: i1 [[C:%.*]], i64 [[A:%.*]], i64 [[B:%.*]]) {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[AND2:%.*]] = and i64 [[B]], 25
+; CHECK-NEXT:    [[S:%.*]] = select i1 [[C]], i64 [[A]], i64 [[AND2]]
+; CHECK-NEXT:    [[RET1:%.*]] = and i64 [[S]], 8
+; CHECK-NEXT:    [[RET2:%.*]] = and i64 [[S]], 16
+; CHECK-NEXT:    call void @use(i64 [[RET1]])
+; CHECK-NEXT:    call void @use2(i64 [[RET2]], i64 [[AND2]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  %and1 = and i64 %a, 24
+  %and2 = and i64 %b, 25
+  %s = select i1 %c, i64 %and1, i64 %and2
+  %ret1 = and i64 %s, 8
+  %ret2 = and i64 %s, 16
+  call void @use(i64 %ret1)
+  call void @use2(i64 %ret2, i64 %and2)
+  ret void
+}
+
+define void @select_vectorized(i1 %c, <2 x i8> %a, <2 x i8> %b) {
+; CHECK-LABEL: define void @select_vectorized(
+; CHECK-SAME: i1 [[C:%.*]], <2 x i8> [[A:%.*]], <2 x i8> [[B:%.*]]) {
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[AND1:%.*]] = and <2 x i8> [[A]], <i8 28, i8 28>
+; CHECK-NEXT:    [[AND2:%.*]] = and <2 x i8> [[B]], <i8 29, i8 29>
+; CHECK-NEXT:    [[S:%.*]] = select i1 [[C]], <2 x i8> [[AND1]], <2 x i8> [[AND2]]
+; CHECK-NEXT:    [[RET1:%.*]] = and <2 x i8> [[S]], <i8 4, i8 4>
+; CHECK-NEXT:    [[RET2:%.*]] = and <2 x i8> [[S]], <i8 12, i8 12>
+; CHECK-NEXT:    call void @use3(<2 x i8> [[RET1]])
+; CHECK-NEXT:    call void @use3(<2 x i8> [[RET2]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  %and1 = and <2 x i8> %a, <i8 28, i8 28>
+  %and2 = and <2 x i8> %b, <i8 29, i8 29>
+  %s = select i1 %c, <2 x i8> %and1, <2 x i8> %and2
+  %ret1 = and <2 x i8> %s, <i8 4, i8 4>
+  %ret2 = and <2 x i8> %s, <i8 12, i8 12>
+  call void @use3(<2 x i8> %ret1)
+  call void @use3(<2 x i8> %ret2)
+  ret void
+}
+
+declare void @use(i64)
+declare void @use2(i64, i64)
+declare void @use3(<2 x i8>)

llvm/lib/Transforms/Scalar/BDCE.cpp

PR Link: llvm/llvm-project#79688

nikic

...wat? This should not be looking at selects. You should only be looking at the demanded bits of the and.

antoniofrighetto · 2024-01-27T13:55:36Z

Could you kindly provide an example showing how not restricting this to the select may be beneficial? We don't need to specialize it for the select, but restricting the input space looked reasonable to me.

You should only be looking at the demanded bits of the and.

You mean we should only be looking at the demanded bits of the instruction itself when the operand analyzed is the and? I don't see how the optimization can take place if the instruction examined is not the multi-use instruction.

nikic · 2024-01-27T14:12:27Z

Could you kindly provide an example showing how not restricting this to the select may be beneficial? We don't need to specialize it for the select, but restricting the input space looked reasonable to me.

You can replace the select with pretty much any instruction that transfers demanded bits, e.g. https://alive2.llvm.org/ce/z/MF1Ry3.

You should only be looking at the demanded bits of the and.

You mean we should only be looking at the demanded bits of the instruction itself when the operand analyzed is the and? I don't see how the optimization can take place if the instruction examined is not the multi-use instruction.

I would expect you to query the demanded bits of the and instruction, similar to the code directly above that handles sext. Unless I'm missing something, that will cover the motivating pattern.

XChy · 2024-01-27T14:18:44Z

For a single BDCE pass, I think a minimal testcase can be like:

define void @src(i64 %a) {
entry:
  %and = and i64 %a, 24
  %ret1 = and i64 %and, 8
  %ret2 = and i64 %and, 16
  call void @use(i64 %ret1)
  call void @use(i64 %ret2)
  ret void
}

define void @tgt(i64 %a) {
entry:
  %ret1 = and i64 %a, 8
  %ret2 = and i64 %a, 16
  call void @use(i64 %ret1)
  call void @use(i64 %ret2)
  ret void
}

antoniofrighetto · 2024-01-27T18:35:13Z

I would expect you to query the demanded bits of the and instruction, similar to the code directly above that handles sext. Unless I'm missing something, that will cover the motivating pattern.

Reviewed, thanks, somehow I thought it would be easier to handle the and/or as operands of the instruction being analyzed (and get its current demanded bits).

llvm/lib/Transforms/Scalar/BDCE.cpp

PR Link: llvm/llvm-project#79688

nikic

Thanks, this looks a lot more like what I had in mind.

nikic · 2024-01-28T08:39:36Z

llvm/test/Transforms/BDCE/and-or-multiuse.ll

+entry:
+  %and1 = and i64 %a, 24                          ; Mask:          0001 1000
+  %and2 = and i64 %b, 25                          ; Mask:          0001 1001
+  %or = or i64 %and1, %and2


Can you please add a test variant with or disjoint, to show that the flag gets dropped correctly?

nikic · 2024-01-28T08:40:49Z

llvm/test/Transforms/BDCE/dead-uses.ll

@@ -9,8 +9,7 @@ declare <2 x i32> @llvm.fshr.v2i32(<2 x i32>, <2 x i32>, <2 x i32>)
 ; First fshr operand is dead.
 define i32 @pr39771_fshr_multi_use_instr(i32 %a) {
 ; CHECK-LABEL: @pr39771_fshr_multi_use_instr(
-; CHECK-NEXT:    [[X:%.*]] = or i32 [[A:%.*]], 0


Can you change the or operand to 1 (or something else that doesn't optimize away) instead? Dropping it is not really the point of this tests.

antoniofrighetto · 2024-01-29T07:24:56Z

Reviewed, thanks.

nikic

LGTM

XChy

LGTM.

Simplify multi-use `and`/`or`/`xor` when these last do not affect the demanded bits being considered. Fixes: llvm#78596. Proofs: https://alive2.llvm.org/ce/z/EjuWHa.

antoniofrighetto requested review from nikic, dtcxzyw and XChy January 27, 2024 10:45

llvmbot added the llvm:transforms label Jan 27, 2024

dtcxzyw reviewed Jan 27, 2024

View reviewed changes

dtcxzyw added a commit to dtcxzyw/llvm-opt-benchmark that referenced this pull request Jan 27, 2024

pre-commit: test PR79688

8bf0530

PR Link: llvm/llvm-project#79688

dtcxzyw mentioned this pull request Jan 27, 2024

pre-commit: test PR79688 dtcxzyw/llvm-opt-benchmark#157

Closed

nikic requested changes Jan 27, 2024

View reviewed changes

antoniofrighetto force-pushed the feature/bdce-multiuse-select-of-ands branch from ac76f66 to c862fbb Compare January 27, 2024 18:38

antoniofrighetto changed the title ~~[BDCE] Handle multi-use select of ands on demanded bits~~ [BDCE] Handle multi-use and/or on demanded bits Jan 27, 2024

antoniofrighetto force-pushed the feature/bdce-multiuse-select-of-ands branch from c862fbb to 524bf49 Compare January 27, 2024 18:58

XChy reviewed Jan 28, 2024

View reviewed changes

llvm/lib/Transforms/Scalar/BDCE.cpp Outdated Show resolved Hide resolved

llvm/lib/Transforms/Scalar/BDCE.cpp Outdated Show resolved Hide resolved

dtcxzyw added a commit to dtcxzyw/llvm-opt-benchmark that referenced this pull request Jan 28, 2024

pre-commit: test PR79688

75f5842

PR Link: llvm/llvm-project#79688

nikic reviewed Jan 28, 2024

View reviewed changes

antoniofrighetto force-pushed the feature/bdce-multiuse-select-of-ands branch from 524bf49 to f98b584 Compare January 29, 2024 07:23

antoniofrighetto requested review from nikic, dtcxzyw and XChy January 29, 2024 13:13

nikic approved these changes Jan 29, 2024

View reviewed changes

XChy changed the title ~~[BDCE] Handle multi-use and/or on demanded bits~~ [BDCE] Handle multi-use and/or/xor on demanded bits Jan 29, 2024

XChy approved these changes Jan 29, 2024

View reviewed changes

antoniofrighetto force-pushed the feature/bdce-multiuse-select-of-ands branch from f98b584 to 3b03a6a Compare January 29, 2024 17:59

antoniofrighetto added 2 commits January 29, 2024 19:03

[BDCE] Introduce test for PR79688 (NFC)

0a15ead

[BDCE] Handle multi-use binary ops upon demanded bits

2073782

Simplify multi-use `and`/`or`/`xor` when these last do not affect the demanded bits being considered. Fixes: llvm#78596. Proofs: https://alive2.llvm.org/ce/z/EjuWHa.

antoniofrighetto force-pushed the feature/bdce-multiuse-select-of-ands branch from 3b03a6a to 2073782 Compare January 29, 2024 18:03

antoniofrighetto merged commit 2073782 into llvm:main Jan 29, 2024
3 of 4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BDCE] Handle multi-use `and`/`or`/`xor` on demanded bits #79688

[BDCE] Handle multi-use `and`/`or`/`xor` on demanded bits #79688

antoniofrighetto commented Jan 27, 2024 •

edited

llvmbot commented Jan 27, 2024

nikic left a comment

antoniofrighetto commented Jan 27, 2024

nikic commented Jan 27, 2024 •

edited

XChy commented Jan 27, 2024

antoniofrighetto commented Jan 27, 2024

nikic left a comment

nikic Jan 28, 2024

nikic Jan 28, 2024

antoniofrighetto commented Jan 29, 2024

nikic left a comment

XChy left a comment

[BDCE] Handle multi-use and/or/xor on demanded bits #79688

[BDCE] Handle multi-use and/or/xor on demanded bits #79688

Conversation

antoniofrighetto commented Jan 27, 2024 • edited

llvmbot commented Jan 27, 2024

nikic left a comment

Choose a reason for hiding this comment

antoniofrighetto commented Jan 27, 2024

nikic commented Jan 27, 2024 • edited

XChy commented Jan 27, 2024

antoniofrighetto commented Jan 27, 2024

nikic left a comment

Choose a reason for hiding this comment

nikic Jan 28, 2024

Choose a reason for hiding this comment

nikic Jan 28, 2024

Choose a reason for hiding this comment

antoniofrighetto commented Jan 29, 2024

nikic left a comment

Choose a reason for hiding this comment

XChy left a comment

Choose a reason for hiding this comment

[BDCE] Handle multi-use `and`/`or`/`xor` on demanded bits #79688

[BDCE] Handle multi-use `and`/`or`/`xor` on demanded bits #79688

antoniofrighetto commented Jan 27, 2024 •

edited

nikic commented Jan 27, 2024 •

edited