[SROA] Allow llvm.launder.invariant.group intrinsic to be splittable

[antoniofrighetto]

Fixes: #72035.

[llvmbot]

@llvm/pr-subscribers-llvm-transforms

Author: Antonio Frighetto (antoniofrighetto)

Changes

Fixes: #72035.

---

Full diff: https://github.com/llvm/llvm-project/pull/72056.diff

2 Files Affected:

• (modified) llvm/lib/Transforms/Scalar/SROA.cpp (+6-1)
• (modified) llvm/test/Transforms/SROA/invariant-group.ll (+30)

diff --git a/llvm/lib/Transforms/Scalar/SROA.cpp b/llvm/lib/Transforms/Scalar/SROA.cpp
index 321525d3741d0a2..095f12cd414b803 100644
--- a/llvm/lib/Transforms/Scalar/SROA.cpp
+++ b/llvm/lib/Transforms/Scalar/SROA.cpp
@@ -1144,6 +1144,7 @@ class AllocaSlices::SliceBuilder : public PtrUseVisitor<SliceBuilder> {
     }
 
     if (II.isLaunderOrStripInvariantGroup()) {
+      insertUse(II, Offset, AllocSize, true);
       enqueueUsers(II);
       return;
     }
@@ -3328,7 +3329,8 @@ class llvm::sroa::AllocaSliceRewriter
   }
 
   bool visitIntrinsicInst(IntrinsicInst &II) {
-    assert((II.isLifetimeStartOrEnd() || II.isDroppable()) &&
+    assert((II.isLifetimeStartOrEnd() || II.isLaunderOrStripInvariantGroup() ||
+            II.isDroppable()) &&
            "Unexpected intrinsic!");
     LLVM_DEBUG(dbgs() << "    original: " << II << "\n");
 
@@ -3342,6 +3344,9 @@ class llvm::sroa::AllocaSliceRewriter
       return true;
     }
 
+    if (II.isLaunderOrStripInvariantGroup())
+      return true;
+
     assert(II.getArgOperand(1) == OldPtr);
     // Lifetime intrinsics are only promotable if they cover the whole alloca.
     // Therefore, we drop lifetime intrinsics which don't cover the whole
diff --git a/llvm/test/Transforms/SROA/invariant-group.ll b/llvm/test/Transforms/SROA/invariant-group.ll
index 083fa72026b85e6..eceeca2c997bab9 100644
--- a/llvm/test/Transforms/SROA/invariant-group.ll
+++ b/llvm/test/Transforms/SROA/invariant-group.ll
@@ -79,6 +79,36 @@ define void @g() {
   ret void
 }
 
+define void @store_and_launder() {
+; CHECK-LABEL: @store_and_launder(
+; CHECK-NEXT:    ret void
+;
+  %valptr = alloca i32, align 4
+  store i32 0, ptr %valptr, align 4
+  %barr = call ptr @llvm.launder.invariant.group.p0(ptr %valptr)
+  ret void
+}
+
+define i32 @launder_and_load() {
+; CHECK-LABEL: @launder_and_load(
+; CHECK-NEXT:    ret i32 undef
+;
+  %valptr = alloca i32, align 4
+  %barr = call ptr @llvm.launder.invariant.group.p0(ptr %valptr)
+  %v2 = load i32, ptr %valptr
+  ret i32 %v2
+}
+
+define void @launder_and_ptr_arith() {
+; CHECK-LABEL: @launder_and_ptr_arith(
+; CHECK-NEXT:    ret void
+;
+  %valptr = alloca i32, align 4
+  %barr = call ptr @llvm.launder.invariant.group.p0(ptr %valptr)
+  %a2 = getelementptr inbounds i32, ptr %valptr, i32 0
+  ret void
+}
+
 !0 = !{}
 ;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
 ; CHECK-MODIFY-CFG: {{.*}}

[nikic]

Can you please test a case where part of the alloca needs to be retained?

[antoniofrighetto]

Not exactly sure if the optimized IR of partial_use_of_struct test is what we want.

[nikic]

Not exactly sure if the optimized IR of partial_use_of_struct test is what we want.

That's not it. I'm looking for a case where the alloca is split and some parts are promoted, but others aren't, e.g. because they are accessed via volatile operations. There should still be an alloca left at the end, and the launder.invariant.group needs to be rewritten (and possibly duplicated) appropriately.

[antoniofrighetto]

I think it should look better now.

[nikic]

Can you please also add some invariant group metadata to this test? It looks like the launder.invariant intrinsic gets dropped, which is okay only if the metadata also gets dropped.

[antoniofrighetto]

Added, thanks! AFAIU, in our case, the invariant.group should be added to memory accesses to the same pointer operand.

[nikic]

LGTM

[nikic]

Suggested change

	call void @use(i32* nonnull %valptr)
	call void @use(ptr nonnull %valptr)