Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MacroFusion] Add IsPostRA to indicate whether running in post-ra scheduler #77567

Closed

Conversation

wangpc-pp
Copy link
Contributor

This can save some time to know whether MacroFusion mutation is
running in post-ra scheduler.

And this can be used in #77461.

@llvmbot
Copy link
Collaborator

llvmbot commented Jan 10, 2024

@llvm/pr-subscribers-backend-x86
@llvm/pr-subscribers-backend-risc-v
@llvm/pr-subscribers-backend-arm

@llvm/pr-subscribers-backend-aarch64

Author: Wang Pengcheng (wangpc-pp)

Changes

This can save some time to know whether MacroFusion mutation is
running in post-ra scheduler.

And this can be used in #77461.


Patch is 33.37 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/77567.diff

25 Files Affected:

  • (modified) llvm/include/llvm/CodeGen/MacroFusion.h (+3-2)
  • (modified) llvm/lib/CodeGen/MacroFusion.cpp (+8-6)
  • (modified) llvm/lib/Target/AArch64/AArch64MacroFusion.cpp (+5-4)
  • (modified) llvm/lib/Target/AArch64/AArch64MacroFusion.h (+2-1)
  • (modified) llvm/lib/Target/AArch64/AArch64TargetMachine.cpp (+2-2)
  • (modified) llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.cpp (+5-3)
  • (modified) llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.h (+2-1)
  • (modified) llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp (+2-2)
  • (modified) llvm/lib/Target/AMDGPU/GCNVOPDUtils.cpp (+4-3)
  • (modified) llvm/lib/Target/ARM/ARMMacroFusion.cpp (+6-4)
  • (modified) llvm/lib/Target/ARM/ARMMacroFusion.h (+2-1)
  • (modified) llvm/lib/Target/ARM/ARMTargetMachine.cpp (+2-2)
  • (modified) llvm/lib/Target/PowerPC/PPCMacroFusion.cpp (+5-3)
  • (modified) llvm/lib/Target/PowerPC/PPCMacroFusion.h (+2-1)
  • (modified) llvm/lib/Target/PowerPC/PPCTargetMachine.cpp (+2-2)
  • (modified) llvm/lib/Target/RISCV/RISCVMacroFusion.cpp (+30-24)
  • (modified) llvm/lib/Target/RISCV/RISCVMacroFusion.h (+2-1)
  • (modified) llvm/lib/Target/RISCV/RISCVSubtarget.cpp (+1-1)
  • (modified) llvm/lib/Target/RISCV/RISCVTargetMachine.cpp (+2-2)
  • (modified) llvm/lib/Target/X86/X86MacroFusion.cpp (+5-3)
  • (modified) llvm/lib/Target/X86/X86MacroFusion.h (+1-1)
  • (modified) llvm/lib/Target/X86/X86Subtarget.cpp (+1-1)
  • (modified) llvm/lib/Target/X86/X86TargetMachine.cpp (+2-2)
  • (modified) llvm/test/TableGen/MacroFusion.td (+5-7)
  • (modified) llvm/utils/TableGen/MacroFusionPredicatorEmitter.cpp (+9-11)
diff --git a/llvm/include/llvm/CodeGen/MacroFusion.h b/llvm/include/llvm/CodeGen/MacroFusion.h
index 191c906e9ef6c6..e418cbb4e7d5f6 100644
--- a/llvm/include/llvm/CodeGen/MacroFusion.h
+++ b/llvm/include/llvm/CodeGen/MacroFusion.h
@@ -32,7 +32,7 @@ class SUnit;
 using MacroFusionPredTy = bool (*)(const TargetInstrInfo &TII,
                                    const TargetSubtargetInfo &STI,
                                    const MachineInstr *FirstMI,
-                                   const MachineInstr &SecondMI);
+                                   const MachineInstr &SecondMI, bool IsPostRA);
 
 /// Checks if the number of cluster edges between SU and its predecessors is
 /// less than FuseLimit
@@ -50,11 +50,12 @@ bool fuseInstructionPair(ScheduleDAGInstrs &DAG, SUnit &FirstSU,
 /// for instructions that benefit according to the target-specific
 /// predicate functions. shouldScheduleAdjacent will be true if any of the
 /// provided predicates are true.
+/// If IsPostRA is true, this mutation will run in post RA scheduler.
 /// If BranchOnly is true, only branch instructions with one of their
 /// predecessors will be fused.
 std::unique_ptr<ScheduleDAGMutation>
 createMacroFusionDAGMutation(ArrayRef<MacroFusionPredTy> Predicates,
-                             bool BranchOnly = false);
+                             bool IsPostRA, bool BranchOnly = false);
 
 } // end namespace llvm
 
diff --git a/llvm/lib/CodeGen/MacroFusion.cpp b/llvm/lib/CodeGen/MacroFusion.cpp
index 5bd6ca0978a4b1..b7b39c4fbdb698 100644
--- a/llvm/lib/CodeGen/MacroFusion.cpp
+++ b/llvm/lib/CodeGen/MacroFusion.cpp
@@ -138,13 +138,15 @@ namespace {
 /// be fused by the processor into a single operation.
 class MacroFusion : public ScheduleDAGMutation {
   std::vector<MacroFusionPredTy> Predicates;
+  bool IsPostRA;
   bool FuseBlock;
   bool scheduleAdjacentImpl(ScheduleDAGInstrs &DAG, SUnit &AnchorSU);
 
 public:
-  MacroFusion(ArrayRef<MacroFusionPredTy> Predicates, bool FuseBlock)
-      : Predicates(Predicates.begin(), Predicates.end()), FuseBlock(FuseBlock) {
-  }
+  MacroFusion(ArrayRef<MacroFusionPredTy> Predicates, bool IsPostRA,
+              bool FuseBlock)
+      : Predicates(Predicates.begin(), Predicates.end()), IsPostRA(IsPostRA),
+        FuseBlock(FuseBlock) {}
 
   void apply(ScheduleDAGInstrs *DAGInstrs) override;
 
@@ -161,7 +163,7 @@ bool MacroFusion::shouldScheduleAdjacent(const TargetInstrInfo &TII,
                                          const MachineInstr *FirstMI,
                                          const MachineInstr &SecondMI) {
   return llvm::any_of(Predicates, [&](MacroFusionPredTy Predicate) {
-    return Predicate(TII, STI, FirstMI, SecondMI);
+    return Predicate(TII, STI, FirstMI, SecondMI, IsPostRA);
   });
 }
 
@@ -213,8 +215,8 @@ bool MacroFusion::scheduleAdjacentImpl(ScheduleDAGInstrs &DAG, SUnit &AnchorSU)
 
 std::unique_ptr<ScheduleDAGMutation>
 llvm::createMacroFusionDAGMutation(ArrayRef<MacroFusionPredTy> Predicates,
-                                   bool BranchOnly) {
+                                   bool IsPostRA, bool BranchOnly) {
   if (EnableMacroFusion)
-    return std::make_unique<MacroFusion>(Predicates, !BranchOnly);
+    return std::make_unique<MacroFusion>(Predicates, IsPostRA, !BranchOnly);
   return nullptr;
 }
diff --git a/llvm/lib/Target/AArch64/AArch64MacroFusion.cpp b/llvm/lib/Target/AArch64/AArch64MacroFusion.cpp
index 05d60872bf51ac..bc67488abcaea8 100644
--- a/llvm/lib/Target/AArch64/AArch64MacroFusion.cpp
+++ b/llvm/lib/Target/AArch64/AArch64MacroFusion.cpp
@@ -443,8 +443,9 @@ static bool isAddSub2RegAndConstOnePair(const MachineInstr *FirstMI,
 static bool shouldScheduleAdjacent(const TargetInstrInfo &TII,
                                    const TargetSubtargetInfo &TSI,
                                    const MachineInstr *FirstMI,
-                                   const MachineInstr &SecondMI) {
-  const AArch64Subtarget &ST = static_cast<const AArch64Subtarget&>(TSI);
+                                   const MachineInstr &SecondMI,
+                                   bool IsPostRA) {
+  const AArch64Subtarget &ST = static_cast<const AArch64Subtarget &>(TSI);
 
   // All checking functions assume that the 1st instr is a wildcard if it is
   // unspecified.
@@ -477,6 +478,6 @@ static bool shouldScheduleAdjacent(const TargetInstrInfo &TII,
 }
 
 std::unique_ptr<ScheduleDAGMutation>
-llvm::createAArch64MacroFusionDAGMutation() {
-  return createMacroFusionDAGMutation(shouldScheduleAdjacent);
+llvm::createAArch64MacroFusionDAGMutation(bool IsPostRA) {
+  return createMacroFusionDAGMutation(shouldScheduleAdjacent, IsPostRA);
 }
diff --git a/llvm/lib/Target/AArch64/AArch64MacroFusion.h b/llvm/lib/Target/AArch64/AArch64MacroFusion.h
index 2999e7a8aa909c..807db002c762e0 100644
--- a/llvm/lib/Target/AArch64/AArch64MacroFusion.h
+++ b/llvm/lib/Target/AArch64/AArch64MacroFusion.h
@@ -21,7 +21,8 @@ namespace llvm {
 /// Note that you have to add:
 ///   DAG.addMutation(createAArch64MacroFusionDAGMutation());
 /// to AArch64PassConfig::createMachineScheduler() to have an effect.
-std::unique_ptr<ScheduleDAGMutation> createAArch64MacroFusionDAGMutation();
+std::unique_ptr<ScheduleDAGMutation>
+createAArch64MacroFusionDAGMutation(bool IsPostRA);
 
 } // llvm
 
diff --git a/llvm/lib/Target/AArch64/AArch64TargetMachine.cpp b/llvm/lib/Target/AArch64/AArch64TargetMachine.cpp
index 144610e021c58e..a5648f2856888d 100644
--- a/llvm/lib/Target/AArch64/AArch64TargetMachine.cpp
+++ b/llvm/lib/Target/AArch64/AArch64TargetMachine.cpp
@@ -496,7 +496,7 @@ class AArch64PassConfig : public TargetPassConfig {
     DAG->addMutation(createLoadClusterDAGMutation(DAG->TII, DAG->TRI));
     DAG->addMutation(createStoreClusterDAGMutation(DAG->TII, DAG->TRI));
     if (ST.hasFusion())
-      DAG->addMutation(createAArch64MacroFusionDAGMutation());
+      DAG->addMutation(createAArch64MacroFusionDAGMutation(/*IsPostRA=*/false));
     return DAG;
   }
 
@@ -509,7 +509,7 @@ class AArch64PassConfig : public TargetPassConfig {
     if (ST.hasFusion()) {
       // Run the Macro Fusion after RA again since literals are expanded from
       // pseudos then (v. addPreSched2()).
-      DAG->addMutation(createAArch64MacroFusionDAGMutation());
+      DAG->addMutation(createAArch64MacroFusionDAGMutation(/*IsPostRA=*/true));
       return DAG;
     }
 
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.cpp b/llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.cpp
index 0cbabf3895a67e..08d764650ece29 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.cpp
@@ -26,7 +26,8 @@ namespace {
 static bool shouldScheduleAdjacent(const TargetInstrInfo &TII_,
                                    const TargetSubtargetInfo &TSI,
                                    const MachineInstr *FirstMI,
-                                   const MachineInstr &SecondMI) {
+                                   const MachineInstr &SecondMI,
+                                   bool IsPostRA) {
   const SIInstrInfo &TII = static_cast<const SIInstrInfo&>(TII_);
 
   switch (SecondMI.getOpcode()) {
@@ -59,8 +60,9 @@ static bool shouldScheduleAdjacent(const TargetInstrInfo &TII_,
 
 namespace llvm {
 
-std::unique_ptr<ScheduleDAGMutation> createAMDGPUMacroFusionDAGMutation() {
-  return createMacroFusionDAGMutation(shouldScheduleAdjacent);
+std::unique_ptr<ScheduleDAGMutation>
+createAMDGPUMacroFusionDAGMutation(bool IsPostRA) {
+  return createMacroFusionDAGMutation(shouldScheduleAdjacent, IsPostRA);
 }
 
 } // end namespace llvm
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.h b/llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.h
index ad198a301dbe40..6366f97cbcc804 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.h
+++ b/llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.h
@@ -17,7 +17,8 @@ namespace llvm {
 /// Note that you have to add:
 ///   DAG.addMutation(createAMDGPUMacroFusionDAGMutation());
 /// to AMDGPUPassConfig::createMachineScheduler() to have an effect.
-std::unique_ptr<ScheduleDAGMutation> createAMDGPUMacroFusionDAGMutation();
+std::unique_ptr<ScheduleDAGMutation>
+createAMDGPUMacroFusionDAGMutation(bool IsPostRA);
 
 } // llvm
 
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp b/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
index 0f3bb3e7b0d8d0..1a790ac2070425 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
@@ -461,7 +461,7 @@ createGCNMaxOccupancyMachineScheduler(MachineSchedContext *C) {
   if (ST.shouldClusterStores())
     DAG->addMutation(createStoreClusterDAGMutation(DAG->TII, DAG->TRI));
   DAG->addMutation(createIGroupLPDAGMutation(/*IsPostRA=*/false));
-  DAG->addMutation(createAMDGPUMacroFusionDAGMutation());
+  DAG->addMutation(createAMDGPUMacroFusionDAGMutation(/*IsPostRA=*/false));
   DAG->addMutation(createAMDGPUExportClusteringDAGMutation());
   return DAG;
 }
@@ -498,7 +498,7 @@ createIterativeILPMachineScheduler(MachineSchedContext *C) {
   DAG->addMutation(createLoadClusterDAGMutation(DAG->TII, DAG->TRI));
   if (ST.shouldClusterStores())
     DAG->addMutation(createStoreClusterDAGMutation(DAG->TII, DAG->TRI));
-  DAG->addMutation(createAMDGPUMacroFusionDAGMutation());
+  DAG->addMutation(createAMDGPUMacroFusionDAGMutation(/*IsPostRA=*/false));
   return DAG;
 }
 
diff --git a/llvm/lib/Target/AMDGPU/GCNVOPDUtils.cpp b/llvm/lib/Target/AMDGPU/GCNVOPDUtils.cpp
index 33c208495c500e..6328edd70feede 100644
--- a/llvm/lib/Target/AMDGPU/GCNVOPDUtils.cpp
+++ b/llvm/lib/Target/AMDGPU/GCNVOPDUtils.cpp
@@ -123,7 +123,8 @@ bool llvm::checkVOPDRegConstraints(const SIInstrInfo &TII,
 static bool shouldScheduleVOPDAdjacent(const TargetInstrInfo &TII,
                                        const TargetSubtargetInfo &TSI,
                                        const MachineInstr *FirstMI,
-                                       const MachineInstr &SecondMI) {
+                                       const MachineInstr &SecondMI,
+                                       bool IsPostRA) {
   const SIInstrInfo &STII = static_cast<const SIInstrInfo &>(TII);
   unsigned Opc2 = SecondMI.getOpcode();
   auto SecondCanBeVOPD = AMDGPU::getCanBeVOPD(Opc2);
@@ -165,7 +166,7 @@ struct VOPDPairingMutation : ScheduleDAGMutation {
     std::vector<SUnit>::iterator ISUI, JSUI;
     for (ISUI = DAG->SUnits.begin(); ISUI != DAG->SUnits.end(); ++ISUI) {
       const MachineInstr *IMI = ISUI->getInstr();
-      if (!shouldScheduleAdjacent(TII, ST, nullptr, *IMI))
+      if (!shouldScheduleAdjacent(TII, ST, nullptr, *IMI, /*IsPostRA=*/true))
         continue;
       if (!hasLessThanNumFused(*ISUI, 2))
         continue;
@@ -175,7 +176,7 @@ struct VOPDPairingMutation : ScheduleDAGMutation {
           continue;
         const MachineInstr *JMI = JSUI->getInstr();
         if (!hasLessThanNumFused(*JSUI, 2) ||
-            !shouldScheduleAdjacent(TII, ST, IMI, *JMI))
+            !shouldScheduleAdjacent(TII, ST, IMI, *JMI, /*IsPostRA=*/true))
           continue;
         if (fuseInstructionPair(*DAG, *ISUI, *JSUI))
           break;
diff --git a/llvm/lib/Target/ARM/ARMMacroFusion.cpp b/llvm/lib/Target/ARM/ARMMacroFusion.cpp
index 5aeb7abe92a38c..cbe0742560a134 100644
--- a/llvm/lib/Target/ARM/ARMMacroFusion.cpp
+++ b/llvm/lib/Target/ARM/ARMMacroFusion.cpp
@@ -51,8 +51,9 @@ static bool isLiteralsPair(const MachineInstr *FirstMI,
 static bool shouldScheduleAdjacent(const TargetInstrInfo &TII,
                                    const TargetSubtargetInfo &TSI,
                                    const MachineInstr *FirstMI,
-                                   const MachineInstr &SecondMI) {
-  const ARMSubtarget &ST = static_cast<const ARMSubtarget&>(TSI);
+                                   const MachineInstr &SecondMI,
+                                   bool IsPostRA) {
+  const ARMSubtarget &ST = static_cast<const ARMSubtarget &>(TSI);
 
   if (ST.hasFuseAES() && isAESPair(FirstMI, SecondMI))
     return true;
@@ -62,8 +63,9 @@ static bool shouldScheduleAdjacent(const TargetInstrInfo &TII,
   return false;
 }
 
-std::unique_ptr<ScheduleDAGMutation> createARMMacroFusionDAGMutation() {
-  return createMacroFusionDAGMutation(shouldScheduleAdjacent);
+std::unique_ptr<ScheduleDAGMutation>
+createARMMacroFusionDAGMutation(bool IsPostRA) {
+  return createMacroFusionDAGMutation(shouldScheduleAdjacent, IsPostRA);
 }
 
 } // end namespace llvm
diff --git a/llvm/lib/Target/ARM/ARMMacroFusion.h b/llvm/lib/Target/ARM/ARMMacroFusion.h
index 4896a4a2544dbf..3fea3d7671865d 100644
--- a/llvm/lib/Target/ARM/ARMMacroFusion.h
+++ b/llvm/lib/Target/ARM/ARMMacroFusion.h
@@ -21,7 +21,8 @@ namespace llvm {
 /// Note that you have to add:
 ///   DAG.addMutation(createARMMacroFusionDAGMutation());
 /// to ARMPassConfig::createMachineScheduler() to have an effect.
-std::unique_ptr<ScheduleDAGMutation> createARMMacroFusionDAGMutation();
+std::unique_ptr<ScheduleDAGMutation>
+createARMMacroFusionDAGMutation(bool IsPostRA);
 
 } // llvm
 
diff --git a/llvm/lib/Target/ARM/ARMTargetMachine.cpp b/llvm/lib/Target/ARM/ARMTargetMachine.cpp
index a99773691df123..6aeea12f8dcae4 100644
--- a/llvm/lib/Target/ARM/ARMTargetMachine.cpp
+++ b/llvm/lib/Target/ARM/ARMTargetMachine.cpp
@@ -358,7 +358,7 @@ class ARMPassConfig : public TargetPassConfig {
     // add DAG Mutations here.
     const ARMSubtarget &ST = C->MF->getSubtarget<ARMSubtarget>();
     if (ST.hasFusion())
-      DAG->addMutation(createARMMacroFusionDAGMutation());
+      DAG->addMutation(createARMMacroFusionDAGMutation(/*IsPostRA=*/false));
     return DAG;
   }
 
@@ -368,7 +368,7 @@ class ARMPassConfig : public TargetPassConfig {
     // add DAG Mutations here.
     const ARMSubtarget &ST = C->MF->getSubtarget<ARMSubtarget>();
     if (ST.hasFusion())
-      DAG->addMutation(createARMMacroFusionDAGMutation());
+      DAG->addMutation(createARMMacroFusionDAGMutation(/*IsPostRA=*/true));
     return DAG;
   }
 
diff --git a/llvm/lib/Target/PowerPC/PPCMacroFusion.cpp b/llvm/lib/Target/PowerPC/PPCMacroFusion.cpp
index 7ad6ef8c39286d..4638879e15eb18 100644
--- a/llvm/lib/Target/PowerPC/PPCMacroFusion.cpp
+++ b/llvm/lib/Target/PowerPC/PPCMacroFusion.cpp
@@ -234,7 +234,8 @@ static bool checkOpConstraints(FusionFeature::FusionKind Kd,
 static bool shouldScheduleAdjacent(const TargetInstrInfo &TII,
                                    const TargetSubtargetInfo &TSI,
                                    const MachineInstr *FirstMI,
-                                   const MachineInstr &SecondMI) {
+                                   const MachineInstr &SecondMI,
+                                   bool IsPostRA) {
   // We use the PPC namespace to avoid the need to prefix opcodes with PPC:: in
   // the def file.
   using namespace PPC;
@@ -286,8 +287,9 @@ static bool shouldScheduleAdjacent(const TargetInstrInfo &TII,
 
 namespace llvm {
 
-std::unique_ptr<ScheduleDAGMutation> createPowerPCMacroFusionDAGMutation() {
-  return createMacroFusionDAGMutation(shouldScheduleAdjacent);
+std::unique_ptr<ScheduleDAGMutation>
+createPowerPCMacroFusionDAGMutation(bool IsPostRA) {
+  return createMacroFusionDAGMutation(shouldScheduleAdjacent, IsPostRA);
 }
 
 } // end namespace llvm
diff --git a/llvm/lib/Target/PowerPC/PPCMacroFusion.h b/llvm/lib/Target/PowerPC/PPCMacroFusion.h
index cbf49ee779ceb7..2ef4a6d690cc19 100644
--- a/llvm/lib/Target/PowerPC/PPCMacroFusion.h
+++ b/llvm/lib/Target/PowerPC/PPCMacroFusion.h
@@ -21,7 +21,8 @@ namespace llvm {
 /// Note that you have to add:
 ///   DAG.addMutation(createPowerPCMacroFusionDAGMutation());
 /// to PPCPassConfig::createMachineScheduler() to have an effect.
-std::unique_ptr<ScheduleDAGMutation> createPowerPCMacroFusionDAGMutation();
+std::unique_ptr<ScheduleDAGMutation>
+createPowerPCMacroFusionDAGMutation(bool IsPostRA);
 } // llvm
 
 #endif // LLVM_LIB_TARGET_POWERPC_PPCMACROFUSION_H
diff --git a/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp b/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp
index d676fa86a10e77..2bfad35fe6ddaf 100644
--- a/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp
+++ b/llvm/lib/Target/PowerPC/PPCTargetMachine.cpp
@@ -317,7 +317,7 @@ static ScheduleDAGInstrs *createPPCMachineScheduler(MachineSchedContext *C) {
   if (ST.hasStoreFusion())
     DAG->addMutation(createStoreClusterDAGMutation(DAG->TII, DAG->TRI));
   if (ST.hasFusion())
-    DAG->addMutation(createPowerPCMacroFusionDAGMutation());
+    DAG->addMutation(createPowerPCMacroFusionDAGMutation(/*IsPostRA=*/false));
 
   return DAG;
 }
@@ -333,7 +333,7 @@ static ScheduleDAGInstrs *createPPCPostMachineScheduler(
   if (ST.hasStoreFusion())
     DAG->addMutation(createStoreClusterDAGMutation(DAG->TII, DAG->TRI));
   if (ST.hasFusion())
-    DAG->addMutation(createPowerPCMacroFusionDAGMutation());
+    DAG->addMutation(createPowerPCMacroFusionDAGMutation(/*IsPostRA=*/true));
   return DAG;
 }
 
diff --git a/llvm/lib/Target/RISCV/RISCVMacroFusion.cpp b/llvm/lib/Target/RISCV/RISCVMacroFusion.cpp
index f948f05b22f772..c1174503039fb2 100644
--- a/llvm/lib/Target/RISCV/RISCVMacroFusion.cpp
+++ b/llvm/lib/Target/RISCV/RISCVMacroFusion.cpp
@@ -18,7 +18,8 @@
 
 using namespace llvm;
 
-static bool checkRegisters(Register FirstDest, const MachineInstr &SecondMI) {
+static bool checkRegisters(Register FirstDest, const MachineInstr &SecondMI,
+                           bool IsPostRA) {
   if (!SecondMI.getOperand(1).isReg())
     return false;
 
@@ -26,7 +27,7 @@ static bool checkRegisters(Register FirstDest, const MachineInstr &SecondMI) {
     return false;
 
   // If the input is virtual make sure this is the only user.
-  if (FirstDest.isVirtual()) {
+  if (!IsPostRA) {
     auto &MRI = SecondMI.getMF()->getRegInfo();
     return MRI.hasOneNonDBGUse(FirstDest);
   }
@@ -37,7 +38,8 @@ static bool checkRegisters(Register FirstDest, const MachineInstr &SecondMI) {
 // Fuse load with add:
 // add rd, rs1, rs2
 // ld rd, 0(rd)
-static bool isLDADD(const MachineInstr *FirstMI, const MachineInstr &SecondMI) {
+static bool isLDADD(const MachineInstr *FirstMI, const MachineInstr &SecondMI,
+                    bool IsPostRA) {
   if (SecondMI.getOpcode() != RISCV::LD)
     return false;
 
@@ -55,13 +57,14 @@ static bool isLDADD(const MachineInstr *FirstMI, const MachineInstr &SecondMI) {
   if (FirstMI->getOpcode() != RISCV::ADD)
     return true;
 
-  return checkRegisters(FirstMI->getOperand(0).getReg(), SecondMI);
+  return checkRegisters(FirstMI->getOperand(0).getReg(), SecondMI, IsPostRA);
 }
 
 // Fuse zero extension of halfword:
 // slli rd, rs1, 48
 // srli rd, rd, 48
-static bool isZExtH(const MachineInstr *FirstMI, const MachineInstr &SecondMI) {
+static bool isZExtH(const MachineInstr *FirstMI, const MachineInstr &SecondMI,
+                    bool IsPostRA) {
   if (SecondMI.getOpcode() != RISCV::SRLI)
     return false;
 
@@ -82,13 +85,14 @@ static bool isZExtH(const MachineInstr *FirstMI, const MachineInstr &SecondMI) {
   if (FirstMI->getOperand(2).getImm() != 48)
     return false;
 
-  return checkRegisters(FirstMI->getOperand(0).getReg(), SecondMI);
+  return checkRegisters(FirstMI->getOperand(0).getReg(), SecondMI, IsPostRA);
 }
 
 // Fuse zero extension of word:
 // slli rd, rs1, 32
 // srli rd, rd, 32
-static bool isZExtW(const MachineInstr *FirstMI, const MachineInstr &SecondMI) {
+static bool isZExtW(const MachineInstr *FirstMI, const MachineInstr &SecondMI,
+                    bool IsPostRA) {
   if (SecondMI.getOpcode() != RISCV::SRLI)
     return false;
 
@@ -109,7 +113,7 @@ static bool isZExtW(const MachineInstr *FirstMI, const MachineInstr &SecondMI) {
   if (FirstMI->getOperand(2).getImm() != 32)
     return false;
 
-  return checkRegisters(FirstMI->getOperand(0).getReg(), SecondMI);
+  return checkRegisters(FirstMI->getOperand(0).getReg(), SecondMI, IsPostRA);
 }
 
 // Fuse shifted zero extension of word:
@@ -117,7 +121,7 @@ static bool isZExtW(const MachineInstr *FirstMI, const MachineInstr &SecondMI) {
 // srli rd, rd, x
 // where 0 <= x < 32
 static bool isShiftedZExtW(const MachineInstr *FirstMI,
-                           const MachineInstr &SecondMI) {
+                           const MachineInstr &SecondMI, bool IsPostRA) {
   if (SecondMI.getOpcode() != RISCV::SRLI)
     return false;
 
@@ -139,14 +143,14 @@ static bool isShiftedZExtW(const MachineInstr *FirstMI,
   if (FirstMI->getOperand(2).getImm() != 32)
     return false;
 
-  return checkReg...
[truncated]

Copy link

github-actions bot commented Jan 10, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

@wangpc-pp wangpc-pp force-pushed the main-macro-fusion-pre-post-regalloc branch from 7ec7039 to 87d30f8 Compare January 10, 2024 08:21
@topperc
Copy link
Collaborator

topperc commented Jan 10, 2024

Could we just check the novregs property on the MachineFunction?

@wangpc-pp
Copy link
Contributor Author

Could we just check the novregs property on the MachineFunction?

Oh I don't know there is such property. Thanks! I will try!

This can save some time to know whether MacroFusion mutation is
running in post-ra scheduler.

And this can be used in llvm#77461.
@wangpc-pp wangpc-pp force-pushed the main-macro-fusion-pre-post-regalloc branch from 04ff07f to 8a038f5 Compare January 10, 2024 17:24
<< "if (FirstDest.isVirtual() && !MRI.hasOneNonDBGUse(FirstDest))\n";
OS.indent(4) << " return false;\n";
OS.indent(2) << "}\n";
OS.indent(2) << "if (!IsPostRA && "
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why this is an improvement over checking isVirtual? Are we even guaranteed that a macro-fused instruction can't have a physical register in the pre-RA scheduler?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right, I'm thoughtless about this. I'll revert this part.

@topperc
Copy link
Collaborator

topperc commented Jan 10, 2024

And this can be used in #77461.

How will this be used in that patch?

@wangpc-pp
Copy link
Contributor Author

And this can be used in #77461.

How will this be used in that patch?

We only need to mark fusible pairs as Fusible in MIFlag in pre-ra scheduling. We don't care about it after RA.

@wangpc-pp
Copy link
Contributor Author

Close as I think we don't need it.

@wangpc-pp wangpc-pp closed this Jan 11, 2024
@wangpc-pp wangpc-pp deleted the main-macro-fusion-pre-post-regalloc branch January 11, 2024 06:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants