Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TableGen] Enhance testability of TableGen-based macro fusion #73075

Closed

Conversation

wangpc-pp
Copy link
Contributor

We add an option -mfusion like -mattr to add/remove supported
macro fusions, so that we can test each macro fusion separately via
llc.

This PR is stacked on #72219, #72222, #72223

The user can provide multiple predicators to MacroFusion and the
DAG mutation will be applied if one of them is evalated to true.

`ShouldSchedulePredTy` is renamed to `MacroFusionPredTy`.
`FusionPredicate` is used to predicate if target instruction matches
the requirement. The targets can be firstMI, secondMI or both.

The `Fusion` contains a list of `FusionPredicate`. The generated code
will be like:
```
bool isNAME(const TargetInstrInfo &TII,
            const TargetSubtargetInfo &STI,
            const MachineInstr *FirstMI,
            const MachineInstr &SecondMI) {
  auto &MRI = SecondMI.getMF()->getRegInfo();
  /* Predicates */
  return true;
}
```

A boilerplate class called `SimpleFusion` is added. `SimpleFusion` has
a predefined structure of predicates and accepts predicate for
`firstMI`, predicate for `secondMI` and epilog/prolog as arguments.
The generated code for `SimpleFusion` will be like:
```
bool isNAME(const TargetInstrInfo &TII,
            const TargetSubtargetInfo &STI,
            const MachineInstr *FirstMI,
            const MachineInstr &SecondMI) {
  auto &MRI = SecondMI.getMF()->getRegInfo();
  /* Prolog */
  /* Predicate for `SecondMI` */
  /* Wildcard */
  /* Predicate for `FirstMI` */
  /* Check One Use */
  /* Tie registers */
  /* Epilog */
  return true;
}
```
A list of `MacroFusion` is added to `SchedMachineModel` to indicate
supported macro fusions of this processor model.

For visibility, C++ enums of all MacroFusion are generated and new
`hasMacroFusion` is added to Subtarget to test if a MacroFusion is
supported.
@llvmbot
Copy link
Collaborator

llvmbot commented Nov 22, 2023

@llvm/pr-subscribers-backend-x86
@llvm/pr-subscribers-mc
@llvm/pr-subscribers-backend-amdgpu
@llvm/pr-subscribers-backend-arm

@llvm/pr-subscribers-backend-aarch64

Author: Wang Pengcheng (wangpc-pp)

Changes

We add an option -mfusion like -mattr to add/remove supported
macro fusions, so that we can test each macro fusion separately via
llc.

This PR is stacked on #72219, #72222, #72223


Patch is 44.76 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/73075.diff

25 Files Affected:

  • (modified) llvm/include/llvm/CodeGen/MacroFusion.h (+9-9)
  • (modified) llvm/include/llvm/CodeGen/TargetSubtargetInfo.h (+27-1)
  • (modified) llvm/include/llvm/MC/MCSchedule.h (+9)
  • (modified) llvm/include/llvm/MC/MCSubtargetInfo.h (+14)
  • (modified) llvm/include/llvm/Target/TargetInstrPredicate.td (+6)
  • (modified) llvm/include/llvm/Target/TargetSchedule.td (+118)
  • (modified) llvm/lib/CodeGen/MacroFusion.cpp (+27-12)
  • (modified) llvm/lib/CodeGen/TargetSubtargetInfo.cpp (+58-2)
  • (modified) llvm/lib/MC/MCSchedule.cpp (+1)
  • (modified) llvm/lib/MC/MCSubtargetInfo.cpp (+2)
  • (modified) llvm/lib/Target/AArch64/AArch64MacroFusion.cpp (+1-1)
  • (modified) llvm/lib/Target/AMDGPU/AMDGPUMacroFusion.cpp (+2-2)
  • (modified) llvm/lib/Target/AMDGPU/GCNVOPDUtils.cpp (+2-2)
  • (modified) llvm/lib/Target/ARM/ARMMacroFusion.cpp (+2-2)
  • (modified) llvm/lib/Target/PowerPC/PPCMacroFusion.cpp (+2-2)
  • (modified) llvm/lib/Target/RISCV/RISCVMacroFusion.cpp (+1-1)
  • (modified) llvm/lib/Target/X86/X86MacroFusion.cpp (+2-3)
  • (modified) llvm/unittests/CodeGen/MFCommon.inc (+1-1)
  • (modified) llvm/utils/TableGen/CMakeLists.txt (+1)
  • (modified) llvm/utils/TableGen/CodeGenSchedule.cpp (+15)
  • (modified) llvm/utils/TableGen/CodeGenSchedule.h (+11)
  • (added) llvm/utils/TableGen/MacroFusionPredicatorEmitter.cpp (+229)
  • (modified) llvm/utils/TableGen/PredicateExpander.cpp (+8)
  • (modified) llvm/utils/TableGen/PredicateExpander.h (+1)
  • (modified) llvm/utils/TableGen/SubtargetEmitter.cpp (+55-5)
diff --git a/llvm/include/llvm/CodeGen/MacroFusion.h b/llvm/include/llvm/CodeGen/MacroFusion.h
index ea2c7a5faae385a9..a97f776335368c7c 100644
--- a/llvm/include/llvm/CodeGen/MacroFusion.h
+++ b/llvm/include/llvm/CodeGen/MacroFusion.h
@@ -14,8 +14,8 @@
 #ifndef LLVM_CODEGEN_MACROFUSION_H
 #define LLVM_CODEGEN_MACROFUSION_H
 
-#include <functional>
 #include <memory>
+#include <vector>
 
 namespace llvm {
 
@@ -29,10 +29,10 @@ class SUnit;
 /// Check if the instr pair, FirstMI and SecondMI, should be fused
 /// together. Given SecondMI, when FirstMI is unspecified, then check if
 /// SecondMI may be part of a fused pair at all.
-using ShouldSchedulePredTy = std::function<bool(const TargetInstrInfo &TII,
-                                                const TargetSubtargetInfo &TSI,
-                                                const MachineInstr *FirstMI,
-                                                const MachineInstr &SecondMI)>;
+using MacroFusionPredTy = bool (*)(const TargetInstrInfo &TII,
+                                   const TargetSubtargetInfo &STI,
+                                   const MachineInstr *FirstMI,
+                                   const MachineInstr &SecondMI);
 
 /// Checks if the number of cluster edges between SU and its predecessors is
 /// less than FuseLimit
@@ -48,15 +48,15 @@ bool fuseInstructionPair(ScheduleDAGInstrs &DAG, SUnit &FirstSU,
 
 /// Create a DAG scheduling mutation to pair instructions back to back
 /// for instructions that benefit according to the target-specific
-/// shouldScheduleAdjacent predicate function.
+/// predicate functions.
 std::unique_ptr<ScheduleDAGMutation>
-createMacroFusionDAGMutation(ShouldSchedulePredTy shouldScheduleAdjacent);
+createMacroFusionDAGMutation(std::vector<MacroFusionPredTy> Predicates);
 
 /// Create a DAG scheduling mutation to pair branch instructions with one
 /// of their predecessors back to back for instructions that benefit according
-/// to the target-specific shouldScheduleAdjacent predicate function.
+/// to the target-specific predicate functions.
 std::unique_ptr<ScheduleDAGMutation>
-createBranchMacroFusionDAGMutation(ShouldSchedulePredTy shouldScheduleAdjacent);
+createBranchMacroFusionDAGMutation(std::vector<MacroFusionPredTy> Predicates);
 
 } // end namespace llvm
 
diff --git a/llvm/include/llvm/CodeGen/TargetSubtargetInfo.h b/llvm/include/llvm/CodeGen/TargetSubtargetInfo.h
index 55ef95c285431906..7091776b04a445a7 100644
--- a/llvm/include/llvm/CodeGen/TargetSubtargetInfo.h
+++ b/llvm/include/llvm/CodeGen/TargetSubtargetInfo.h
@@ -16,6 +16,7 @@
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/CodeGen/MacroFusion.h"
 #include "llvm/CodeGen/PBQPRAConstraint.h"
 #include "llvm/CodeGen/SchedulerRegistry.h"
 #include "llvm/IR/GlobalValue.h"
@@ -53,6 +54,22 @@ class TargetRegisterInfo;
 class TargetSchedModel;
 class Triple;
 
+//===----------------------------------------------------------------------===//
+
+/// Used to provide information for macro fusion.
+struct MacroFusionEntry {
+  const char *Name;       ///< Name of macro fusion
+  MacroFusionPredTy Pred; ///< Predicator function of macro fusion
+
+  /// Compare routine for std::lower_bound
+  bool operator<(StringRef S) const { return StringRef(Name) < S; }
+
+  /// Compare routine for std::is_sorted.
+  bool operator<(const MacroFusionEntry &Other) const {
+    return StringRef(Name) < StringRef(Other.Name);
+  }
+};
+
 //===----------------------------------------------------------------------===//
 ///
 /// TargetSubtargetInfo - Generic base class for all target subtargets.  All
@@ -60,6 +77,9 @@ class Triple;
 /// be exposed through a TargetSubtargetInfo-derived class.
 ///
 class TargetSubtargetInfo : public MCSubtargetInfo {
+private:
+  ArrayRef<MacroFusionEntry> MacroFusionTable;
+
 protected: // Can only create subclasses...
   TargetSubtargetInfo(const Triple &TT, StringRef CPU, StringRef TuneCPU,
                       StringRef FS, ArrayRef<SubtargetFeatureKV> PF,
@@ -67,7 +87,10 @@ class TargetSubtargetInfo : public MCSubtargetInfo {
                       const MCWriteProcResEntry *WPR,
                       const MCWriteLatencyEntry *WL,
                       const MCReadAdvanceEntry *RA, const InstrStage *IS,
-                      const unsigned *OC, const unsigned *FP);
+                      const unsigned *OC, const unsigned *FP,
+                      ArrayRef<MacroFusionEntry> MF);
+
+  void overrideFusionBits();
 
 public:
   // AntiDepBreakMode - Type of anti-dependence breaking that should
@@ -323,6 +346,9 @@ class TargetSubtargetInfo : public MCSubtargetInfo {
   /// helps removing redundant copies generated by register allocator when
   /// handling complex eviction chains.
   virtual bool enableSpillageCopyElimination() const { return false; }
+
+  /// Get the list of MacroFusion predicates.
+  virtual std::vector<MacroFusionPredTy> getMacroFusions() const;
 };
 
 } // end namespace llvm
diff --git a/llvm/include/llvm/MC/MCSchedule.h b/llvm/include/llvm/MC/MCSchedule.h
index 98ebe42cfd133b54..aa187e5cb400672d 100644
--- a/llvm/include/llvm/MC/MCSchedule.h
+++ b/llvm/include/llvm/MC/MCSchedule.h
@@ -14,6 +14,7 @@
 #ifndef LLVM_MC_MCSCHEDULE_H
 #define LLVM_MC_MCSCHEDULE_H
 
+#include "llvm/ADT/Bitset.h"
 #include "llvm/Config/llvm-config.h"
 #include "llvm/Support/DataTypes.h"
 #include <cassert>
@@ -196,6 +197,9 @@ struct MCExtraProcessorInfo {
   unsigned StoreQueueID;
 };
 
+const unsigned MaxMacroFusions = 256;
+using MacroFusionBitset = Bitset<MaxMacroFusions>;
+
 /// Machine model for scheduling, bundling, and heuristics.
 ///
 /// The machine model directly provides basic information about the
@@ -325,9 +329,14 @@ struct MCSchedModel {
   const InstrItinerary *InstrItineraries;
 
   const MCExtraProcessorInfo *ExtraProcessorInfo;
+  const MacroFusionBitset *MacroFusionBits;
 
   bool hasExtraProcessorInfo() const { return ExtraProcessorInfo; }
 
+  const MacroFusionBitset *getMacroFusionBits() const {
+    return MacroFusionBits;
+  }
+
   unsigned getProcessorID() const { return ProcID; }
 
   /// Does this machine model include instruction-level scheduling.
diff --git a/llvm/include/llvm/MC/MCSubtargetInfo.h b/llvm/include/llvm/MC/MCSubtargetInfo.h
index f172a799aa3331c8..1ae5134f047198fc 100644
--- a/llvm/include/llvm/MC/MCSubtargetInfo.h
+++ b/llvm/include/llvm/MC/MCSubtargetInfo.h
@@ -92,6 +92,8 @@ class MCSubtargetInfo {
   FeatureBitset FeatureBits;           // Feature bits for current CPU + FS
   std::string FeatureString;           // Feature string
 
+  MacroFusionBitset FusionBits; // Fusion bits
+
 public:
   MCSubtargetInfo(const MCSubtargetInfo &) = default;
   MCSubtargetInfo(const Triple &TT, StringRef CPU, StringRef TuneCPU,
@@ -120,6 +122,15 @@ class MCSubtargetInfo {
     return FeatureBits[Feature];
   }
 
+  const MacroFusionBitset &getMacroFusionBits() const { return FusionBits; }
+  void enableMacroFusion(unsigned MacroFusion) { FusionBits.set(MacroFusion); }
+  void disableMacroFusion(unsigned MacroFusion) {
+    FusionBits.reset(MacroFusion);
+  }
+  bool hasMacroFusion(unsigned MacroFusion) const {
+    return FusionBits.test(MacroFusion);
+  }
+
 protected:
   /// Initialize the scheduling model and feature bits.
   ///
@@ -295,6 +306,9 @@ class MCSubtargetInfo {
 
   /// \return if target want to issue a prefetch in address space \p AS.
   virtual bool shouldPrefetchAddressSpace(unsigned AS) const;
+
+  /// Enable macro fusion for this subtarget.
+  virtual bool enableMacroFusion() const { return FusionBits.any(); }
 };
 
 } // end namespace llvm
diff --git a/llvm/include/llvm/Target/TargetInstrPredicate.td b/llvm/include/llvm/Target/TargetInstrPredicate.td
index 9f2cde9d923050a8..82c4c7b23a49b6ac 100644
--- a/llvm/include/llvm/Target/TargetInstrPredicate.td
+++ b/llvm/include/llvm/Target/TargetInstrPredicate.td
@@ -95,6 +95,12 @@ class MCOperandPredicate<int Index> : MCInstPredicate {
 // Return true if machine operand at position `Index` is a register operand.
 class CheckIsRegOperand<int Index> : MCOperandPredicate<Index>;
 
+// Return true if machine operand at position `Index` is a virtual register operand.
+class CheckIsVRegOperand<int Index> : MCOperandPredicate<Index>;
+
+// Return true if machine operand at position `Index` is not a virtual register operand.
+class CheckIsNotVRegOperand<int Index> : CheckNot<CheckIsVRegOperand<Index>>;
+
 // Return true if machine operand at position `Index` is an immediate operand.
 class CheckIsImmOperand<int Index> : MCOperandPredicate<Index>;
 
diff --git a/llvm/include/llvm/Target/TargetSchedule.td b/llvm/include/llvm/Target/TargetSchedule.td
index 949baa5d2105c451..e81b6048b939ea59 100644
--- a/llvm/include/llvm/Target/TargetSchedule.td
+++ b/llvm/include/llvm/Target/TargetSchedule.td
@@ -53,6 +53,7 @@
 include "llvm/Target/TargetItinerary.td"
 
 class Predicate; // Forward def
+class Fusion;
 
 // DAG operator that interprets the DAG args as Instruction defs.
 def instrs;
@@ -122,6 +123,9 @@ class SchedMachineModel {
   // using intervals via ResourceSegments (see
   // llvm/include/llvm/CodeGen/MachineScheduler.h).
   bit EnableIntervals = false;
+
+  // List of Fusion.
+  list<Fusion> MacroFusions = [];
 }
 
 def NoSchedModel : SchedMachineModel {
@@ -584,3 +588,117 @@ class MemoryQueue<ProcResourceKind PR> {
 
 class LoadQueue<ProcResourceKind LDQueue> : MemoryQueue<LDQueue>;
 class StoreQueue<ProcResourceKind STQueue> : MemoryQueue<STQueue>;
+
+// The target instruction that FusionPredicate will evaluate on.
+class FusionTarget;
+def first : FusionTarget;
+def second : FusionTarget;
+def both : FusionTarget;
+
+// Base class of FusionPredicate, etc. The avaliable variables are:
+// * const TargetInstrInfo &TII
+// * const TargetSubtargetInfo &STI
+// * const MachineRegisterInfo &MRI
+// * const MachineInstr *FirstMI
+// * const MachineInstr &SecondMI
+class FusionPredicate<FusionTarget target> {
+  FusionTarget Target = target;
+}
+class FirstFusionPredicate: FusionPredicate<first>;
+class SecondFusionPredicate: FusionPredicate<second>;
+class BothFusionPredicate: FusionPredicate<both>;
+
+// FusionPredicate with raw code predicate.
+class FusionPredicateWithCode<code pred> : FusionPredicate<both> {
+  code Predicate = pred;
+}
+
+// FusionPredicate with MCInstPredicate.
+class FusionPredicateWithMCInstPredicate<FusionTarget target, MCInstPredicate pred>
+  : FusionPredicate<target> {
+  MCInstPredicate Predicate = pred;
+}
+class FirstFusionPredicateWithMCInstPredicate<MCInstPredicate pred>
+  : FusionPredicateWithMCInstPredicate<first, pred>;
+class SecondFusionPredicateWithMCInstPredicate<MCInstPredicate pred>
+  : FusionPredicateWithMCInstPredicate<second, pred>;
+// The pred will be applied on both firstMI and secondMI.
+class BothFusionPredicateWithMCInstPredicate<MCInstPredicate pred>
+  : FusionPredicateWithMCInstPredicate<second, pred>;
+
+// Tie firstOpIdx and secondOpIdx. The operand of `FirstMI` at position
+// `firstOpIdx` should be the same as the operand of `SenondMI` at position
+// `secondOpIdx`.
+class TieReg<int firstOpIdx, int secondOpIdx> : BothFusionPredicate {
+  int FirstOpIdx = firstOpIdx;
+  int SecondOpIdx = secondOpIdx;
+}
+
+// A predicate for wildcard. The generated code will be like:
+// ```
+// if (!FirstMI)
+//   return ReturnValue;
+// ```
+class WildcardPred<bit ret> : FirstFusionPredicate {
+  bit ReturnValue = ret;
+}
+def WildcardFalse : WildcardPred<0>;
+def WildcardTrue : WildcardPred<1>;
+
+// Indicates that the destination register of `FirstMI` should be have one
+// use if it is an virtual register.
+class OneUsePred : FirstFusionPredicate;
+def OneUse : OneUsePred;
+
+// Handled by MacroFusionPredicatorEmitter backend.
+// The generated predicator will be like:
+// ```
+// bool isNAME(const TargetInstrInfo &TII,
+//             const TargetSubtargetInfo &STI,
+//             const MachineInstr *FirstMI,
+//             const MachineInstr &SecondMI) {
+//   auto &MRI = SecondMI.getMF()->getRegInfo();
+//   /* Predicates */
+//   return true;
+// }
+// ```
+class Fusion<string name, list<FusionPredicate> predicates> {
+  string Name = name;
+  list<FusionPredicate> Predicates = predicates;
+}
+
+// The generated predicator will be like:
+// ```
+// bool isNAME(const TargetInstrInfo &TII,
+//             const TargetSubtargetInfo &STI,
+//             const MachineInstr *FirstMI,
+//             const MachineInstr &SecondMI) {
+//   auto &MRI = SecondMI.getMF()->getRegInfo();
+//   /* Prolog */
+//   /* Predicate for `SecondMI` */
+//   /* Wildcard */
+//   /* Predicate for `FirstMI` */
+//   /* Check One Use */
+//   /* Tie registers */
+//   /* Epilog */
+//   return true;
+// }
+// ```
+class SimpleFusion<string name, MCInstPredicate firstPred, MCInstPredicate secondPred,
+                   list<FusionPredicate> prolog = [],
+                   list<FusionPredicate> epilog = []>
+  : Fusion<name, !listconcat(
+                    prolog,
+                    [
+                      SecondFusionPredicateWithMCInstPredicate<secondPred>,
+                      WildcardTrue,
+                      FirstFusionPredicateWithMCInstPredicate<firstPred>,
+                      SecondFusionPredicateWithMCInstPredicate<
+                        CheckAny<[
+                          CheckIsVRegOperand<0>,
+                          CheckSameRegOperand<0, 1>
+                        ]>>,
+                      OneUse,
+                      TieReg<0, 1>,
+                    ],
+                    epilog)>;
diff --git a/llvm/lib/CodeGen/MacroFusion.cpp b/llvm/lib/CodeGen/MacroFusion.cpp
index fa5df68b8abcc0f6..1ce2f49763b076fa 100644
--- a/llvm/lib/CodeGen/MacroFusion.cpp
+++ b/llvm/lib/CodeGen/MacroFusion.cpp
@@ -137,19 +137,35 @@ namespace {
 /// Post-process the DAG to create cluster edges between instrs that may
 /// be fused by the processor into a single operation.
 class MacroFusion : public ScheduleDAGMutation {
-  ShouldSchedulePredTy shouldScheduleAdjacent;
+  std::vector<MacroFusionPredTy> Predicates;
   bool FuseBlock;
   bool scheduleAdjacentImpl(ScheduleDAGInstrs &DAG, SUnit &AnchorSU);
 
 public:
-  MacroFusion(ShouldSchedulePredTy shouldScheduleAdjacent, bool FuseBlock)
-    : shouldScheduleAdjacent(shouldScheduleAdjacent), FuseBlock(FuseBlock) {}
+  MacroFusion(std::vector<MacroFusionPredTy> Predicates, bool FuseBlock)
+      : Predicates(std::move(Predicates)), FuseBlock(FuseBlock) {}
 
   void apply(ScheduleDAGInstrs *DAGInstrs) override;
+
+  bool shouldScheduleAdjacent(const TargetInstrInfo &TII,
+                              const TargetSubtargetInfo &STI,
+                              const MachineInstr *FirstMI,
+                              const MachineInstr &SecondMI);
 };
 
 } // end anonymous namespace
 
+bool MacroFusion::shouldScheduleAdjacent(const TargetInstrInfo &TII,
+                                         const TargetSubtargetInfo &STI,
+                                         const MachineInstr *FirstMI,
+                                         const MachineInstr &SecondMI) {
+  for (MacroFusionPredTy Predicate : Predicates) {
+    if (Predicate(TII, STI, FirstMI, SecondMI))
+      return true;
+  }
+  return false;
+}
+
 void MacroFusion::apply(ScheduleDAGInstrs *DAG) {
   if (FuseBlock)
     // For each of the SUnits in the scheduling block, try to fuse the instr in
@@ -197,17 +213,16 @@ bool MacroFusion::scheduleAdjacentImpl(ScheduleDAGInstrs &DAG, SUnit &AnchorSU)
 }
 
 std::unique_ptr<ScheduleDAGMutation>
-llvm::createMacroFusionDAGMutation(
-     ShouldSchedulePredTy shouldScheduleAdjacent) {
-  if(EnableMacroFusion)
-    return std::make_unique<MacroFusion>(shouldScheduleAdjacent, true);
+llvm::createMacroFusionDAGMutation(std::vector<MacroFusionPredTy> Predicates) {
+  if (EnableMacroFusion) {
+    return std::make_unique<MacroFusion>(Predicates, true);
+  }
   return nullptr;
 }
 
-std::unique_ptr<ScheduleDAGMutation>
-llvm::createBranchMacroFusionDAGMutation(
-     ShouldSchedulePredTy shouldScheduleAdjacent) {
-  if(EnableMacroFusion)
-    return std::make_unique<MacroFusion>(shouldScheduleAdjacent, false);
+std::unique_ptr<ScheduleDAGMutation> llvm::createBranchMacroFusionDAGMutation(
+    std::vector<MacroFusionPredTy> Predicates) {
+  if (EnableMacroFusion)
+    return std::make_unique<MacroFusion>(Predicates, false);
   return nullptr;
 }
diff --git a/llvm/lib/CodeGen/TargetSubtargetInfo.cpp b/llvm/lib/CodeGen/TargetSubtargetInfo.cpp
index 6c97bc0568bdeeee..9e3f1f4171611ec5 100644
--- a/llvm/lib/CodeGen/TargetSubtargetInfo.cpp
+++ b/llvm/lib/CodeGen/TargetSubtargetInfo.cpp
@@ -14,16 +14,62 @@
 
 using namespace llvm;
 
+static cl::list<std::string> MFusions("mfusion", cl::CommaSeparated,
+                                      cl::desc("Target specific macro fusions"),
+                                      cl::value_desc("a1,+a2,-a3,..."));
+
 TargetSubtargetInfo::TargetSubtargetInfo(
     const Triple &TT, StringRef CPU, StringRef TuneCPU, StringRef FS,
     ArrayRef<SubtargetFeatureKV> PF, ArrayRef<SubtargetSubTypeKV> PD,
     const MCWriteProcResEntry *WPR, const MCWriteLatencyEntry *WL,
     const MCReadAdvanceEntry *RA, const InstrStage *IS, const unsigned *OC,
-    const unsigned *FP)
-    : MCSubtargetInfo(TT, CPU, TuneCPU, FS, PF, PD, WPR, WL, RA, IS, OC, FP) {}
+    const unsigned *FP, ArrayRef<MacroFusionEntry> MF)
+    : MCSubtargetInfo(TT, CPU, TuneCPU, FS, PF, PD, WPR, WL, RA, IS, OC, FP),
+      MacroFusionTable(MF) {
+  // assert if MacroFusionTable is not sorted.
+  assert(llvm::is_sorted(MacroFusionTable));
+  overrideFusionBits();
+}
 
 TargetSubtargetInfo::~TargetSubtargetInfo() = default;
 
+void TargetSubtargetInfo::overrideFusionBits() {
+  if (MFusions.getNumOccurrences() != 0) {
+    for (std::string &MFusion : MFusions) {
+      char Prefix = MFusion[0];
+      bool Disable = Prefix == '-';
+      if (Prefix == '+' || Prefix == '-')
+        MFusion = MFusion.substr(1);
+
+      // MacroFusionTable is sorted.
+      const auto *Pos = std::lower_bound(
+          MacroFusionTable.begin(), MacroFusionTable.end(), MFusion,
+          [](const MacroFusionEntry &LHS, const std::string &RHS) {
+            int CmpName = StringRef(LHS.Name).compare(RHS);
+            if (CmpName < 0)
+              return true;
+            if (CmpName > 0)
+              return false;
+            return false;
+          });
+
+      if (Pos == MacroFusionTable.end()) {
+        errs() << "'" << MFusion
+               << "' is not a recognized macro fusion for this "
+               << "target (ignoring it)\n";
+        continue;
+      }
+
+      // The index is the same as the enum value.
+      unsigned Idx = Pos - MacroFusionTable.begin();
+      if (Disable)
+        disableMacroFusion(Idx);
+      else
+        enableMacroFusion(Idx);
+    }
+  }
+}
+
 bool TargetSubtargetInfo::enableAtomicExpand() const {
   return true;
 }
@@ -58,3 +104,13 @@ bool TargetSubtargetInfo::useAA() const {
 }
 
 void TargetSubtargetInfo::mirFileLoaded(MachineFunction &MF) const { }
+
+std::vector<MacroFusionPredTy> TargetSubtargetInfo::getMacroFusions() const {
+  std::vector<MacroFusionPredTy> Fusions;
+  const MacroFusionBitset &Bits = getMacroFusionBits();
+  for (unsigned I = 0; I < MacroFusionTable.size(); I++)
+    if (Bits[I])
+      Fusions.push_back(MacroFusionTable[I].Pred);
+
+  return Fusions;
+}
diff --git a/llvm/lib/MC/MCSchedule.cpp b/llvm/lib/MC/MCSchedule.cpp
index 990a693559a77769..19c36cb0e58d9c10 100644
--- a/llvm/lib/MC/MCSchedule.cpp
+++ b/llvm/lib/MC/MCSchedule.cpp
@@ -37,6 +37,7 @@ const MCSchedModel MCSchedModel::Default = {DefaultIssueWidth,
                                             0,
                                             0,
                                             nullptr,
+                                            nullptr,
                                             nullptr};
 
 int MCSchedModel::computeInstrLatency(const MCSubtargetInfo &STI,
diff --git a/llvm/lib/MC/MCSubtargetInfo.cpp b/llvm/lib/MC/MCSubtargetInfo.cpp
index 8ee823e0377b730e..8ea1aca92e048df1 100644
--- a/llvm/lib/MC/MCSubtargetInfo.cpp
+++ b/llvm/lib/MC/MCSubtargetInfo.cpp
@@ -215,6 +215,8 @@ void MCSubtargetInfo::InitMCProcessorInfo(StringRef CPU, StringRef TuneCPU,
     CPUSchedModel = &getSchedModelForCPU(TuneCPU);
   else
     CPUSchedMode...
[truncated]

Copy link

github-actions bot commented Nov 22, 2023

✅ With the latest revision this PR passed the C/C++ code formatter.

We add an option `-mfusion` like `-mattr` to add/remove supported
macro fusions, so that we can test each macro fusion separately via
`llc`.

This PR is stacked on llvm#72219, llvm#72222, llvm#72223
const auto *Pos = std::lower_bound(
MacroFusionTable.begin(), MacroFusionTable.end(), MFusion,
[](const MacroFusionEntry &LHS, const std::string &RHS) {
int CmpName = StringRef(LHS.Name).compare(RHS);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this just return String(LHS.Name).compare(RHS) < 0

@topperc
Copy link
Collaborator

topperc commented Nov 22, 2023

Could we have tblgen autogenerate the macrofusion bits onto the end of the existing subtarget feature bits and autogenerate has*() functions for them like other subtarget features. It seems like we're now duplicating some of the -mattr functionality.

@wangpc-pp
Copy link
Contributor Author

Could we have tblgen autogenerate the macrofusion bits onto the end of the existing subtarget feature bits and autogenerate has*() functions for them like other subtarget features. It seems like we're now duplicating some of the -mattr functionality.

Yes, I was thinking about making Fusion inherit from SubtargetFuture so that we can reuse current -mattr mechanism. My concern is that MAX_SUBTARGET_FEATURES is 320 now and there are more 170 subtarget features in RISCV (270 for AArch64, 223 for ARM and 183 for X86). For the foreseeable future, RISCV, as a modular ISA, may have more than 320 features. We may need to increase MAX_SUBTARGET_FEATURES in the near future if we add macrofusion bits to subtarget feature bits.
But it may not be a big problem, I will post another patch for this solution.

@wangpc-pp
Copy link
Contributor Author

Close this PR as I prefer #73115.

@wangpc-pp wangpc-pp closed this Nov 22, 2023
@wangpc-pp wangpc-pp deleted the main-tablegen-macro-fusion-testability branch March 29, 2024 04:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants