Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AMDGPU] Add synthetic regclass WWM_VGPR_32 for wwm reg operands #86008

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

cdevadas
Copy link
Collaborator

This new class will have the same set of registers as VGPR_32 class
but their actual allocatable set can vary. This synthetic class won't
be enabled until regalloc pipeline so that they won't be incorrectly
used for cases where actual VGPR_32 class machine operands are expected.
This class will be enabled during SGPR spill lowering to create virtual registers
for the operands of whole-wave mode instructions that handle the SGPR
spill to VGPRs lanes and will eventually get allocated during vgpr-regalloc
pipeline. After regalloc, this special class will be disabled.

Currently register classes can only be defined statically.
There is no way targets can define classes during codegen
for a specific period of time.

This patch includes a new field named `Synthetic` in the
regclass definition with value false by default. This
field, when set to true, indicates that such regclasses are
disabled by default and won't take part in various regclass
queries like getAllocatableClass,  getMinimalPhysRegClass,
etc. and in fact, they won't be allocatable, but targets can
enable them dynamically so that they behave like other register
classes.

The static definition of such classes are done in *RegisterInfo.td
file like other classes for tablegen to include them in all the
tables it auto-generates. The Synthetic field would be set to true
for such classes.
Some targets can have register classes with identical set
of registers. But their actual allocation order can vary.
The allocation mask computed based on target constraints
would help achieve this during register allocation.
This new class will have the same set of registers as VGPR_32 class
but their actual allocatable set can vary. This synthetic class won't
be enabled until regalloc pipeline, so that they won't be incorrectly
used for cases where actual VGPR_32 class machine operands are expected.
It will be enabled during SGPR spill lowering to create virtual registers
for the operands of whole-wave mode instructions that handle the SGPR
spill to VGPRs lanes and will eventually get allocated during vgpr-regalloc
pipeline. After regalloc, this special class will be disabled.
@llvmbot
Copy link
Collaborator

llvmbot commented Mar 20, 2024

@llvm/pr-subscribers-llvm-selectiondag
@llvm/pr-subscribers-backend-loongarch
@llvm/pr-subscribers-backend-arm
@llvm/pr-subscribers-backend-systemz
@llvm/pr-subscribers-backend-risc-v
@llvm/pr-subscribers-backend-x86

@llvm/pr-subscribers-llvm-regalloc

Author: Christudasan Devadasan (cdevadas)

Changes

This new class will have the same set of registers as VGPR_32 class
but their actual allocatable set can vary. This synthetic class won't
be enabled until regalloc pipeline so that they won't be incorrectly
used for cases where actual VGPR_32 class machine operands are expected.
This class will be enabled during SGPR spill lowering to create virtual registers
for the operands of whole-wave mode instructions that handle the SGPR
spill to VGPRs lanes and will eventually get allocated during vgpr-regalloc
pipeline. After regalloc, this special class will be disabled.


Patch is 624.13 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/86008.diff

134 Files Affected:

  • (modified) llvm/include/llvm/CodeGen/LiveStacks.h (+1)
  • (modified) llvm/include/llvm/CodeGen/MachineInstr.h (+5-3)
  • (modified) llvm/include/llvm/CodeGen/MachineRegisterInfo.h (+34)
  • (modified) llvm/include/llvm/CodeGen/RegisterBankInfo.h (+2-1)
  • (modified) llvm/include/llvm/CodeGen/TargetLowering.h (+1)
  • (modified) llvm/include/llvm/CodeGen/TargetRegisterInfo.h (+20-11)
  • (modified) llvm/include/llvm/MC/MCRegisterInfo.h (+4)
  • (modified) llvm/include/llvm/Target/Target.td (+6)
  • (modified) llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp (+9-5)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp (+4-2)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp (+5-2)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h (+3-1)
  • (modified) llvm/lib/CodeGen/DetectDeadLanes.cpp (+4-4)
  • (modified) llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp (+22-14)
  • (modified) llvm/lib/CodeGen/GlobalISel/Utils.cpp (+2-2)
  • (modified) llvm/lib/CodeGen/LiveStacks.cpp (+2-1)
  • (modified) llvm/lib/CodeGen/MIRParser/MIRParser.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/MachineCSE.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/MachineCombiner.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/MachineInstr.cpp (+12-8)
  • (modified) llvm/lib/CodeGen/MachineRegisterInfo.cpp (+45-3)
  • (modified) llvm/lib/CodeGen/MachineVerifier.cpp (+3-3)
  • (modified) llvm/lib/CodeGen/PeepholeOptimizer.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/PrologEpilogInserter.cpp (+6-3)
  • (modified) llvm/lib/CodeGen/RegAllocGreedy.cpp (+3-2)
  • (modified) llvm/lib/CodeGen/RegisterBank.cpp (+2-2)
  • (modified) llvm/lib/CodeGen/RegisterBankInfo.cpp (+6-4)
  • (modified) llvm/lib/CodeGen/RegisterClassInfo.cpp (+10-1)
  • (modified) llvm/lib/CodeGen/RegisterCoalescer.cpp (+8-8)
  • (modified) llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp (+2-1)
  • (modified) llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp (+12-10)
  • (modified) llvm/lib/CodeGen/SelectionDAG/ScheduleDAGFast.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/SelectionDAG/ScheduleDAGRRList.cpp (+1-2)
  • (modified) llvm/lib/CodeGen/SelectionDAG/ScheduleDAGSDNodes.cpp (+5-3)
  • (modified) llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp (+5-3)
  • (modified) llvm/lib/CodeGen/StackMaps.cpp (+4-2)
  • (modified) llvm/lib/CodeGen/TailDuplicator.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/TargetLoweringBase.cpp (+2-1)
  • (modified) llvm/lib/CodeGen/TargetRegisterInfo.cpp (+52-41)
  • (modified) llvm/lib/CodeGen/TwoAddressInstructionPass.cpp (+3-4)
  • (modified) llvm/lib/Target/AArch64/AArch64FrameLowering.cpp (+2-1)
  • (modified) llvm/lib/Target/AArch64/AArch64InstrInfo.cpp (+6-5)
  • (modified) llvm/lib/Target/AArch64/AArch64LoadStoreOptimizer.cpp (+16-9)
  • (modified) llvm/lib/Target/AMDGPU/GCNRewritePartialRegUses.cpp (+3-2)
  • (modified) llvm/lib/Target/AMDGPU/SIISelLowering.cpp (+4-3)
  • (modified) llvm/lib/Target/AMDGPU/SIISelLowering.h (+1)
  • (modified) llvm/lib/Target/AMDGPU/SIInstrInfo.cpp (+6-7)
  • (modified) llvm/lib/Target/AMDGPU/SIInstrInfo.h (+1-1)
  • (modified) llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp (+74-49)
  • (modified) llvm/lib/Target/AMDGPU/SILowerWWMCopies.cpp (+4-4)
  • (modified) llvm/lib/Target/AMDGPU/SIMachineFunctionInfo.cpp (+1-1)
  • (modified) llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp (+9-11)
  • (modified) llvm/lib/Target/AMDGPU/SIRegisterInfo.h (+4-3)
  • (modified) llvm/lib/Target/AMDGPU/SIRegisterInfo.td (+10)
  • (modified) llvm/lib/Target/ARM/ARMBaseRegisterInfo.cpp (+6-6)
  • (modified) llvm/lib/Target/ARM/ARMBaseRegisterInfo.h (+2-1)
  • (modified) llvm/lib/Target/AVR/AVRAsmPrinter.cpp (+2-1)
  • (modified) llvm/lib/Target/AVR/AVRFrameLowering.cpp (+4-2)
  • (modified) llvm/lib/Target/CSKY/CSKYFrameLowering.cpp (+4-2)
  • (modified) llvm/lib/Target/Hexagon/BitTracker.cpp (+1-1)
  • (modified) llvm/lib/Target/Hexagon/HexagonBitTracker.cpp (+2-1)
  • (modified) llvm/lib/Target/Hexagon/HexagonExpandCondsets.cpp (+1-1)
  • (modified) llvm/lib/Target/Hexagon/HexagonFrameLowering.cpp (+7-4)
  • (modified) llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp (+3-1)
  • (modified) llvm/lib/Target/Hexagon/HexagonVLIWPacketizer.cpp (+6-3)
  • (modified) llvm/lib/Target/Hexagon/RDFCopy.cpp (+5-3)
  • (modified) llvm/lib/Target/LoongArch/LoongArchFrameLowering.cpp (+2-1)
  • (modified) llvm/lib/Target/Mips/MipsFrameLowering.cpp (+2-1)
  • (modified) llvm/lib/Target/Mips/MipsSEFrameLowering.cpp (+4-2)
  • (modified) llvm/lib/Target/PowerPC/PPCFrameLowering.cpp (+3-2)
  • (modified) llvm/lib/Target/PowerPC/PPCInstrInfo.cpp (+4-4)
  • (modified) llvm/lib/Target/PowerPC/PPCRegisterInfo.cpp (+2-1)
  • (modified) llvm/lib/Target/RISCV/RISCVFrameLowering.cpp (+6-3)
  • (modified) llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp (+1-1)
  • (modified) llvm/lib/Target/SystemZ/SystemZFrameLowering.cpp (+4-2)
  • (modified) llvm/lib/Target/SystemZ/SystemZInstrInfo.cpp (+2-2)
  • (modified) llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp (+4-5)
  • (modified) llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.cpp (+1-1)
  • (modified) llvm/lib/Target/WebAssembly/WebAssemblyReplacePhysRegs.cpp (+1-1)
  • (modified) llvm/lib/Target/X86/X86FrameLowering.cpp (+6-3)
  • (modified) llvm/lib/Target/X86/X86InstrInfo.cpp (+2-2)
  • (modified) llvm/lib/Target/X86/X86RegisterInfo.cpp (+10-11)
  • (modified) llvm/lib/Target/X86/X86RegisterInfo.h (+4-3)
  • (modified) llvm/lib/Target/XCore/XCoreFrameLowering.cpp (+4-2)
  • (modified) llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll (+32-42)
  • (modified) llvm/test/CodeGen/AMDGPU/GlobalISel/irtranslator-inline-asm.ll (+7-7)
  • (modified) llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll (+5-8)
  • (modified) llvm/test/CodeGen/AMDGPU/cf-loop-on-constant.ll (+5-35)
  • (modified) llvm/test/CodeGen/AMDGPU/collapse-endcf.ll (+59-92)
  • (modified) llvm/test/CodeGen/AMDGPU/div_i128.ll (+245-255)
  • (modified) llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir (+33-38)
  • (modified) llvm/test/CodeGen/AMDGPU/flat-scratch-init.ll (+4-10)
  • (modified) llvm/test/CodeGen/AMDGPU/fold-reload-into-exec.mir (+12-12)
  • (modified) llvm/test/CodeGen/AMDGPU/fold-reload-into-m0.mir (+2-2)
  • (modified) llvm/test/CodeGen/AMDGPU/identical-subrange-spill-infloop.ll (+263-266)
  • (modified) llvm/test/CodeGen/AMDGPU/inline-asm.i128.ll (+12-12)
  • (modified) llvm/test/CodeGen/AMDGPU/kernel-vgpr-spill-mubuf-with-voffset.ll (+5-19)
  • (modified) llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll (+102-108)
  • (modified) llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll (+131-142)
  • (modified) llvm/test/CodeGen/AMDGPU/partial-regcopy-and-spill-missed-at-regalloc.ll (+8-8)
  • (modified) llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll (+334-386)
  • (modified) llvm/test/CodeGen/AMDGPU/preserve-wwm-copy-dst-reg.ll (+2-8)
  • (modified) llvm/test/CodeGen/AMDGPU/rem_i128.ll (+640-648)
  • (modified) llvm/test/CodeGen/AMDGPU/scc-clobbered-sgpr-to-vmem-spill.ll (+226-230)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spill-dead-frame-in-dbg-value.mir (+7-7)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll (+67-83)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir (+2-3)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spill-partially-undef.mir (+6-6)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spills-split-regalloc.ll (+1-9)
  • (modified) llvm/test/CodeGen/AMDGPU/si-spill-sgpr-stack.ll (+2-3)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-csr-frame-ptr-reg-copy.ll (+4-4)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-reg-tuple-super-reg-use.mir (+4-6)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-sgpr-to-virtual-vgpr.mir (+43-43)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-vector-superclass.ll (+1-1)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-vgpr-to-agpr-update-regscavenger.ll (+3-10)
  • (modified) llvm/test/CodeGen/AMDGPU/spill192.mir (+7-7)
  • (modified) llvm/test/CodeGen/AMDGPU/spill224.mir (+8-8)
  • (modified) llvm/test/CodeGen/AMDGPU/spill288.mir (+10-10)
  • (modified) llvm/test/CodeGen/AMDGPU/spill320.mir (+11-11)
  • (modified) llvm/test/CodeGen/AMDGPU/spill352.mir (+12-12)
  • (modified) llvm/test/CodeGen/AMDGPU/spill384.mir (+13-13)
  • (modified) llvm/test/CodeGen/AMDGPU/stacksave_stackrestore.ll (+13-32)
  • (modified) llvm/test/CodeGen/AMDGPU/vgpr-spill-placement-issue61083.ll (+8-15)
  • (modified) llvm/test/CodeGen/AMDGPU/whole-wave-register-copy.ll (+4-6)
  • (modified) llvm/test/CodeGen/AMDGPU/whole-wave-register-spill.ll (+10-14)
  • (modified) llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll (+25-32)
  • (modified) llvm/test/CodeGen/AMDGPU/wwm-reserved.ll (+174-254)
  • (modified) llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/amdgpu_isel.ll.expected (+2-2)
  • (modified) llvm/unittests/CodeGen/MFCommon.inc (+40-2)
  • (modified) llvm/unittests/CodeGen/MachineInstrTest.cpp (+11-2)
  • (modified) llvm/utils/TableGen/CodeGenRegisters.cpp (+5-2)
  • (modified) llvm/utils/TableGen/CodeGenRegisters.h (+1)
  • (modified) llvm/utils/TableGen/RegisterInfoEmitter.cpp (+2-1)
diff --git a/llvm/include/llvm/CodeGen/LiveStacks.h b/llvm/include/llvm/CodeGen/LiveStacks.h
index 2edc2985f0ee66..b9e55987383901 100644
--- a/llvm/include/llvm/CodeGen/LiveStacks.h
+++ b/llvm/include/llvm/CodeGen/LiveStacks.h
@@ -34,6 +34,7 @@ class TargetRegisterInfo;
 
 class LiveStacks : public MachineFunctionPass {
   const TargetRegisterInfo *TRI = nullptr;
+  const MachineRegisterInfo *MRI = nullptr;
 
   /// Special pool allocator for VNInfo's (LiveInterval val#).
   ///
diff --git a/llvm/include/llvm/CodeGen/MachineInstr.h b/llvm/include/llvm/CodeGen/MachineInstr.h
index fcdd73d8b65fdd..c7067d90a3ce06 100644
--- a/llvm/include/llvm/CodeGen/MachineInstr.h
+++ b/llvm/include/llvm/CodeGen/MachineInstr.h
@@ -1586,7 +1586,7 @@ class MachineInstr
   const TargetRegisterClass *getRegClassConstraintEffectForVReg(
       Register Reg, const TargetRegisterClass *CurRC,
       const TargetInstrInfo *TII, const TargetRegisterInfo *TRI,
-      bool ExploreBundle = false) const;
+      const MachineRegisterInfo &MRI, bool ExploreBundle = false) const;
 
   /// Applies the constraints (def/use) implied by the \p OpIdx operand
   /// to the given \p CurRC.
@@ -1600,7 +1600,8 @@ class MachineInstr
   const TargetRegisterClass *
   getRegClassConstraintEffect(unsigned OpIdx, const TargetRegisterClass *CurRC,
                               const TargetInstrInfo *TII,
-                              const TargetRegisterInfo *TRI) const;
+                              const TargetRegisterInfo *TRI,
+                              const MachineRegisterInfo &MRI) const;
 
   /// Add a tie between the register operands at DefIdx and UseIdx.
   /// The tie will cause the register allocator to ensure that the two
@@ -2005,7 +2006,8 @@ class MachineInstr
   /// If the related operand does not constrained Reg, this returns CurRC.
   const TargetRegisterClass *getRegClassConstraintEffectForVRegImpl(
       unsigned OpIdx, Register Reg, const TargetRegisterClass *CurRC,
-      const TargetInstrInfo *TII, const TargetRegisterInfo *TRI) const;
+      const TargetInstrInfo *TII, const TargetRegisterInfo *TRI,
+      const MachineRegisterInfo &MRI) const;
 
   /// Stores extra instruction information inline or allocates as ExtraInfo
   /// based on the number of pointers.
diff --git a/llvm/include/llvm/CodeGen/MachineRegisterInfo.h b/llvm/include/llvm/CodeGen/MachineRegisterInfo.h
index 09d9a0b4ec402f..78414f31d1bf82 100644
--- a/llvm/include/llvm/CodeGen/MachineRegisterInfo.h
+++ b/llvm/include/llvm/CodeGen/MachineRegisterInfo.h
@@ -94,6 +94,16 @@ class MachineRegisterInfo {
   /// all registers that were disabled are removed from the list.
   SmallVector<MCPhysReg, 16> UpdatedCSRs;
 
+  /// The Synthetic field of regclasses. Targets can alter this vector
+  /// to enable classes dynamically during codegen.
+  BitVector RegClassSyntheticInfo;
+
+  /// To have more restrictions on the allocation order for each register class.
+  /// For some targets, two distinct register classes can have the same set of
+  /// registers, but their allocatable set can vary. This vector helps targets
+  /// to dynamically determine the actual allocatable set for RCs.
+  std::vector<BitVector> RegClassAllocationMasks;
+
   /// RegAllocHints - This vector records register allocation hints for
   /// virtual registers. For each virtual register, it keeps a pair of hint
   /// type and hints vector making up the allocation hints. Only the first
@@ -257,6 +267,30 @@ class MachineRegisterInfo {
   /// Notice that it will override ant previously disabled/saved CSRs.
   void setCalleeSavedRegs(ArrayRef<MCPhysReg> CSRs);
 
+  /// Initialize the RegClassSyntheticInfo. It sets the bit position as
+  /// exactly as the tablegened Synthetic field. Targets can later flip this
+  /// field to enable/disable the regclass whenever required.
+  void initializeRegClassSyntheticInfo();
+
+  /// Change the synthetic info for the regclass \p RC from \p Value.
+  void changeSyntheticInfoForRC(const TargetRegisterClass *RC, bool Value);
+
+  /// This function checks if \p RC is enabled or not so that it can be included
+  /// in various regclass related queries.
+  bool isEnabled(const TargetRegisterClass *RC) const;
+
+  /// Update dynamically determined allocation mask for register classes.
+  void updateAllocationMaskForRCs(std::vector<BitVector> &&Mask);
+
+  // Return the allocation mask for regclass \p RC.
+  const BitVector &getAllocationMaskForRC(const TargetRegisterClass &RC) const;
+
+  /// True when the target has dynamically determined allocation mask for its
+  /// register classes.
+  bool hasAllocationMaskForRCs() const {
+    return RegClassAllocationMasks.size();
+  }
+
   // Strictly for use by MachineInstr.cpp.
   void addRegOperandToUseList(MachineOperand *MO);
 
diff --git a/llvm/include/llvm/CodeGen/RegisterBankInfo.h b/llvm/include/llvm/CodeGen/RegisterBankInfo.h
index 62c4a57a605d68..117f381512a1c6 100644
--- a/llvm/include/llvm/CodeGen/RegisterBankInfo.h
+++ b/llvm/include/llvm/CodeGen/RegisterBankInfo.h
@@ -445,7 +445,8 @@ class RegisterBankInfo {
   /// Get the MinimalPhysRegClass for Reg.
   /// \pre Reg is a physical register.
   const TargetRegisterClass *
-  getMinimalPhysRegClass(Register Reg, const TargetRegisterInfo &TRI) const;
+  getMinimalPhysRegClass(Register Reg, const TargetRegisterInfo &TRI,
+                         const MachineRegisterInfo &MRI) const;
 
   /// Try to get the mapping of \p MI.
   /// See getInstrMapping for more details on what a mapping represents.
diff --git a/llvm/include/llvm/CodeGen/TargetLowering.h b/llvm/include/llvm/CodeGen/TargetLowering.h
index 59fad88f91b1d1..5ef963cae55286 100644
--- a/llvm/include/llvm/CodeGen/TargetLowering.h
+++ b/llvm/include/llvm/CodeGen/TargetLowering.h
@@ -4384,6 +4384,7 @@ class TargetLowering : public TargetLoweringBase {
   /// Returns 'true' is the edge is necessary, 'false' otherwise
   virtual bool checkForPhysRegDependency(SDNode *Def, SDNode *User, unsigned Op,
                                          const TargetRegisterInfo *TRI,
+                                         const MachineRegisterInfo &MRI,
                                          const TargetInstrInfo *TII,
                                          unsigned &PhysReg, int &Cost) const {
     return false;
diff --git a/llvm/include/llvm/CodeGen/TargetRegisterInfo.h b/llvm/include/llvm/CodeGen/TargetRegisterInfo.h
index 117d3f71829747..a3d68d6743ede5 100644
--- a/llvm/include/llvm/CodeGen/TargetRegisterInfo.h
+++ b/llvm/include/llvm/CodeGen/TargetRegisterInfo.h
@@ -121,6 +121,9 @@ class TargetRegisterClass {
   /// Return true if this register class has a defined BaseClassOrder.
   bool isBaseClass() const { return MC->isBaseClass(); }
 
+  /// Return true if this register class is marked synthetic.
+  bool isSynthetic() const { return MC->isSynthetic(); }
+
   /// Return true if the specified TargetRegisterClass
   /// is a proper sub-class of this TargetRegisterClass.
   bool hasSubClass(const TargetRegisterClass *RC) const {
@@ -338,20 +341,23 @@ class TargetRegisterInfo : public MCRegisterInfo {
   /// Returns the Register Class of a physical register of the given type,
   /// picking the most sub register class of the right type that contains this
   /// physreg.
-  const TargetRegisterClass *getMinimalPhysRegClass(MCRegister Reg,
-                                                    MVT VT = MVT::Other) const;
+  const TargetRegisterClass *
+  getMinimalPhysRegClass(MCRegister Reg, const MachineRegisterInfo &MRI,
+                         MVT VT = MVT::Other) const;
 
   /// Returns the Register Class of a physical register of the given type,
   /// picking the most sub register class of the right type that contains this
   /// physreg. If there is no register class compatible with the given type,
   /// returns nullptr.
-  const TargetRegisterClass *getMinimalPhysRegClassLLT(MCRegister Reg,
-                                                       LLT Ty = LLT()) const;
+  const TargetRegisterClass *
+  getMinimalPhysRegClassLLT(MCRegister Reg, const MachineRegisterInfo &MRI,
+                            LLT Ty = LLT()) const;
 
   /// Return the maximal subclass of the given register class that is
   /// allocatable or NULL.
   const TargetRegisterClass *
-    getAllocatableClass(const TargetRegisterClass *RC) const;
+  getAllocatableClass(const TargetRegisterClass *RC,
+                      const MachineRegisterInfo &MRI) const;
 
   /// Returns a bitset indexed by register number indicating if a register is
   /// allocatable or not. If a register class is specified, returns the subset
@@ -630,7 +636,8 @@ class TargetRegisterInfo : public MCRegisterInfo {
   /// TableGen will synthesize missing A sub-classes.
   virtual const TargetRegisterClass *
   getMatchingSuperRegClass(const TargetRegisterClass *A,
-                           const TargetRegisterClass *B, unsigned Idx) const;
+                           const TargetRegisterClass *B, unsigned Idx,
+                           const MachineRegisterInfo &MRI) const;
 
   // For a copy-like instruction that defines a register of class DefRC with
   // subreg index DefSubReg, reading from another source with class SrcRC and
@@ -639,7 +646,8 @@ class TargetRegisterInfo : public MCRegisterInfo {
   virtual bool shouldRewriteCopySrc(const TargetRegisterClass *DefRC,
                                     unsigned DefSubReg,
                                     const TargetRegisterClass *SrcRC,
-                                    unsigned SrcSubReg) const;
+                                    unsigned SrcSubReg,
+                                    const MachineRegisterInfo &MRI) const;
 
   /// Returns the largest legal sub-class of RC that
   /// supports the sub-register index Idx.
@@ -769,10 +777,11 @@ class TargetRegisterInfo : public MCRegisterInfo {
   /// corresponding argument register class.
   ///
   /// The function returns NULL if no register class can be found.
-  const TargetRegisterClass*
+  const TargetRegisterClass *
   getCommonSuperRegClass(const TargetRegisterClass *RCA, unsigned SubA,
                          const TargetRegisterClass *RCB, unsigned SubB,
-                         unsigned &PreA, unsigned &PreB) const;
+                         unsigned &PreA, unsigned &PreB,
+                         const MachineRegisterInfo &MRI) const;
 
   //===--------------------------------------------------------------------===//
   // Register Class Information
@@ -809,8 +818,8 @@ class TargetRegisterInfo : public MCRegisterInfo {
   /// Find the largest common subclass of A and B.
   /// Return NULL if there is no common subclass.
   const TargetRegisterClass *
-  getCommonSubClass(const TargetRegisterClass *A,
-                    const TargetRegisterClass *B) const;
+  getCommonSubClass(const TargetRegisterClass *A, const TargetRegisterClass *B,
+                    const MachineRegisterInfo &MRI) const;
 
   /// Returns a TargetRegisterClass used for pointer values.
   /// If a target supports multiple different pointer register classes,
diff --git a/llvm/include/llvm/MC/MCRegisterInfo.h b/llvm/include/llvm/MC/MCRegisterInfo.h
index fb4d11ec1d4d16..1a2ff58c31f5a6 100644
--- a/llvm/include/llvm/MC/MCRegisterInfo.h
+++ b/llvm/include/llvm/MC/MCRegisterInfo.h
@@ -47,6 +47,7 @@ class MCRegisterClass {
   const int8_t CopyCost;
   const bool Allocatable;
   const bool BaseClass;
+  const bool Synthetic;
 
   /// getID() - Return the register class ID number.
   ///
@@ -101,6 +102,9 @@ class MCRegisterClass {
 
   /// Return true if this register class has a defined BaseClassOrder.
   bool isBaseClass() const { return BaseClass; }
+  /// isSynthetic - Return true if this is a synthetic class. This field helps
+  /// targets to dynamically enable the regclass during codegen.
+  bool isSynthetic() const { return Synthetic; }
 };
 
 /// MCRegisterDesc - This record contains information about a particular
diff --git a/llvm/include/llvm/Target/Target.td b/llvm/include/llvm/Target/Target.td
index cb1c0ed2513d45..0cf9159030bc52 100644
--- a/llvm/include/llvm/Target/Target.td
+++ b/llvm/include/llvm/Target/Target.td
@@ -338,6 +338,12 @@ class RegisterClass<string namespace, list<ValueType> regTypes, int alignment,
   // Target-specific flags. This becomes the TSFlags field in TargetRegisterClass.
   bits<8> TSFlags = 0;
 
+  // If set to true, the register class won't take part in various regclass queries
+  // by default. This allows targets to dynamically enable classes for a period
+  // during codegen so that they can be turned allocatable, copyable, spillable,
+  // and/or make them available for various regclass queries.
+  bit Synthetic = false;
+
   // If set then consider this register class to be the base class for registers in
   // its MemberList.  The base class for registers present in multiple base register
   // classes will be resolved in the order defined by this value, with lower values
diff --git a/llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp b/llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp
index ed6ce6bc73d38c..93860aae8acfdf 100644
--- a/llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp
+++ b/llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp
@@ -604,7 +604,7 @@ bool AggressiveAntiDepBreaker::FindSuitableFreeRegisters(
   // check every use of the register and find the largest register class
   // that can be used in all of them.
   const TargetRegisterClass *SuperRC =
-    TRI->getMinimalPhysRegClass(SuperReg, MVT::Other);
+      TRI->getMinimalPhysRegClass(SuperReg, MRI, MVT::Other);
 
   ArrayRef<MCPhysReg> Order = RegClassInfo.getOrder(SuperRC);
   if (Order.empty()) {
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp b/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp
index 14f2a363f9be61..7b403fee7ed120 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp
@@ -813,7 +813,7 @@ void DwarfCompileUnit::applyConcreteDbgVariableAttributes(
   auto AddEntry = [&](const DbgValueLocEntry &Entry,
                       DIExpressionCursor &Cursor) {
     if (Entry.isLocation()) {
-      if (!DwarfExpr.addMachineRegExpression(TRI, Cursor,
+      if (!DwarfExpr.addMachineRegExpression(TRI, Asm->MF->getRegInfo(), Cursor,
                                              Entry.getLoc().getReg()))
         return false;
     } else if (Entry.isInt()) {
@@ -910,7 +910,8 @@ void DwarfCompileUnit::applyConcreteDbgVariableAttributes(const Loc::MMI &MMI,
       addOpAddress(*Loc, FrameSymbol);
     else
       DwarfExpr.addMachineRegExpression(
-          *Asm->MF->getSubtarget().getRegisterInfo(), Cursor, FrameReg);
+          *Asm->MF->getSubtarget().getRegisterInfo(), Asm->MF->getRegInfo(),
+          Cursor, FrameReg);
     DwarfExpr.addExpression(std::move(Cursor));
   }
   if (Asm->TM.getTargetTriple().isNVPTX() && DD->tuneForGDB()) {
@@ -940,7 +941,8 @@ void DwarfCompileUnit::applyConcreteDbgVariableAttributes(
     DIExpressionCursor Cursor(Expr.getElements());
     DwarfExpr.beginEntryValueExpression(Cursor);
     DwarfExpr.addMachineRegExpression(
-        *Asm->MF->getSubtarget().getRegisterInfo(), Cursor, Register);
+        *Asm->MF->getSubtarget().getRegisterInfo(), Asm->MF->getRegInfo(),
+        Cursor, Register);
     DwarfExpr.addExpression(std::move(Cursor));
   }
   addBlock(VariableDie, dwarf::DW_AT_location, DwarfExpr.finalize());
@@ -1557,7 +1559,8 @@ void DwarfCompileUnit::addAddress(DIE &Die, dwarf::Attribute Attribute,
 
   DIExpressionCursor Cursor({});
   const TargetRegisterInfo &TRI = *Asm->MF->getSubtarget().getRegisterInfo();
-  if (!DwarfExpr.addMachineRegExpression(TRI, Cursor, Location.getReg()))
+  if (!DwarfExpr.addMachineRegExpression(TRI, Asm->MF->getRegInfo(), Cursor,
+                                         Location.getReg()))
     return;
   DwarfExpr.addExpression(std::move(Cursor));
 
@@ -1587,7 +1590,8 @@ void DwarfCompileUnit::addComplexAddress(const DIExpression *DIExpr, DIE &Die,
     DwarfExpr.beginEntryValueExpression(Cursor);
 
   const TargetRegisterInfo &TRI = *Asm->MF->getSubtarget().getRegisterInfo();
-  if (!DwarfExpr.addMachineRegExpression(TRI, Cursor, Location.getReg()))
+  if (!DwarfExpr.addMachineRegExpression(TRI, Asm->MF->getRegInfo(), Cursor,
+                                         Location.getReg()))
     return;
   DwarfExpr.addExpression(std::move(Cursor));
 
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp b/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
index 6b5ad62e083e3b..0b372d7b0e1bf1 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
@@ -2652,7 +2652,8 @@ void DwarfDebug::emitDebugLocValue(const AsmPrinter &AP, const DIBasicType *BT,
     DwarfExpr.beginEntryValueExpression(ExprCursor);
 
     const TargetRegisterInfo &TRI = *AP.MF->getSubtarget().getRegisterInfo();
-    if (!DwarfExpr.addMachineRegExpression(TRI, ExprCursor, Location.getReg()))
+    if (!DwarfExpr.addMachineRegExpression(TRI, AP.MF->getRegInfo(), ExprCursor,
+                                           Location.getReg()))
       return;
     return DwarfExpr.addExpression(std::move(ExprCursor));
   }
@@ -2673,7 +2674,8 @@ void DwarfDebug::emitDebugLocValue(const AsmPrinter &AP, const DIBasicType *BT,
         DwarfExpr.setMemoryLocationKind();
 
       const TargetRegisterInfo &TRI = *AP.MF->getSubtarget().getRegisterInfo();
-      if (!DwarfExpr.addMachineRegExpression(TRI, Cursor, Location.getReg()))
+      if (!DwarfExpr.addMachineRegExpression(TRI, AP.MF->getRegInfo(), Cursor,
+                                             Location.getReg()))
         return false;
     } else if (Entry.isTargetIndexLocation()) {
       TargetIndexLocation Loc = Entry.getTargetIndexLocation();
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp b/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp
index a74d43897d45b3..6d4794d27effbd 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp
@@ -97,6 +97,7 @@ void DwarfExpression::addAnd(unsigned Mask) {
 }
 
 bool DwarfExpression::addMachineReg(const TargetRegisterInfo &TRI,
+                                    const MachineRegisterInfo &MRI,
                                     llvm::Register MachineReg,
                                     unsigned MaxSize) {
   if (!MachineReg.isPhysical()) {
@@ -134,7 +135,7 @@ bool DwarfExpression::addMachineReg(const TargetRegisterInfo &TRI,
   // For example, Q0 on ARM is a composition of D0+D1.
   unsigned CurPos = 0;
   // The size of the register in bits.
-  const TargetRegisterClass *RC = TRI.getMinimalPhysRegClass(MachineReg);
+  const TargetRegisterClass *RC = TRI.getMinimalPhysRegClass(MachineReg, MRI);
   unsigned RegSize = TRI.getRegSizeInBits(*RC);
   // Keep track of the bits in the register we already emitted, so we
   // can avoid emitting redundant aliasing subregs. Because this is
@@ -248,11 +249,13 @@ void DwarfExpression::addConstantFP(const APFloat &APF, const AsmPrinter &AP) {
 }
 
 bool DwarfExpression::addMachineRegExpression(const TargetRegisterInfo &TRI,
+                                              const MachineRegisterInfo &MRI,
                                               DIExpressionCursor &ExprCursor,
                                               llvm::Register MachineReg,
                                               unsigned FragmentOffsetInBits) {
   auto Fragment = ExprCursor.getFragmentInfo();
-  if (!addMachineReg(TRI, MachineReg, Fragment ? Fragment->SizeInBits : ~1U)) {
+  if (!addMachineReg(TRI, MRI, MachineReg,
+                     Fragment ? Fragment->SizeInBits : ~1U)) {
     LocationKind = Unknown;
     return false;
   }
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h b/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h
index 667a9efc6f6c04..70ab73b5996b48 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h
@@ -245,7 +245,8 @@ class DwarfExpression {
   /// multiple subregisters that alias the register.
   ///
   /// \return false if no DWARF register exists for MachineReg.
-  bool addMachineReg(const TargetRegisterInfo &TRI, llvm::Register MachineReg,
+  bool addMachineReg(const TargetRegisterInfo &TRI,
+                     const MachineRegisterInfo &MRI, llvm::Register MachineReg,
                      unsign...
[truncated]

@llvmbot
Copy link
Collaborator

llvmbot commented Mar 20, 2024

@llvm/pr-subscribers-backend-amdgpu

Author: Christudasan Devadasan (cdevadas)

Changes

This new class will have the same set of registers as VGPR_32 class
but their actual allocatable set can vary. This synthetic class won't
be enabled until regalloc pipeline so that they won't be incorrectly
used for cases where actual VGPR_32 class machine operands are expected.
This class will be enabled during SGPR spill lowering to create virtual registers
for the operands of whole-wave mode instructions that handle the SGPR
spill to VGPRs lanes and will eventually get allocated during vgpr-regalloc
pipeline. After regalloc, this special class will be disabled.


Patch is 624.13 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/86008.diff

134 Files Affected:

  • (modified) llvm/include/llvm/CodeGen/LiveStacks.h (+1)
  • (modified) llvm/include/llvm/CodeGen/MachineInstr.h (+5-3)
  • (modified) llvm/include/llvm/CodeGen/MachineRegisterInfo.h (+34)
  • (modified) llvm/include/llvm/CodeGen/RegisterBankInfo.h (+2-1)
  • (modified) llvm/include/llvm/CodeGen/TargetLowering.h (+1)
  • (modified) llvm/include/llvm/CodeGen/TargetRegisterInfo.h (+20-11)
  • (modified) llvm/include/llvm/MC/MCRegisterInfo.h (+4)
  • (modified) llvm/include/llvm/Target/Target.td (+6)
  • (modified) llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp (+9-5)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp (+4-2)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp (+5-2)
  • (modified) llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h (+3-1)
  • (modified) llvm/lib/CodeGen/DetectDeadLanes.cpp (+4-4)
  • (modified) llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp (+22-14)
  • (modified) llvm/lib/CodeGen/GlobalISel/Utils.cpp (+2-2)
  • (modified) llvm/lib/CodeGen/LiveStacks.cpp (+2-1)
  • (modified) llvm/lib/CodeGen/MIRParser/MIRParser.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/MachineCSE.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/MachineCombiner.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/MachineInstr.cpp (+12-8)
  • (modified) llvm/lib/CodeGen/MachineRegisterInfo.cpp (+45-3)
  • (modified) llvm/lib/CodeGen/MachineVerifier.cpp (+3-3)
  • (modified) llvm/lib/CodeGen/PeepholeOptimizer.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/PrologEpilogInserter.cpp (+6-3)
  • (modified) llvm/lib/CodeGen/RegAllocGreedy.cpp (+3-2)
  • (modified) llvm/lib/CodeGen/RegisterBank.cpp (+2-2)
  • (modified) llvm/lib/CodeGen/RegisterBankInfo.cpp (+6-4)
  • (modified) llvm/lib/CodeGen/RegisterClassInfo.cpp (+10-1)
  • (modified) llvm/lib/CodeGen/RegisterCoalescer.cpp (+8-8)
  • (modified) llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp (+2-1)
  • (modified) llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp (+12-10)
  • (modified) llvm/lib/CodeGen/SelectionDAG/ScheduleDAGFast.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/SelectionDAG/ScheduleDAGRRList.cpp (+1-2)
  • (modified) llvm/lib/CodeGen/SelectionDAG/ScheduleDAGSDNodes.cpp (+5-3)
  • (modified) llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp (+5-3)
  • (modified) llvm/lib/CodeGen/StackMaps.cpp (+4-2)
  • (modified) llvm/lib/CodeGen/TailDuplicator.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/TargetLoweringBase.cpp (+2-1)
  • (modified) llvm/lib/CodeGen/TargetRegisterInfo.cpp (+52-41)
  • (modified) llvm/lib/CodeGen/TwoAddressInstructionPass.cpp (+3-4)
  • (modified) llvm/lib/Target/AArch64/AArch64FrameLowering.cpp (+2-1)
  • (modified) llvm/lib/Target/AArch64/AArch64InstrInfo.cpp (+6-5)
  • (modified) llvm/lib/Target/AArch64/AArch64LoadStoreOptimizer.cpp (+16-9)
  • (modified) llvm/lib/Target/AMDGPU/GCNRewritePartialRegUses.cpp (+3-2)
  • (modified) llvm/lib/Target/AMDGPU/SIISelLowering.cpp (+4-3)
  • (modified) llvm/lib/Target/AMDGPU/SIISelLowering.h (+1)
  • (modified) llvm/lib/Target/AMDGPU/SIInstrInfo.cpp (+6-7)
  • (modified) llvm/lib/Target/AMDGPU/SIInstrInfo.h (+1-1)
  • (modified) llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp (+74-49)
  • (modified) llvm/lib/Target/AMDGPU/SILowerWWMCopies.cpp (+4-4)
  • (modified) llvm/lib/Target/AMDGPU/SIMachineFunctionInfo.cpp (+1-1)
  • (modified) llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp (+9-11)
  • (modified) llvm/lib/Target/AMDGPU/SIRegisterInfo.h (+4-3)
  • (modified) llvm/lib/Target/AMDGPU/SIRegisterInfo.td (+10)
  • (modified) llvm/lib/Target/ARM/ARMBaseRegisterInfo.cpp (+6-6)
  • (modified) llvm/lib/Target/ARM/ARMBaseRegisterInfo.h (+2-1)
  • (modified) llvm/lib/Target/AVR/AVRAsmPrinter.cpp (+2-1)
  • (modified) llvm/lib/Target/AVR/AVRFrameLowering.cpp (+4-2)
  • (modified) llvm/lib/Target/CSKY/CSKYFrameLowering.cpp (+4-2)
  • (modified) llvm/lib/Target/Hexagon/BitTracker.cpp (+1-1)
  • (modified) llvm/lib/Target/Hexagon/HexagonBitTracker.cpp (+2-1)
  • (modified) llvm/lib/Target/Hexagon/HexagonExpandCondsets.cpp (+1-1)
  • (modified) llvm/lib/Target/Hexagon/HexagonFrameLowering.cpp (+7-4)
  • (modified) llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp (+3-1)
  • (modified) llvm/lib/Target/Hexagon/HexagonVLIWPacketizer.cpp (+6-3)
  • (modified) llvm/lib/Target/Hexagon/RDFCopy.cpp (+5-3)
  • (modified) llvm/lib/Target/LoongArch/LoongArchFrameLowering.cpp (+2-1)
  • (modified) llvm/lib/Target/Mips/MipsFrameLowering.cpp (+2-1)
  • (modified) llvm/lib/Target/Mips/MipsSEFrameLowering.cpp (+4-2)
  • (modified) llvm/lib/Target/PowerPC/PPCFrameLowering.cpp (+3-2)
  • (modified) llvm/lib/Target/PowerPC/PPCInstrInfo.cpp (+4-4)
  • (modified) llvm/lib/Target/PowerPC/PPCRegisterInfo.cpp (+2-1)
  • (modified) llvm/lib/Target/RISCV/RISCVFrameLowering.cpp (+6-3)
  • (modified) llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp (+1-1)
  • (modified) llvm/lib/Target/SystemZ/SystemZFrameLowering.cpp (+4-2)
  • (modified) llvm/lib/Target/SystemZ/SystemZInstrInfo.cpp (+2-2)
  • (modified) llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp (+4-5)
  • (modified) llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.cpp (+1-1)
  • (modified) llvm/lib/Target/WebAssembly/WebAssemblyReplacePhysRegs.cpp (+1-1)
  • (modified) llvm/lib/Target/X86/X86FrameLowering.cpp (+6-3)
  • (modified) llvm/lib/Target/X86/X86InstrInfo.cpp (+2-2)
  • (modified) llvm/lib/Target/X86/X86RegisterInfo.cpp (+10-11)
  • (modified) llvm/lib/Target/X86/X86RegisterInfo.h (+4-3)
  • (modified) llvm/lib/Target/XCore/XCoreFrameLowering.cpp (+4-2)
  • (modified) llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll (+32-42)
  • (modified) llvm/test/CodeGen/AMDGPU/GlobalISel/irtranslator-inline-asm.ll (+7-7)
  • (modified) llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll (+5-8)
  • (modified) llvm/test/CodeGen/AMDGPU/cf-loop-on-constant.ll (+5-35)
  • (modified) llvm/test/CodeGen/AMDGPU/collapse-endcf.ll (+59-92)
  • (modified) llvm/test/CodeGen/AMDGPU/div_i128.ll (+245-255)
  • (modified) llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir (+33-38)
  • (modified) llvm/test/CodeGen/AMDGPU/flat-scratch-init.ll (+4-10)
  • (modified) llvm/test/CodeGen/AMDGPU/fold-reload-into-exec.mir (+12-12)
  • (modified) llvm/test/CodeGen/AMDGPU/fold-reload-into-m0.mir (+2-2)
  • (modified) llvm/test/CodeGen/AMDGPU/identical-subrange-spill-infloop.ll (+263-266)
  • (modified) llvm/test/CodeGen/AMDGPU/inline-asm.i128.ll (+12-12)
  • (modified) llvm/test/CodeGen/AMDGPU/kernel-vgpr-spill-mubuf-with-voffset.ll (+5-19)
  • (modified) llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll (+102-108)
  • (modified) llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll (+131-142)
  • (modified) llvm/test/CodeGen/AMDGPU/partial-regcopy-and-spill-missed-at-regalloc.ll (+8-8)
  • (modified) llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll (+334-386)
  • (modified) llvm/test/CodeGen/AMDGPU/preserve-wwm-copy-dst-reg.ll (+2-8)
  • (modified) llvm/test/CodeGen/AMDGPU/rem_i128.ll (+640-648)
  • (modified) llvm/test/CodeGen/AMDGPU/scc-clobbered-sgpr-to-vmem-spill.ll (+226-230)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spill-dead-frame-in-dbg-value.mir (+7-7)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll (+67-83)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir (+2-3)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spill-partially-undef.mir (+6-6)
  • (modified) llvm/test/CodeGen/AMDGPU/sgpr-spills-split-regalloc.ll (+1-9)
  • (modified) llvm/test/CodeGen/AMDGPU/si-spill-sgpr-stack.ll (+2-3)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-csr-frame-ptr-reg-copy.ll (+4-4)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-reg-tuple-super-reg-use.mir (+4-6)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-sgpr-to-virtual-vgpr.mir (+43-43)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-vector-superclass.ll (+1-1)
  • (modified) llvm/test/CodeGen/AMDGPU/spill-vgpr-to-agpr-update-regscavenger.ll (+3-10)
  • (modified) llvm/test/CodeGen/AMDGPU/spill192.mir (+7-7)
  • (modified) llvm/test/CodeGen/AMDGPU/spill224.mir (+8-8)
  • (modified) llvm/test/CodeGen/AMDGPU/spill288.mir (+10-10)
  • (modified) llvm/test/CodeGen/AMDGPU/spill320.mir (+11-11)
  • (modified) llvm/test/CodeGen/AMDGPU/spill352.mir (+12-12)
  • (modified) llvm/test/CodeGen/AMDGPU/spill384.mir (+13-13)
  • (modified) llvm/test/CodeGen/AMDGPU/stacksave_stackrestore.ll (+13-32)
  • (modified) llvm/test/CodeGen/AMDGPU/vgpr-spill-placement-issue61083.ll (+8-15)
  • (modified) llvm/test/CodeGen/AMDGPU/whole-wave-register-copy.ll (+4-6)
  • (modified) llvm/test/CodeGen/AMDGPU/whole-wave-register-spill.ll (+10-14)
  • (modified) llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll (+25-32)
  • (modified) llvm/test/CodeGen/AMDGPU/wwm-reserved.ll (+174-254)
  • (modified) llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/amdgpu_isel.ll.expected (+2-2)
  • (modified) llvm/unittests/CodeGen/MFCommon.inc (+40-2)
  • (modified) llvm/unittests/CodeGen/MachineInstrTest.cpp (+11-2)
  • (modified) llvm/utils/TableGen/CodeGenRegisters.cpp (+5-2)
  • (modified) llvm/utils/TableGen/CodeGenRegisters.h (+1)
  • (modified) llvm/utils/TableGen/RegisterInfoEmitter.cpp (+2-1)
diff --git a/llvm/include/llvm/CodeGen/LiveStacks.h b/llvm/include/llvm/CodeGen/LiveStacks.h
index 2edc2985f0ee66..b9e55987383901 100644
--- a/llvm/include/llvm/CodeGen/LiveStacks.h
+++ b/llvm/include/llvm/CodeGen/LiveStacks.h
@@ -34,6 +34,7 @@ class TargetRegisterInfo;
 
 class LiveStacks : public MachineFunctionPass {
   const TargetRegisterInfo *TRI = nullptr;
+  const MachineRegisterInfo *MRI = nullptr;
 
   /// Special pool allocator for VNInfo's (LiveInterval val#).
   ///
diff --git a/llvm/include/llvm/CodeGen/MachineInstr.h b/llvm/include/llvm/CodeGen/MachineInstr.h
index fcdd73d8b65fdd..c7067d90a3ce06 100644
--- a/llvm/include/llvm/CodeGen/MachineInstr.h
+++ b/llvm/include/llvm/CodeGen/MachineInstr.h
@@ -1586,7 +1586,7 @@ class MachineInstr
   const TargetRegisterClass *getRegClassConstraintEffectForVReg(
       Register Reg, const TargetRegisterClass *CurRC,
       const TargetInstrInfo *TII, const TargetRegisterInfo *TRI,
-      bool ExploreBundle = false) const;
+      const MachineRegisterInfo &MRI, bool ExploreBundle = false) const;
 
   /// Applies the constraints (def/use) implied by the \p OpIdx operand
   /// to the given \p CurRC.
@@ -1600,7 +1600,8 @@ class MachineInstr
   const TargetRegisterClass *
   getRegClassConstraintEffect(unsigned OpIdx, const TargetRegisterClass *CurRC,
                               const TargetInstrInfo *TII,
-                              const TargetRegisterInfo *TRI) const;
+                              const TargetRegisterInfo *TRI,
+                              const MachineRegisterInfo &MRI) const;
 
   /// Add a tie between the register operands at DefIdx and UseIdx.
   /// The tie will cause the register allocator to ensure that the two
@@ -2005,7 +2006,8 @@ class MachineInstr
   /// If the related operand does not constrained Reg, this returns CurRC.
   const TargetRegisterClass *getRegClassConstraintEffectForVRegImpl(
       unsigned OpIdx, Register Reg, const TargetRegisterClass *CurRC,
-      const TargetInstrInfo *TII, const TargetRegisterInfo *TRI) const;
+      const TargetInstrInfo *TII, const TargetRegisterInfo *TRI,
+      const MachineRegisterInfo &MRI) const;
 
   /// Stores extra instruction information inline or allocates as ExtraInfo
   /// based on the number of pointers.
diff --git a/llvm/include/llvm/CodeGen/MachineRegisterInfo.h b/llvm/include/llvm/CodeGen/MachineRegisterInfo.h
index 09d9a0b4ec402f..78414f31d1bf82 100644
--- a/llvm/include/llvm/CodeGen/MachineRegisterInfo.h
+++ b/llvm/include/llvm/CodeGen/MachineRegisterInfo.h
@@ -94,6 +94,16 @@ class MachineRegisterInfo {
   /// all registers that were disabled are removed from the list.
   SmallVector<MCPhysReg, 16> UpdatedCSRs;
 
+  /// The Synthetic field of regclasses. Targets can alter this vector
+  /// to enable classes dynamically during codegen.
+  BitVector RegClassSyntheticInfo;
+
+  /// To have more restrictions on the allocation order for each register class.
+  /// For some targets, two distinct register classes can have the same set of
+  /// registers, but their allocatable set can vary. This vector helps targets
+  /// to dynamically determine the actual allocatable set for RCs.
+  std::vector<BitVector> RegClassAllocationMasks;
+
   /// RegAllocHints - This vector records register allocation hints for
   /// virtual registers. For each virtual register, it keeps a pair of hint
   /// type and hints vector making up the allocation hints. Only the first
@@ -257,6 +267,30 @@ class MachineRegisterInfo {
   /// Notice that it will override ant previously disabled/saved CSRs.
   void setCalleeSavedRegs(ArrayRef<MCPhysReg> CSRs);
 
+  /// Initialize the RegClassSyntheticInfo. It sets the bit position as
+  /// exactly as the tablegened Synthetic field. Targets can later flip this
+  /// field to enable/disable the regclass whenever required.
+  void initializeRegClassSyntheticInfo();
+
+  /// Change the synthetic info for the regclass \p RC from \p Value.
+  void changeSyntheticInfoForRC(const TargetRegisterClass *RC, bool Value);
+
+  /// This function checks if \p RC is enabled or not so that it can be included
+  /// in various regclass related queries.
+  bool isEnabled(const TargetRegisterClass *RC) const;
+
+  /// Update dynamically determined allocation mask for register classes.
+  void updateAllocationMaskForRCs(std::vector<BitVector> &&Mask);
+
+  // Return the allocation mask for regclass \p RC.
+  const BitVector &getAllocationMaskForRC(const TargetRegisterClass &RC) const;
+
+  /// True when the target has dynamically determined allocation mask for its
+  /// register classes.
+  bool hasAllocationMaskForRCs() const {
+    return RegClassAllocationMasks.size();
+  }
+
   // Strictly for use by MachineInstr.cpp.
   void addRegOperandToUseList(MachineOperand *MO);
 
diff --git a/llvm/include/llvm/CodeGen/RegisterBankInfo.h b/llvm/include/llvm/CodeGen/RegisterBankInfo.h
index 62c4a57a605d68..117f381512a1c6 100644
--- a/llvm/include/llvm/CodeGen/RegisterBankInfo.h
+++ b/llvm/include/llvm/CodeGen/RegisterBankInfo.h
@@ -445,7 +445,8 @@ class RegisterBankInfo {
   /// Get the MinimalPhysRegClass for Reg.
   /// \pre Reg is a physical register.
   const TargetRegisterClass *
-  getMinimalPhysRegClass(Register Reg, const TargetRegisterInfo &TRI) const;
+  getMinimalPhysRegClass(Register Reg, const TargetRegisterInfo &TRI,
+                         const MachineRegisterInfo &MRI) const;
 
   /// Try to get the mapping of \p MI.
   /// See getInstrMapping for more details on what a mapping represents.
diff --git a/llvm/include/llvm/CodeGen/TargetLowering.h b/llvm/include/llvm/CodeGen/TargetLowering.h
index 59fad88f91b1d1..5ef963cae55286 100644
--- a/llvm/include/llvm/CodeGen/TargetLowering.h
+++ b/llvm/include/llvm/CodeGen/TargetLowering.h
@@ -4384,6 +4384,7 @@ class TargetLowering : public TargetLoweringBase {
   /// Returns 'true' is the edge is necessary, 'false' otherwise
   virtual bool checkForPhysRegDependency(SDNode *Def, SDNode *User, unsigned Op,
                                          const TargetRegisterInfo *TRI,
+                                         const MachineRegisterInfo &MRI,
                                          const TargetInstrInfo *TII,
                                          unsigned &PhysReg, int &Cost) const {
     return false;
diff --git a/llvm/include/llvm/CodeGen/TargetRegisterInfo.h b/llvm/include/llvm/CodeGen/TargetRegisterInfo.h
index 117d3f71829747..a3d68d6743ede5 100644
--- a/llvm/include/llvm/CodeGen/TargetRegisterInfo.h
+++ b/llvm/include/llvm/CodeGen/TargetRegisterInfo.h
@@ -121,6 +121,9 @@ class TargetRegisterClass {
   /// Return true if this register class has a defined BaseClassOrder.
   bool isBaseClass() const { return MC->isBaseClass(); }
 
+  /// Return true if this register class is marked synthetic.
+  bool isSynthetic() const { return MC->isSynthetic(); }
+
   /// Return true if the specified TargetRegisterClass
   /// is a proper sub-class of this TargetRegisterClass.
   bool hasSubClass(const TargetRegisterClass *RC) const {
@@ -338,20 +341,23 @@ class TargetRegisterInfo : public MCRegisterInfo {
   /// Returns the Register Class of a physical register of the given type,
   /// picking the most sub register class of the right type that contains this
   /// physreg.
-  const TargetRegisterClass *getMinimalPhysRegClass(MCRegister Reg,
-                                                    MVT VT = MVT::Other) const;
+  const TargetRegisterClass *
+  getMinimalPhysRegClass(MCRegister Reg, const MachineRegisterInfo &MRI,
+                         MVT VT = MVT::Other) const;
 
   /// Returns the Register Class of a physical register of the given type,
   /// picking the most sub register class of the right type that contains this
   /// physreg. If there is no register class compatible with the given type,
   /// returns nullptr.
-  const TargetRegisterClass *getMinimalPhysRegClassLLT(MCRegister Reg,
-                                                       LLT Ty = LLT()) const;
+  const TargetRegisterClass *
+  getMinimalPhysRegClassLLT(MCRegister Reg, const MachineRegisterInfo &MRI,
+                            LLT Ty = LLT()) const;
 
   /// Return the maximal subclass of the given register class that is
   /// allocatable or NULL.
   const TargetRegisterClass *
-    getAllocatableClass(const TargetRegisterClass *RC) const;
+  getAllocatableClass(const TargetRegisterClass *RC,
+                      const MachineRegisterInfo &MRI) const;
 
   /// Returns a bitset indexed by register number indicating if a register is
   /// allocatable or not. If a register class is specified, returns the subset
@@ -630,7 +636,8 @@ class TargetRegisterInfo : public MCRegisterInfo {
   /// TableGen will synthesize missing A sub-classes.
   virtual const TargetRegisterClass *
   getMatchingSuperRegClass(const TargetRegisterClass *A,
-                           const TargetRegisterClass *B, unsigned Idx) const;
+                           const TargetRegisterClass *B, unsigned Idx,
+                           const MachineRegisterInfo &MRI) const;
 
   // For a copy-like instruction that defines a register of class DefRC with
   // subreg index DefSubReg, reading from another source with class SrcRC and
@@ -639,7 +646,8 @@ class TargetRegisterInfo : public MCRegisterInfo {
   virtual bool shouldRewriteCopySrc(const TargetRegisterClass *DefRC,
                                     unsigned DefSubReg,
                                     const TargetRegisterClass *SrcRC,
-                                    unsigned SrcSubReg) const;
+                                    unsigned SrcSubReg,
+                                    const MachineRegisterInfo &MRI) const;
 
   /// Returns the largest legal sub-class of RC that
   /// supports the sub-register index Idx.
@@ -769,10 +777,11 @@ class TargetRegisterInfo : public MCRegisterInfo {
   /// corresponding argument register class.
   ///
   /// The function returns NULL if no register class can be found.
-  const TargetRegisterClass*
+  const TargetRegisterClass *
   getCommonSuperRegClass(const TargetRegisterClass *RCA, unsigned SubA,
                          const TargetRegisterClass *RCB, unsigned SubB,
-                         unsigned &PreA, unsigned &PreB) const;
+                         unsigned &PreA, unsigned &PreB,
+                         const MachineRegisterInfo &MRI) const;
 
   //===--------------------------------------------------------------------===//
   // Register Class Information
@@ -809,8 +818,8 @@ class TargetRegisterInfo : public MCRegisterInfo {
   /// Find the largest common subclass of A and B.
   /// Return NULL if there is no common subclass.
   const TargetRegisterClass *
-  getCommonSubClass(const TargetRegisterClass *A,
-                    const TargetRegisterClass *B) const;
+  getCommonSubClass(const TargetRegisterClass *A, const TargetRegisterClass *B,
+                    const MachineRegisterInfo &MRI) const;
 
   /// Returns a TargetRegisterClass used for pointer values.
   /// If a target supports multiple different pointer register classes,
diff --git a/llvm/include/llvm/MC/MCRegisterInfo.h b/llvm/include/llvm/MC/MCRegisterInfo.h
index fb4d11ec1d4d16..1a2ff58c31f5a6 100644
--- a/llvm/include/llvm/MC/MCRegisterInfo.h
+++ b/llvm/include/llvm/MC/MCRegisterInfo.h
@@ -47,6 +47,7 @@ class MCRegisterClass {
   const int8_t CopyCost;
   const bool Allocatable;
   const bool BaseClass;
+  const bool Synthetic;
 
   /// getID() - Return the register class ID number.
   ///
@@ -101,6 +102,9 @@ class MCRegisterClass {
 
   /// Return true if this register class has a defined BaseClassOrder.
   bool isBaseClass() const { return BaseClass; }
+  /// isSynthetic - Return true if this is a synthetic class. This field helps
+  /// targets to dynamically enable the regclass during codegen.
+  bool isSynthetic() const { return Synthetic; }
 };
 
 /// MCRegisterDesc - This record contains information about a particular
diff --git a/llvm/include/llvm/Target/Target.td b/llvm/include/llvm/Target/Target.td
index cb1c0ed2513d45..0cf9159030bc52 100644
--- a/llvm/include/llvm/Target/Target.td
+++ b/llvm/include/llvm/Target/Target.td
@@ -338,6 +338,12 @@ class RegisterClass<string namespace, list<ValueType> regTypes, int alignment,
   // Target-specific flags. This becomes the TSFlags field in TargetRegisterClass.
   bits<8> TSFlags = 0;
 
+  // If set to true, the register class won't take part in various regclass queries
+  // by default. This allows targets to dynamically enable classes for a period
+  // during codegen so that they can be turned allocatable, copyable, spillable,
+  // and/or make them available for various regclass queries.
+  bit Synthetic = false;
+
   // If set then consider this register class to be the base class for registers in
   // its MemberList.  The base class for registers present in multiple base register
   // classes will be resolved in the order defined by this value, with lower values
diff --git a/llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp b/llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp
index ed6ce6bc73d38c..93860aae8acfdf 100644
--- a/llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp
+++ b/llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp
@@ -604,7 +604,7 @@ bool AggressiveAntiDepBreaker::FindSuitableFreeRegisters(
   // check every use of the register and find the largest register class
   // that can be used in all of them.
   const TargetRegisterClass *SuperRC =
-    TRI->getMinimalPhysRegClass(SuperReg, MVT::Other);
+      TRI->getMinimalPhysRegClass(SuperReg, MRI, MVT::Other);
 
   ArrayRef<MCPhysReg> Order = RegClassInfo.getOrder(SuperRC);
   if (Order.empty()) {
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp b/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp
index 14f2a363f9be61..7b403fee7ed120 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp
@@ -813,7 +813,7 @@ void DwarfCompileUnit::applyConcreteDbgVariableAttributes(
   auto AddEntry = [&](const DbgValueLocEntry &Entry,
                       DIExpressionCursor &Cursor) {
     if (Entry.isLocation()) {
-      if (!DwarfExpr.addMachineRegExpression(TRI, Cursor,
+      if (!DwarfExpr.addMachineRegExpression(TRI, Asm->MF->getRegInfo(), Cursor,
                                              Entry.getLoc().getReg()))
         return false;
     } else if (Entry.isInt()) {
@@ -910,7 +910,8 @@ void DwarfCompileUnit::applyConcreteDbgVariableAttributes(const Loc::MMI &MMI,
       addOpAddress(*Loc, FrameSymbol);
     else
       DwarfExpr.addMachineRegExpression(
-          *Asm->MF->getSubtarget().getRegisterInfo(), Cursor, FrameReg);
+          *Asm->MF->getSubtarget().getRegisterInfo(), Asm->MF->getRegInfo(),
+          Cursor, FrameReg);
     DwarfExpr.addExpression(std::move(Cursor));
   }
   if (Asm->TM.getTargetTriple().isNVPTX() && DD->tuneForGDB()) {
@@ -940,7 +941,8 @@ void DwarfCompileUnit::applyConcreteDbgVariableAttributes(
     DIExpressionCursor Cursor(Expr.getElements());
     DwarfExpr.beginEntryValueExpression(Cursor);
     DwarfExpr.addMachineRegExpression(
-        *Asm->MF->getSubtarget().getRegisterInfo(), Cursor, Register);
+        *Asm->MF->getSubtarget().getRegisterInfo(), Asm->MF->getRegInfo(),
+        Cursor, Register);
     DwarfExpr.addExpression(std::move(Cursor));
   }
   addBlock(VariableDie, dwarf::DW_AT_location, DwarfExpr.finalize());
@@ -1557,7 +1559,8 @@ void DwarfCompileUnit::addAddress(DIE &Die, dwarf::Attribute Attribute,
 
   DIExpressionCursor Cursor({});
   const TargetRegisterInfo &TRI = *Asm->MF->getSubtarget().getRegisterInfo();
-  if (!DwarfExpr.addMachineRegExpression(TRI, Cursor, Location.getReg()))
+  if (!DwarfExpr.addMachineRegExpression(TRI, Asm->MF->getRegInfo(), Cursor,
+                                         Location.getReg()))
     return;
   DwarfExpr.addExpression(std::move(Cursor));
 
@@ -1587,7 +1590,8 @@ void DwarfCompileUnit::addComplexAddress(const DIExpression *DIExpr, DIE &Die,
     DwarfExpr.beginEntryValueExpression(Cursor);
 
   const TargetRegisterInfo &TRI = *Asm->MF->getSubtarget().getRegisterInfo();
-  if (!DwarfExpr.addMachineRegExpression(TRI, Cursor, Location.getReg()))
+  if (!DwarfExpr.addMachineRegExpression(TRI, Asm->MF->getRegInfo(), Cursor,
+                                         Location.getReg()))
     return;
   DwarfExpr.addExpression(std::move(Cursor));
 
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp b/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
index 6b5ad62e083e3b..0b372d7b0e1bf1 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp
@@ -2652,7 +2652,8 @@ void DwarfDebug::emitDebugLocValue(const AsmPrinter &AP, const DIBasicType *BT,
     DwarfExpr.beginEntryValueExpression(ExprCursor);
 
     const TargetRegisterInfo &TRI = *AP.MF->getSubtarget().getRegisterInfo();
-    if (!DwarfExpr.addMachineRegExpression(TRI, ExprCursor, Location.getReg()))
+    if (!DwarfExpr.addMachineRegExpression(TRI, AP.MF->getRegInfo(), ExprCursor,
+                                           Location.getReg()))
       return;
     return DwarfExpr.addExpression(std::move(ExprCursor));
   }
@@ -2673,7 +2674,8 @@ void DwarfDebug::emitDebugLocValue(const AsmPrinter &AP, const DIBasicType *BT,
         DwarfExpr.setMemoryLocationKind();
 
       const TargetRegisterInfo &TRI = *AP.MF->getSubtarget().getRegisterInfo();
-      if (!DwarfExpr.addMachineRegExpression(TRI, Cursor, Location.getReg()))
+      if (!DwarfExpr.addMachineRegExpression(TRI, AP.MF->getRegInfo(), Cursor,
+                                             Location.getReg()))
         return false;
     } else if (Entry.isTargetIndexLocation()) {
       TargetIndexLocation Loc = Entry.getTargetIndexLocation();
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp b/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp
index a74d43897d45b3..6d4794d27effbd 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp
@@ -97,6 +97,7 @@ void DwarfExpression::addAnd(unsigned Mask) {
 }
 
 bool DwarfExpression::addMachineReg(const TargetRegisterInfo &TRI,
+                                    const MachineRegisterInfo &MRI,
                                     llvm::Register MachineReg,
                                     unsigned MaxSize) {
   if (!MachineReg.isPhysical()) {
@@ -134,7 +135,7 @@ bool DwarfExpression::addMachineReg(const TargetRegisterInfo &TRI,
   // For example, Q0 on ARM is a composition of D0+D1.
   unsigned CurPos = 0;
   // The size of the register in bits.
-  const TargetRegisterClass *RC = TRI.getMinimalPhysRegClass(MachineReg);
+  const TargetRegisterClass *RC = TRI.getMinimalPhysRegClass(MachineReg, MRI);
   unsigned RegSize = TRI.getRegSizeInBits(*RC);
   // Keep track of the bits in the register we already emitted, so we
   // can avoid emitting redundant aliasing subregs. Because this is
@@ -248,11 +249,13 @@ void DwarfExpression::addConstantFP(const APFloat &APF, const AsmPrinter &AP) {
 }
 
 bool DwarfExpression::addMachineRegExpression(const TargetRegisterInfo &TRI,
+                                              const MachineRegisterInfo &MRI,
                                               DIExpressionCursor &ExprCursor,
                                               llvm::Register MachineReg,
                                               unsigned FragmentOffsetInBits) {
   auto Fragment = ExprCursor.getFragmentInfo();
-  if (!addMachineReg(TRI, MachineReg, Fragment ? Fragment->SizeInBits : ~1U)) {
+  if (!addMachineReg(TRI, MRI, MachineReg,
+                     Fragment ? Fragment->SizeInBits : ~1U)) {
     LocationKind = Unknown;
     return false;
   }
diff --git a/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h b/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h
index 667a9efc6f6c04..70ab73b5996b48 100644
--- a/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h
+++ b/llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h
@@ -245,7 +245,8 @@ class DwarfExpression {
   /// multiple subregisters that alias the register.
   ///
   /// \return false if no DWARF register exists for MachineReg.
-  bool addMachineReg(const TargetRegisterInfo &TRI, llvm::Register MachineReg,
+  bool addMachineReg(const TargetRegisterInfo &TRI,
+                     const MachineRegisterInfo &MRI, llvm::Register MachineReg,
                      unsign...
[truncated]

Copy link

⚠️ C/C++ code formatter, clang-format found issues in your code. ⚠️

You can test this locally with the following command:
git-clang-format --diff 1f1f569b29f42161ea978328aea60044f16eee49 745b6d377f03b343ad586f7488d9a7571f088bbd -- llvm/include/llvm/CodeGen/LiveStacks.h llvm/include/llvm/CodeGen/MachineInstr.h llvm/include/llvm/CodeGen/MachineRegisterInfo.h llvm/include/llvm/CodeGen/RegisterBankInfo.h llvm/include/llvm/CodeGen/TargetLowering.h llvm/include/llvm/CodeGen/TargetRegisterInfo.h llvm/include/llvm/MC/MCRegisterInfo.h llvm/lib/CodeGen/AggressiveAntiDepBreaker.cpp llvm/lib/CodeGen/AsmPrinter/DwarfCompileUnit.cpp llvm/lib/CodeGen/AsmPrinter/DwarfDebug.cpp llvm/lib/CodeGen/AsmPrinter/DwarfExpression.cpp llvm/lib/CodeGen/AsmPrinter/DwarfExpression.h llvm/lib/CodeGen/DetectDeadLanes.cpp llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp llvm/lib/CodeGen/GlobalISel/Utils.cpp llvm/lib/CodeGen/LiveStacks.cpp llvm/lib/CodeGen/MIRParser/MIRParser.cpp llvm/lib/CodeGen/MachineCSE.cpp llvm/lib/CodeGen/MachineCombiner.cpp llvm/lib/CodeGen/MachineInstr.cpp llvm/lib/CodeGen/MachineRegisterInfo.cpp llvm/lib/CodeGen/MachineVerifier.cpp llvm/lib/CodeGen/PeepholeOptimizer.cpp llvm/lib/CodeGen/PrologEpilogInserter.cpp llvm/lib/CodeGen/RegAllocGreedy.cpp llvm/lib/CodeGen/RegisterBank.cpp llvm/lib/CodeGen/RegisterBankInfo.cpp llvm/lib/CodeGen/RegisterClassInfo.cpp llvm/lib/CodeGen/RegisterCoalescer.cpp llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp llvm/lib/CodeGen/SelectionDAG/InstrEmitter.cpp llvm/lib/CodeGen/SelectionDAG/ScheduleDAGFast.cpp llvm/lib/CodeGen/SelectionDAG/ScheduleDAGRRList.cpp llvm/lib/CodeGen/SelectionDAG/ScheduleDAGSDNodes.cpp llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp llvm/lib/CodeGen/StackMaps.cpp llvm/lib/CodeGen/TailDuplicator.cpp llvm/lib/CodeGen/TargetLoweringBase.cpp llvm/lib/CodeGen/TargetRegisterInfo.cpp llvm/lib/CodeGen/TwoAddressInstructionPass.cpp llvm/lib/Target/AArch64/AArch64FrameLowering.cpp llvm/lib/Target/AArch64/AArch64InstrInfo.cpp llvm/lib/Target/AArch64/AArch64LoadStoreOptimizer.cpp llvm/lib/Target/AMDGPU/GCNRewritePartialRegUses.cpp llvm/lib/Target/AMDGPU/SIISelLowering.cpp llvm/lib/Target/AMDGPU/SIISelLowering.h llvm/lib/Target/AMDGPU/SIInstrInfo.cpp llvm/lib/Target/AMDGPU/SIInstrInfo.h llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp llvm/lib/Target/AMDGPU/SILowerWWMCopies.cpp llvm/lib/Target/AMDGPU/SIMachineFunctionInfo.cpp llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp llvm/lib/Target/AMDGPU/SIRegisterInfo.h llvm/lib/Target/ARM/ARMBaseRegisterInfo.cpp llvm/lib/Target/ARM/ARMBaseRegisterInfo.h llvm/lib/Target/AVR/AVRAsmPrinter.cpp llvm/lib/Target/AVR/AVRFrameLowering.cpp llvm/lib/Target/CSKY/CSKYFrameLowering.cpp llvm/lib/Target/Hexagon/BitTracker.cpp llvm/lib/Target/Hexagon/HexagonBitTracker.cpp llvm/lib/Target/Hexagon/HexagonExpandCondsets.cpp llvm/lib/Target/Hexagon/HexagonFrameLowering.cpp llvm/lib/Target/Hexagon/HexagonInstrInfo.cpp llvm/lib/Target/Hexagon/HexagonVLIWPacketizer.cpp llvm/lib/Target/Hexagon/RDFCopy.cpp llvm/lib/Target/LoongArch/LoongArchFrameLowering.cpp llvm/lib/Target/Mips/MipsFrameLowering.cpp llvm/lib/Target/Mips/MipsSEFrameLowering.cpp llvm/lib/Target/PowerPC/PPCFrameLowering.cpp llvm/lib/Target/PowerPC/PPCInstrInfo.cpp llvm/lib/Target/PowerPC/PPCRegisterInfo.cpp llvm/lib/Target/RISCV/RISCVFrameLowering.cpp llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp llvm/lib/Target/SystemZ/SystemZFrameLowering.cpp llvm/lib/Target/SystemZ/SystemZInstrInfo.cpp llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp llvm/lib/Target/WebAssembly/WebAssemblyInstrInfo.cpp llvm/lib/Target/WebAssembly/WebAssemblyReplacePhysRegs.cpp llvm/lib/Target/X86/X86FrameLowering.cpp llvm/lib/Target/X86/X86InstrInfo.cpp llvm/lib/Target/X86/X86RegisterInfo.cpp llvm/lib/Target/X86/X86RegisterInfo.h llvm/lib/Target/XCore/XCoreFrameLowering.cpp llvm/unittests/CodeGen/MFCommon.inc llvm/unittests/CodeGen/MachineInstrTest.cpp llvm/utils/TableGen/CodeGenRegisters.cpp llvm/utils/TableGen/CodeGenRegisters.h llvm/utils/TableGen/RegisterInfoEmitter.cpp
View the diff from clang-format here.
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 982d1a0c44..25b0f5688a 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -9738,11 +9738,10 @@ void SelectionDAGBuilder::visitInlineAsm(const CallBase &Call,
           Register TiedReg = R->getReg();
           MVT RegVT = R->getSimpleValueType(0);
           const TargetRegisterClass *RC =
-              TiedReg.isVirtual()
-                  ? MRI.getRegClass(TiedReg)
-                  : RegVT != MVT::Untyped
-                        ? TLI.getRegClassFor(RegVT)
-                        : TRI.getMinimalPhysRegClass(TiedReg, MRI);
+              TiedReg.isVirtual() ? MRI.getRegClass(TiedReg)
+              : RegVT != MVT::Untyped
+                  ? TLI.getRegClassFor(RegVT)
+                  : TRI.getMinimalPhysRegClass(TiedReg, MRI);
           for (unsigned i = 0, e = Flag.getNumOperandRegisters(); i != e; ++i)
             Regs.push_back(MRI.createVirtualRegister(RC));
 
diff --git a/llvm/unittests/CodeGen/MFCommon.inc b/llvm/unittests/CodeGen/MFCommon.inc
index 8db39e384d..7bf78ab41a 100644
--- a/llvm/unittests/CodeGen/MFCommon.inc
+++ b/llvm/unittests/CodeGen/MFCommon.inc
@@ -19,11 +19,7 @@ public:
 
 // Dummy regclass fields ...
 namespace {
-enum {
-  NoRegister,
-  GPR0 = 1,
-  NUM_TARGET_REGS
-};
+enum { NoRegister, GPR0 = 1, NUM_TARGET_REGS };
 
 enum {
   GPRsRegClassID = 0,
@@ -31,16 +27,17 @@ enum {
 
 const MCPhysReg GPRs[] = {GPR0};
 
-const uint8_t GPRsBits[] = { 0x00 };
+const uint8_t GPRsBits[] = {0x00};
 
 static const uint32_t GPRsSubClassMask[] = {
-  0x00000001,
+    0x00000001,
 };
 } // namespace
 
-static const TargetRegisterClass *const NullRegClasses[] = { nullptr };
-static const MCRegisterClass BogusMCRegisterClasses[] = {{ GPRs, GPRsBits, 0, 1,
-    sizeof(GPRsBits), GPRsRegClassID, 1, -1, false, true ,false }};
+static const TargetRegisterClass *const NullRegClasses[] = {nullptr};
+static const MCRegisterClass BogusMCRegisterClasses[] = {
+    {GPRs, GPRsBits, 0, 1, sizeof(GPRsBits), GPRsRegClassID, 1, -1, false, true,
+     false}};
 static const TargetRegisterClass GPRsRegClass = {
     &BogusMCRegisterClasses[GPRsRegClassID],
     GPRsSubClassMask,
@@ -52,16 +49,17 @@ static const TargetRegisterClass GPRsRegClass = {
     false,
     false,
     NullRegClasses,
-    nullptr
-};
+    nullptr};
 
- static const TargetRegisterClass *const BogusRegisterClasses[] = {&GPRsRegClass};
+static const TargetRegisterClass *const BogusRegisterClasses[] = {
+    &GPRsRegClass};
 
 class BogusRegisterInfo : public TargetRegisterInfo {
 public:
   BogusRegisterInfo()
-      : TargetRegisterInfo(nullptr, BogusRegisterClasses, BogusRegisterClasses+1,
-                           nullptr, nullptr, LaneBitmask(~0u), nullptr, nullptr) {
+      : TargetRegisterInfo(nullptr, BogusRegisterClasses,
+                           BogusRegisterClasses + 1, nullptr, nullptr,
+                           LaneBitmask(~0u), nullptr, nullptr) {
     InitMCRegisterInfo(nullptr, 0, 0, 0, nullptr, 0, nullptr, 0, nullptr,
                        nullptr, nullptr, nullptr, nullptr, 0, nullptr, nullptr);
   }

@cdevadas
Copy link
Collaborator Author

Depends on #86006 #86007 (all pre-patches already in the commits list)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants