[RISCV] Split vector FP pseudo instructions by SEW #87686
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@llvm/pr-subscribers-backend-risc-v Author: Michael Maitland (michaelmaitland) ChangesThis patch contains a series of commits to split vector FP pseudo instructions by SEW. It is NFC since the existing schedulers keep the same behavior for these instructions regardless of SEW. This is intended to be the equivalent of https://reviews.llvm.org/D153000. I will add @wangpc-pp as co-author. Merging this PR closes #87404. Scheduler changes depending on SEW can come in future patches. There are still some FP instructions that can be split, but they were not split in https://reviews.llvm.org/D153000 so I did not split them here. We can split those in a follow up patch if we'd like. Patch is 112.19 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/87686.diff 7 Files Affected:
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfo.cpp b/llvm/lib/Target/RISCV/RISCVInstrInfo.cpp
index 5582de51b17d19..92a42a3421f971 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfo.cpp
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfo.cpp
@@ -2603,31 +2603,54 @@ std::string RISCVInstrInfo::createMIROperandComment(
}
// clang-format off
-#define CASE_VFMA_OPCODE_COMMON(OP, TYPE, LMUL) \
+#define CASE_VMA_OPCODE_COMMON(OP, TYPE, LMUL) \
RISCV::PseudoV##OP##_##TYPE##_##LMUL
-#define CASE_VFMA_OPCODE_LMULS_M1(OP, TYPE) \
- CASE_VFMA_OPCODE_COMMON(OP, TYPE, M1): \
- case CASE_VFMA_OPCODE_COMMON(OP, TYPE, M2): \
- case CASE_VFMA_OPCODE_COMMON(OP, TYPE, M4): \
- case CASE_VFMA_OPCODE_COMMON(OP, TYPE, M8)
+#define CASE_VMA_OPCODE_LMULS_M1(OP, TYPE) \
+ CASE_VMA_OPCODE_COMMON(OP, TYPE, M1): \
+ case CASE_VMA_OPCODE_COMMON(OP, TYPE, M2): \
+ case CASE_VMA_OPCODE_COMMON(OP, TYPE, M4): \
+ case CASE_VMA_OPCODE_COMMON(OP, TYPE, M8)
-#define CASE_VFMA_OPCODE_LMULS_MF2(OP, TYPE) \
- CASE_VFMA_OPCODE_COMMON(OP, TYPE, MF2): \
- case CASE_VFMA_OPCODE_LMULS_M1(OP, TYPE)
+#define CASE_VMA_OPCODE_LMULS_MF2(OP, TYPE) \
+ CASE_VMA_OPCODE_COMMON(OP, TYPE, MF2): \
+ case CASE_VMA_OPCODE_LMULS_M1(OP, TYPE)
-#define CASE_VFMA_OPCODE_LMULS_MF4(OP, TYPE) \
- CASE_VFMA_OPCODE_COMMON(OP, TYPE, MF4): \
- case CASE_VFMA_OPCODE_LMULS_MF2(OP, TYPE)
+#define CASE_VMA_OPCODE_LMULS_MF4(OP, TYPE) \
+ CASE_VMA_OPCODE_COMMON(OP, TYPE, MF4): \
+ case CASE_VMA_OPCODE_LMULS_MF2(OP, TYPE)
-#define CASE_VFMA_OPCODE_LMULS(OP, TYPE) \
- CASE_VFMA_OPCODE_COMMON(OP, TYPE, MF8): \
- case CASE_VFMA_OPCODE_LMULS_MF4(OP, TYPE)
+#define CASE_VMA_OPCODE_LMULS(OP, TYPE) \
+ CASE_VMA_OPCODE_COMMON(OP, TYPE, MF8): \
+ case CASE_VMA_OPCODE_LMULS_MF4(OP, TYPE)
+
+// VFMA instructions are SEW specific.
+#define CASE_VFMA_OPCODE_COMMON(OP, TYPE, LMUL, SEW) \
+ RISCV::PseudoV##OP##_##TYPE##_##LMUL##_##SEW
+
+#define CASE_VFMA_OPCODE_LMULS_M1(OP, TYPE, SEW) \
+ CASE_VFMA_OPCODE_COMMON(OP, TYPE, M1, SEW): \
+ case CASE_VFMA_OPCODE_COMMON(OP, TYPE, M2, SEW): \
+ case CASE_VFMA_OPCODE_COMMON(OP, TYPE, M4, SEW): \
+ case CASE_VFMA_OPCODE_COMMON(OP, TYPE, M8, SEW)
+
+#define CASE_VFMA_OPCODE_LMULS_MF2(OP, TYPE, SEW) \
+ CASE_VFMA_OPCODE_COMMON(OP, TYPE, MF2, SEW): \
+ case CASE_VFMA_OPCODE_LMULS_M1(OP, TYPE, SEW)
+
+#define CASE_VFMA_OPCODE_LMULS_MF4(OP, TYPE, SEW) \
+ CASE_VFMA_OPCODE_COMMON(OP, TYPE, MF4, SEW): \
+ case CASE_VFMA_OPCODE_LMULS_MF2(OP, TYPE, SEW)
+
+#define CASE_VFMA_OPCODE_VV(OP) \
+ CASE_VFMA_OPCODE_COMMON(OP, VV, MF4, E16): \
+ case CASE_VFMA_OPCODE_LMULS_MF2(OP, VV, E16): \
+ case CASE_VFMA_OPCODE_LMULS_MF2(OP, VV, E32)
#define CASE_VFMA_SPLATS(OP) \
- CASE_VFMA_OPCODE_LMULS_MF4(OP, VFPR16): \
- case CASE_VFMA_OPCODE_LMULS_MF2(OP, VFPR32): \
- case CASE_VFMA_OPCODE_LMULS_M1(OP, VFPR64)
+ CASE_VFMA_OPCODE_LMULS_MF4(OP, VFPR16, E16): \
+ case CASE_VFMA_OPCODE_LMULS_MF2(OP, VFPR32, E32): \
+ case CASE_VFMA_OPCODE_LMULS_M1(OP, VFPR64, E64)
// clang-format on
bool RISCVInstrInfo::findCommutedOpIndices(const MachineInstr &MI,
@@ -2667,16 +2690,16 @@ bool RISCVInstrInfo::findCommutedOpIndices(const MachineInstr &MI,
case CASE_VFMA_SPLATS(FNMSUB):
case CASE_VFMA_SPLATS(FNMACC):
case CASE_VFMA_SPLATS(FNMSAC):
- case CASE_VFMA_OPCODE_LMULS_MF4(FMACC, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FMSAC, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FNMACC, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FNMSAC, VV):
- case CASE_VFMA_OPCODE_LMULS(MADD, VX):
- case CASE_VFMA_OPCODE_LMULS(NMSUB, VX):
- case CASE_VFMA_OPCODE_LMULS(MACC, VX):
- case CASE_VFMA_OPCODE_LMULS(NMSAC, VX):
- case CASE_VFMA_OPCODE_LMULS(MACC, VV):
- case CASE_VFMA_OPCODE_LMULS(NMSAC, VV): {
+ case CASE_VFMA_OPCODE_VV(FMACC):
+ case CASE_VFMA_OPCODE_VV(FMSAC):
+ case CASE_VFMA_OPCODE_VV(FNMACC):
+ case CASE_VFMA_OPCODE_VV(FNMSAC):
+ case CASE_VMA_OPCODE_LMULS(MADD, VX):
+ case CASE_VMA_OPCODE_LMULS(NMSUB, VX):
+ case CASE_VMA_OPCODE_LMULS(MACC, VX):
+ case CASE_VMA_OPCODE_LMULS(NMSAC, VX):
+ case CASE_VMA_OPCODE_LMULS(MACC, VV):
+ case CASE_VMA_OPCODE_LMULS(NMSAC, VV): {
// If the tail policy is undisturbed we can't commute.
assert(RISCVII::hasVecPolicyOp(MI.getDesc().TSFlags));
if ((MI.getOperand(MI.getNumExplicitOperands() - 1).getImm() & 1) == 0)
@@ -2691,12 +2714,12 @@ bool RISCVInstrInfo::findCommutedOpIndices(const MachineInstr &MI,
return false;
return true;
}
- case CASE_VFMA_OPCODE_LMULS_MF4(FMADD, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FMSUB, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FNMADD, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FNMSUB, VV):
- case CASE_VFMA_OPCODE_LMULS(MADD, VV):
- case CASE_VFMA_OPCODE_LMULS(NMSUB, VV): {
+ case CASE_VFMA_OPCODE_VV(FMADD):
+ case CASE_VFMA_OPCODE_VV(FMSUB):
+ case CASE_VFMA_OPCODE_VV(FNMADD):
+ case CASE_VFMA_OPCODE_VV(FNMSUB):
+ case CASE_VMA_OPCODE_LMULS(MADD, VV):
+ case CASE_VMA_OPCODE_LMULS(NMSUB, VV): {
// If the tail policy is undisturbed we can't commute.
assert(RISCVII::hasVecPolicyOp(MI.getDesc().TSFlags));
if ((MI.getOperand(MI.getNumExplicitOperands() - 1).getImm() & 1) == 0)
@@ -2765,33 +2788,67 @@ bool RISCVInstrInfo::findCommutedOpIndices(const MachineInstr &MI,
return TargetInstrInfo::findCommutedOpIndices(MI, SrcOpIdx1, SrcOpIdx2);
}
-#define CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, LMUL) \
+#define CASE_VMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, LMUL) \
case RISCV::PseudoV##OLDOP##_##TYPE##_##LMUL: \
Opc = RISCV::PseudoV##NEWOP##_##TYPE##_##LMUL; \
break;
-#define CASE_VFMA_CHANGE_OPCODE_LMULS_M1(OLDOP, NEWOP, TYPE) \
- CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M1) \
- CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M2) \
- CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M4) \
- CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M8)
+#define CASE_VMA_CHANGE_OPCODE_LMULS_M1(OLDOP, NEWOP, TYPE) \
+ CASE_VMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M1) \
+ CASE_VMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M2) \
+ CASE_VMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M4) \
+ CASE_VMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M8)
+
+#define CASE_VMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, TYPE) \
+ CASE_VMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, MF2) \
+ CASE_VMA_CHANGE_OPCODE_LMULS_M1(OLDOP, NEWOP, TYPE)
+
+#define CASE_VMA_CHANGE_OPCODE_LMULS_MF4(OLDOP, NEWOP, TYPE) \
+ CASE_VMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, MF4) \
+ CASE_VMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, TYPE)
+
+#define CASE_VMA_CHANGE_OPCODE_LMULS(OLDOP, NEWOP, TYPE) \
+ CASE_VMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, MF8) \
+ CASE_VMA_CHANGE_OPCODE_LMULS_MF4(OLDOP, NEWOP, TYPE)
+
+#define CASE_VMA_CHANGE_OPCODE_SPLATS(OLDOP, NEWOP) \
+ CASE_VMA_CHANGE_OPCODE_LMULS_MF4(OLDOP, NEWOP, VFPR16) \
+ CASE_VMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, VFPR32) \
+ CASE_VMA_CHANGE_OPCODE_LMULS_M1(OLDOP, NEWOP, VFPR64)
+
+// VFMA depends on SEW.
+#define CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, LMUL, SEW) \
+ case RISCV::PseudoV##OLDOP##_##TYPE##_##LMUL##_##SEW: \
+ Opc = RISCV::PseudoV##NEWOP##_##TYPE##_##LMUL##_##SEW; \
+ break;
+
+#define CASE_VFMA_CHANGE_OPCODE_LMULS_M1(OLDOP, NEWOP, TYPE, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M1, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M2, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M4, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, M8, SEW)
+
+#define CASE_VFMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, TYPE, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, MF2, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_LMULS_M1(OLDOP, NEWOP, TYPE, SEW)
-#define CASE_VFMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, TYPE) \
- CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, MF2) \
- CASE_VFMA_CHANGE_OPCODE_LMULS_M1(OLDOP, NEWOP, TYPE)
+#define CASE_VFMA_CHANGE_OPCODE_VV(OLDOP, NEWOP) \
+ CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, VV, MF4, E16) \
+ CASE_VFMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, VV, E16) \
+ CASE_VFMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, VV, E32)
-#define CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(OLDOP, NEWOP, TYPE) \
- CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, MF4) \
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, TYPE)
+#define CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(OLDOP, NEWOP, TYPE, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, MF4, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, TYPE, SEW)
-#define CASE_VFMA_CHANGE_OPCODE_LMULS(OLDOP, NEWOP, TYPE) \
- CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, MF8) \
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(OLDOP, NEWOP, TYPE)
+#define CASE_VFMA_CHANGE_OPCODE_LMULS(OLDOP, NEWOP, TYPE, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_COMMON(OLDOP, NEWOP, TYPE, MF8, SEW) \
+ CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(OLDOP, NEWOP, TYPE, SEW)
#define CASE_VFMA_CHANGE_OPCODE_SPLATS(OLDOP, NEWOP) \
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(OLDOP, NEWOP, VFPR16) \
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, VFPR32) \
- CASE_VFMA_CHANGE_OPCODE_LMULS_M1(OLDOP, NEWOP, VFPR64)
+ CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(OLDOP, NEWOP, VFPR16, E16) \
+ CASE_VFMA_CHANGE_OPCODE_LMULS_MF2(OLDOP, NEWOP, VFPR32, E32) \
+ CASE_VFMA_CHANGE_OPCODE_LMULS_M1(OLDOP, NEWOP, VFPR64, E64)
MachineInstr *RISCVInstrInfo::commuteInstructionImpl(MachineInstr &MI,
bool NewMI,
@@ -2830,16 +2887,16 @@ MachineInstr *RISCVInstrInfo::commuteInstructionImpl(MachineInstr &MI,
case CASE_VFMA_SPLATS(FNMADD):
case CASE_VFMA_SPLATS(FNMSAC):
case CASE_VFMA_SPLATS(FNMSUB):
- case CASE_VFMA_OPCODE_LMULS_MF4(FMACC, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FMSAC, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FNMACC, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FNMSAC, VV):
- case CASE_VFMA_OPCODE_LMULS(MADD, VX):
- case CASE_VFMA_OPCODE_LMULS(NMSUB, VX):
- case CASE_VFMA_OPCODE_LMULS(MACC, VX):
- case CASE_VFMA_OPCODE_LMULS(NMSAC, VX):
- case CASE_VFMA_OPCODE_LMULS(MACC, VV):
- case CASE_VFMA_OPCODE_LMULS(NMSAC, VV): {
+ case CASE_VFMA_OPCODE_VV(FMACC):
+ case CASE_VFMA_OPCODE_VV(FMSAC):
+ case CASE_VFMA_OPCODE_VV(FNMACC):
+ case CASE_VFMA_OPCODE_VV(FNMSAC):
+ case CASE_VMA_OPCODE_LMULS(MADD, VX):
+ case CASE_VMA_OPCODE_LMULS(NMSUB, VX):
+ case CASE_VMA_OPCODE_LMULS(MACC, VX):
+ case CASE_VMA_OPCODE_LMULS(NMSAC, VX):
+ case CASE_VMA_OPCODE_LMULS(MACC, VV):
+ case CASE_VMA_OPCODE_LMULS(NMSAC, VV): {
// It only make sense to toggle these between clobbering the
// addend/subtrahend/minuend one of the multiplicands.
assert((OpIdx1 == 1 || OpIdx2 == 1) && "Unexpected opcode index");
@@ -2856,16 +2913,16 @@ MachineInstr *RISCVInstrInfo::commuteInstructionImpl(MachineInstr &MI,
CASE_VFMA_CHANGE_OPCODE_SPLATS(FNMADD, FNMACC)
CASE_VFMA_CHANGE_OPCODE_SPLATS(FNMSAC, FNMSUB)
CASE_VFMA_CHANGE_OPCODE_SPLATS(FNMSUB, FNMSAC)
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(FMACC, FMADD, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(FMSAC, FMSUB, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(FNMACC, FNMADD, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(FNMSAC, FNMSUB, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS(MACC, MADD, VX)
- CASE_VFMA_CHANGE_OPCODE_LMULS(MADD, MACC, VX)
- CASE_VFMA_CHANGE_OPCODE_LMULS(NMSAC, NMSUB, VX)
- CASE_VFMA_CHANGE_OPCODE_LMULS(NMSUB, NMSAC, VX)
- CASE_VFMA_CHANGE_OPCODE_LMULS(MACC, MADD, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS(NMSAC, NMSUB, VV)
+ CASE_VFMA_CHANGE_OPCODE_VV(FMACC, FMADD)
+ CASE_VFMA_CHANGE_OPCODE_VV(FMSAC, FMSUB)
+ CASE_VFMA_CHANGE_OPCODE_VV(FNMACC, FNMADD)
+ CASE_VFMA_CHANGE_OPCODE_VV(FNMSAC, FNMSUB)
+ CASE_VMA_CHANGE_OPCODE_LMULS(MACC, MADD, VX)
+ CASE_VMA_CHANGE_OPCODE_LMULS(MADD, MACC, VX)
+ CASE_VMA_CHANGE_OPCODE_LMULS(NMSAC, NMSUB, VX)
+ CASE_VMA_CHANGE_OPCODE_LMULS(NMSUB, NMSAC, VX)
+ CASE_VMA_CHANGE_OPCODE_LMULS(MACC, MADD, VV)
+ CASE_VMA_CHANGE_OPCODE_LMULS(NMSAC, NMSUB, VV)
}
auto &WorkingMI = cloneIfNew(MI);
@@ -2873,12 +2930,12 @@ MachineInstr *RISCVInstrInfo::commuteInstructionImpl(MachineInstr &MI,
return TargetInstrInfo::commuteInstructionImpl(WorkingMI, /*NewMI=*/false,
OpIdx1, OpIdx2);
}
- case CASE_VFMA_OPCODE_LMULS_MF4(FMADD, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FMSUB, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FNMADD, VV):
- case CASE_VFMA_OPCODE_LMULS_MF4(FNMSUB, VV):
- case CASE_VFMA_OPCODE_LMULS(MADD, VV):
- case CASE_VFMA_OPCODE_LMULS(NMSUB, VV): {
+ case CASE_VFMA_OPCODE_VV(FMADD):
+ case CASE_VFMA_OPCODE_VV(FMSUB):
+ case CASE_VFMA_OPCODE_VV(FNMADD):
+ case CASE_VFMA_OPCODE_VV(FNMSUB):
+ case CASE_VMA_OPCODE_LMULS(MADD, VV):
+ case CASE_VMA_OPCODE_LMULS(NMSUB, VV): {
assert((OpIdx1 == 1 || OpIdx2 == 1) && "Unexpected opcode index");
// If one of the operands, is the addend we need to change opcode.
// Otherwise we're just swapping 2 of the multiplicands.
@@ -2887,12 +2944,12 @@ MachineInstr *RISCVInstrInfo::commuteInstructionImpl(MachineInstr &MI,
switch (MI.getOpcode()) {
default:
llvm_unreachable("Unexpected opcode");
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(FMADD, FMACC, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(FMSUB, FMSAC, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(FNMADD, FNMACC, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS_MF4(FNMSUB, FNMSAC, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS(MADD, MACC, VV)
- CASE_VFMA_CHANGE_OPCODE_LMULS(NMSUB, NMSAC, VV)
+ CASE_VFMA_CHANGE_OPCODE_VV(FMADD, FMACC)
+ CASE_VFMA_CHANGE_OPCODE_VV(FMSUB, FMSAC)
+ CASE_VFMA_CHANGE_OPCODE_VV(FNMADD, FNMACC)
+ CASE_VFMA_CHANGE_OPCODE_VV(FNMSUB, FNMSAC)
+ CASE_VMA_CHANGE_OPCODE_LMULS(MADD, MACC, VV)
+ CASE_VMA_CHANGE_OPCODE_LMULS(NMSUB, NMSAC, VV)
}
auto &WorkingMI = cloneIfNew(MI);
@@ -2908,12 +2965,17 @@ MachineInstr *RISCVInstrInfo::commuteInstructionImpl(MachineInstr &MI,
return TargetInstrInfo::commuteInstructionImpl(MI, NewMI, OpIdx1, OpIdx2);
}
-#undef CASE_VFMA_CHANGE_OPCODE_SPLATS
-#undef CASE_VFMA_CHANGE_OPCODE_LMULS
-#undef CASE_VFMA_CHANGE_OPCODE_COMMON
-#undef CASE_VFMA_SPLATS
-#undef CASE_VFMA_OPCODE_LMULS
+#undef CASE_VMA_OPCODE_COMMON
+#undef CASE_VMA_OPCODE_LMULS_M1
+#undef CASE_VMA_OPCODE_LMULS_MF2
+#undef CASE_VMA_OPCODE_LMULS_MF4
+#undef CASE_VMA_OPCODE_LMULS
#undef CASE_VFMA_OPCODE_COMMON
+#undef CASE_VFMA_OPCODE_LMULS_M1
+#undef CASE_VFMA_OPCODE_LMULS_MF2
+#undef CASE_VFMA_OPCODE_LMULS_MF4
+#undef CASE_VFMA_OPCODE_VV
+#undef CASE_VFMA_SPLATS
// clang-format off
#define CASE_WIDEOP_OPCODE_COMMON(OP, LMUL) \
@@ -2947,6 +3009,40 @@ MachineInstr *RISCVInstrInfo::commuteInstructionImpl(MachineInstr &MI,
CASE_WIDEOP_CHANGE_OPCODE_COMMON(OP, MF8) \
CASE_WIDEOP_CHANGE_OPCODE_LMULS_MF4(OP)
+// FP Widening Ops may by SEW aware. Create SEW aware cases for these cases.
+#define CASE_FP_WIDEOP_OPCODE_COMMON(OP, LMUL, SEW) \
+ RISCV::PseudoV##OP##_##LMUL##_##SEW##_TIED
+
+#define CASE_FP_WIDEOP_OPCODE_LMULS_MF4(OP) \
+ CASE_FP_WIDEOP_OPCODE_COMMON(OP, MF4, E16): \
+ case CASE_FP_WIDEOP_OPCODE_COMMON(OP, MF2, E16): \
+ case CASE_FP_WIDEOP_OPCODE_COMMON(OP, MF2, E32): \
+ case CASE_FP_WIDEOP_OPCODE_COMMON(OP, M1, E16): \
+ case CASE_FP_WIDEOP_OPCODE_COMMON(OP, M1, E32): \
+ case CASE_FP_WIDEOP_OPCODE_COMMON(OP, M2, E16): \
+ case CASE_FP_WIDEOP_OPCODE_COMMON(OP, M2, E32): \
+ case CASE_FP_WIDEOP_OPCODE_COMMON(OP, M4, E16): \
+ case CASE_FP_WIDEOP_OPCODE_COMMON(OP, M4, E32) \
+
+#define CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, LMUL, SEW) \
+ case RISCV::PseudoV##OP##_##LMUL##_##SEW##_TIED: \
+ NewOpc = RISCV::PseudoV##OP##_##LMUL##_##SEW; \
+ break;
+
+#define CASE_FP_WIDEOP_CHANGE_OPCODE_LMULS_MF4(OP) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, MF4, E16) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, MF2, E16) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, MF2, E32) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, M1, E16) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, M1, E32) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, M2, E16) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, M2, E32) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, M4, E16) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_COMMON(OP, M4, E32) \
+
+#define CASE_FP_WIDEOP_CHANGE_OPCODE_LMULS(OP) \
+ CASE_FP_WIDEOP_CHANGE_OPCODE_LMULS_MF4(OP)
+
MachineInstr *RISCVInstrInfo::convertToThreeAddress(MachineInstr &MI,
LiveVariables *LV,
LiveIntervals *LIS) const {
@@ -2954,8 +3050,8 @@ MachineInstr *RISCVInstrInfo::convertToThreeAddress(MachineInstr &MI,
switch (MI.getOpcode()) {
default:
return nullptr;
- case CASE_WIDEOP_OPCODE_LMULS_MF4(FWADD_WV):
- case CASE_WIDEOP_OPCODE_LMULS_MF4(FWSUB_WV): {
+ case CASE_FP_WIDEOP_OPCODE_LMULS_MF4(FWADD_WV):
+ case CASE_FP_WIDEOP_OPCODE_LMULS_MF4(FWSUB_WV): {
assert(RISCVII::hasVecPolicyOp(MI.getDesc().TSFlags) &&
MI.getNumExplicitOperands() == 7 &&
"Expect 7 explicit operands rd, rs2, rs1, rm, vl, sew, policy");
@@ -2968,8 +3064,8 @@ MachineInstr *RISCVInstrInfo::convertToThreeAddress(MachineInstr &MI,
switch (MI.getOpcode()) {
default:
llvm_unreachable("Unexpected opcode");
- CASE_WIDEOP_CHA...
[truncated]
|
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
michaelmaitland
force-pushed
the
vfalu-sew-aware-new
branch
from
19655c7
to
99fae49
Compare
wangpc-pp
reviewed
Apr 8, 2024
michaelmaitland
force-pushed
the
vfalu-sew-aware-new
branch
from
cc9eea5
to
c3999fc
Compare
michaelmaitland
force-pushed
the
vfalu-sew-aware-new
branch
from
c3999fc
to
dd769e6
Compare
I am seeing the MatcherTable size stay the same at 2216191 bytes. |
topperc
reviewed
Apr 8, 2024
| case CASE_VFMA_OPCODE_LMULS_MF2(OP, TYPE, SEW) | ||
|
|
||
| #define CASE_VFMA_OPCODE_VV(OP) \ | ||
| CASE_VFMA_OPCODE_COMMON(OP, VV, MF4, E16): \ |
There was a problem hiding this comment.
Can the first two lines of these be combined to CASE_VFMA_OPCODE_LMULS_MF4(OP, VV, E16)?
There was a problem hiding this comment.
Are we missing test coverage for E64?
There was a problem hiding this comment.
I just pushed a commit which was failing because I forgot to update checks for E64.
There was a problem hiding this comment.
Did failures exist yesterday when I made the original comment?
There was a problem hiding this comment.
Yes. I missed them because the CI was not printing failed cases at the bottom of the log like it used to.
michaelmaitland
force-pushed
the
vfalu-sew-aware-new
branch
from
86ef6a6
to
10ceba0
Compare
Co-authored-by: Wang Pengcheng <wangpengcheng.pp@bytedance.com>
Co-authored-by: Wang Pengcheng <wangpengcheng.pp@bytedance.com>
Co-authored-by: Wang Pengcheng <wangpengcheng.pp@bytedance.com>
…nstructions by SEW Co-authored-by: Wang Pengcheng <wangpengcheng.pp@bytedance.com>
…uctions by SEW Co-authored-by: Wang Pengcheng <wangpengcheng.pp@bytedance.com>
Co-authored-by: Wang Pengcheng <wangpengcheng.pp@bytedance.com>
michaelmaitland
force-pushed
the
vfalu-sew-aware-new
branch
from
10ceba0
to
6bc375a
Compare
michaelmaitland
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch contains a series of commits to split vector FP pseudo instructions by SEW. This is intended to be the equivalent of https://reviews.llvm.org/D153000. I will add @wangpc-pp as co-author. Merging this PR closes #87404. Scheduler changes depending on SEW can come in future patches.
There are still some FP instructions that can be split, but they were not split in https://reviews.llvm.org/D153000 so I did not split them here. We can split those in a follow up patch or in this one if we'd like.