Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RISCV] Add Sched classes for vector crypto instructions #90068

Merged
merged 5 commits into from
May 3, 2024
Merged

[RISCV] Add Sched classes for vector crypto instructions #90068

merged 5 commits into from
May 3, 2024

Conversation

michaelmaitland
Copy link
Contributor 

The vector crypto instructions may have different scheduling behavior
compared to VALU operations. Instead of using scheduling resources that
describe VALU operations, we give these instructions their own
scheduling resources. This is similar to what we did for Zb* instructions.

The sifive-p670 has vector crypto, so we model behavior for these instructions
in the P600SchedModel. The numbers are based off of measurements collected
internally. These numbers are a bit old and new measurements show that they may
not be fully accurate. It is likely that we will refine these numbers in a
follow up patch(s) based on new measurements.

This PR is stacked on #89256.

@llvmbot
Copy link
Collaborator

llvmbot commented Apr 25, 2024

@llvm/pr-subscribers-backend-risc-v

Author: Michael Maitland (michaelmaitland)

Changes 
The vector crypto instructions may have different scheduling behavior
compared to VALU operations. Instead of using scheduling resources that
describe VALU operations, we give these instructions their own
scheduling resources. This is similar to what we did for Zb* instructions.

The sifive-p670 has vector crypto, so we model behavior for these instructions
in the P600SchedModel. The numbers are based off of measurements collected
internally. These numbers are a bit old and new measurements show that they may
not be fully accurate. It is likely that we will refine these numbers in a
follow up patch(s) based on new measurements.

This PR is stacked on #<!-- -->89256.

Patch is 110.35 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/90068.diff

16 Files Affected:

  • (modified) llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td (+123-33)
  • (modified) llvm/lib/Target/RISCV/RISCVSchedRocket.td (+1)
  • (modified) llvm/lib/Target/RISCV/RISCVSchedSiFive7.td (+1)
  • (modified) llvm/lib/Target/RISCV/RISCVSchedSiFiveP400.td (+1)
  • (modified) llvm/lib/Target/RISCV/RISCVSchedSiFiveP600.td (+81)
  • (modified) llvm/lib/Target/RISCV/RISCVSchedSyntacoreSCR1.td (+1)
  • (modified) llvm/lib/Target/RISCV/RISCVSchedXiangShanNanHu.td (+1)
  • (modified) llvm/lib/Target/RISCV/RISCVSchedule.td (+1)
  • (added) llvm/lib/Target/RISCV/RISCVScheduleZvk.td (+202)
  • (added) llvm/test/tools/llvm-mca/RISCV/SiFiveP600/zvbb.s (+461)
  • (added) llvm/test/tools/llvm-mca/RISCV/SiFiveP600/zvbc.s (+113)
  • (added) llvm/test/tools/llvm-mca/RISCV/SiFiveP600/zvkg.s (+128)
  • (added) llvm/test/tools/llvm-mca/RISCV/SiFiveP600/zvkned.s (+204)
  • (added) llvm/test/tools/llvm-mca/RISCV/SiFiveP600/zvknhb.s (+153)
  • (added) llvm/test/tools/llvm-mca/RISCV/SiFiveP600/zvksed.s (+114)
  • (added) llvm/test/tools/llvm-mca/RISCV/SiFiveP600/zvksh.s (+99)
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td b/llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td
index aac7dc444a2de3..d60aea186aa5fe 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td
@@ -24,11 +24,9 @@ def tuimm5 : RISCVOp, TImmLeaf<XLenVT, [{return isUInt<5>(Imm);}]>;
 let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in {
 multiclass VCLMUL_MV_V_X<string opcodestr, bits<6> funct6> {
   def V  : VALUVV<funct6, OPMVV, opcodestr # "." # "vv">,
-           Sched<[WriteVIALUV_WorstCase, ReadVIALUV_WorstCase,
-                  ReadVIALUV_WorstCase, ReadVMask]>;
+           SchedBinaryMC<"WriteVCLMUL", "ReadVCLMUL", "ReadVCLMUL">;
   def X  : VALUVX<funct6, OPMVX, opcodestr # "." # "vx">,
-           Sched<[WriteVIALUX_WorstCase, ReadVIALUV_WorstCase,
-                  ReadVIALUX_WorstCase, ReadVMask]>;
+           SchedBinaryMC<"WriteVCLMULH", "ReadVCLMULH", "ReadVCLMULH">;
 }
 
 class RVInstIVI_VROR<bits<6> funct6, dag outs, dag ins, string opcodestr,
@@ -57,8 +55,7 @@ multiclass VROR_IV_V_X_I<string opcodestr, bits<6> funct6>
   def I : RVInstIVI_VROR<funct6, (outs VR:$vd),
               (ins VR:$vs2, uimm6:$imm, VMaskOp:$vm),
               opcodestr # ".vi", "$vd, $vs2, $imm$vm">,
-         Sched<[WriteVIALUI_WorstCase, ReadVIALUV_WorstCase,
-                ReadVMask]>;
+          SchedUnaryMC<"WriteVROR", "ReadVROR">;
 }
 
 // op vd, vs2, vs1
@@ -109,9 +106,11 @@ class PALUVs2NoVmBinary<bits<6> funct6, bits<5> vs1, RISCVVFormat opv,
 multiclass VAES_MV_V_S<bits<6> funct6_vv, bits<6> funct6_vs, bits<5> vs1,
                          RISCVVFormat opv, string opcodestr> {
   let RVVConstraint = NoConstraint in
-  def NAME # _VV : PALUVs2NoVmBinary<funct6_vv, vs1, opv, opcodestr # ".vv">;
+  def NAME # _VV : PALUVs2NoVmBinary<funct6_vv, vs1, opv, opcodestr # ".vv">,
+                   SchedBinaryMC<"WriteVAESMV", "ReadVAESMV", "ReadVAESMV">;
   let RVVConstraint = VS2Constraint in
-  def NAME # _VS : PALUVs2NoVmBinary<funct6_vs, vs1, opv, opcodestr # ".vs">;
+  def NAME # _VS : PALUVs2NoVmBinary<funct6_vs, vs1, opv, opcodestr # ".vs">,
+                   SchedBinaryMC<"WriteVAESMV", "ReadVAESMV", "ReadVAESMV">;
 }
 } // hasSideEffects = 0, mayLoad = 0, mayStore = 0
 
@@ -142,14 +141,23 @@ let Predicates = [HasStdExtZvkb] in {
 } // Predicates = [HasStdExtZvkb]
 
 let Predicates = [HasStdExtZvkg], RVVConstraint = NoConstraint in {
-  def VGHSH_VV : PALUVVNoVmTernary<0b101100, OPMVV, "vghsh.vv">;
-  def VGMUL_VV : PALUVs2NoVmBinary<0b101000, 0b10001, OPMVV, "vgmul.vv">;
+  def VGHSH_VV : PALUVVNoVmTernary<0b101100, OPMVV, "vghsh.vv">,
+                 SchedTernaryMC<"WriteVGHSH", "ReadVGHSH", "ReadVGHSH",
+                                "ReadVGHSH">;
+  def VGMUL_VV : PALUVs2NoVmBinary<0b101000, 0b10001, OPMVV, "vgmul.vv">,
+                 SchedBinaryMC<"WriteVGMUL", "ReadVGMUL", "ReadVGMUL">;
 } // Predicates = [HasStdExtZvkg]
 
 let Predicates = [HasStdExtZvknhaOrZvknhb], RVVConstraint = Sha2Constraint in {
-  def VSHA2CH_VV : PALUVVNoVmTernary<0b101110, OPMVV, "vsha2ch.vv">;
-  def VSHA2CL_VV : PALUVVNoVmTernary<0b101111, OPMVV, "vsha2cl.vv">;
-  def VSHA2MS_VV : PALUVVNoVmTernary<0b101101, OPMVV, "vsha2ms.vv">;
+  def VSHA2CH_VV : PALUVVNoVmTernary<0b101110, OPMVV, "vsha2ch.vv">,
+                   SchedTernaryMC<"WriteVSHA2CH", "ReadVSHA2CH", "ReadVSHA2CH",
+                                  "ReadVSHA2CH">;
+  def VSHA2CL_VV : PALUVVNoVmTernary<0b101111, OPMVV, "vsha2cl.vv">,
+                   SchedTernaryMC<"WriteVSHA2CL", "ReadVSHA2CL", "ReadVSHA2CL",
+                                  "ReadVSHA2CL">;
+  def VSHA2MS_VV : PALUVVNoVmTernary<0b101101, OPMVV, "vsha2ms.vv">,
+                   SchedTernaryMC<"WriteVSHA2MS", "ReadVSHA2MS", "ReadVSHA2MS",
+                                  "ReadVSHA2MS">;
 } // Predicates = [HasStdExtZvknhaOrZvknhb]
 
 let Predicates = [HasStdExtZvkned] in {
@@ -157,21 +165,27 @@ let Predicates = [HasStdExtZvkned] in {
   defm VAESDM     : VAES_MV_V_S<0b101000, 0b101001, 0b00000, OPMVV, "vaesdm">;
   defm VAESEF     : VAES_MV_V_S<0b101000, 0b101001, 0b00011, OPMVV, "vaesef">;
   defm VAESEM     : VAES_MV_V_S<0b101000, 0b101001, 0b00010, OPMVV, "vaesem">;
-  def  VAESKF1_VI : PALUVINoVm<0b100010, "vaeskf1.vi", uimm5>;
-  def  VAESKF2_VI : PALUVINoVmBinary<0b101010, "vaeskf2.vi", uimm5>;
+  def  VAESKF1_VI : PALUVINoVm<0b100010, "vaeskf1.vi", uimm5>,
+                    SchedUnaryMC<"WriteVAESKF1", "ReadVAESKF2">;
+  def  VAESKF2_VI : PALUVINoVmBinary<0b101010, "vaeskf2.vi", uimm5>,
+                    SchedBinaryMC<"WriteVAESKF2", "ReadVAESKF2", "ReadVAESKF2">;
   let RVVConstraint = VS2Constraint in
-  def  VAESZ_VS   : PALUVs2NoVmBinary<0b101001, 0b00111, OPMVV, "vaesz.vs">;
+  def  VAESZ_VS   : PALUVs2NoVmBinary<0b101001, 0b00111, OPMVV, "vaesz.vs">,
+                    SchedBinaryMC<"WriteVAESZ", "ReadVAESZ", "ReadVAESZ">;
 } // Predicates = [HasStdExtZvkned]
 
 let Predicates = [HasStdExtZvksed] in {
   let RVVConstraint = NoConstraint in
-  def  VSM4K_VI : PALUVINoVm<0b100001, "vsm4k.vi", uimm5>;
+  def  VSM4K_VI : PALUVINoVm<0b100001, "vsm4k.vi", uimm5>,
+                  SchedUnaryMC<"WriteVSM4K", "ReadVSM4K">;
   defm VSM4R    : VAES_MV_V_S<0b101000, 0b101001, 0b10000, OPMVV, "vsm4r">;
 } // Predicates = [HasStdExtZvksed]
 
 let Predicates = [HasStdExtZvksh], RVVConstraint = VS2Constraint in {
-  def VSM3C_VI  : PALUVINoVmBinary<0b101011, "vsm3c.vi", uimm5>;
-  def VSM3ME_VV : PALUVVNoVm<0b100000, OPMVV, "vsm3me.vv">;
+  def VSM3C_VI  : PALUVINoVmBinary<0b101011, "vsm3c.vi", uimm5>,
+                  SchedBinaryMC<"WriteVSM3C", "ReadVSM3C", "ReadVSM3C">;
+  def VSM3ME_VV : PALUVVNoVm<0b100000, OPMVV, "vsm3me.vv">,
+                  SchedUnaryMC<"WriteVSM3ME", "ReadVSM3ME">;
 } // Predicates = [HasStdExtZvksh]
 
 //===----------------------------------------------------------------------===//
@@ -323,10 +337,10 @@ multiclass VPseudoVCLMUL_VV_VX {
   foreach m = MxList in {
     defvar mx = m.MX;
     defm "" : VPseudoBinaryV_VV<m>,
-              SchedBinary<"WriteVIALUV", "ReadVIALUV", "ReadVIALUV", mx,
+              SchedBinary<"WriteVCLMUL", "ReadVCLMUL", "ReadVCLMUL", mx,
                           forceMergeOpRead=true>;
     defm "" : VPseudoBinaryV_VX<m>,
-              SchedBinary<"WriteVIALUX", "ReadVIALUV", "ReadVIALUX", mx,
+              SchedBinary<"WriteVCLMULH", "ReadVCLMULH", "ReadVCLMULH", mx,
                           forceMergeOpRead=true>;
   }
 }
@@ -340,28 +354,104 @@ multiclass VPseudoUnaryV_V<LMULInfo m> {
   }
 }
 
-multiclass VPseudoVALU_V {
+multiclass VPseudoVBREV {
   foreach m = MxList in {
     defvar mx = m.MX;
     defm "" : VPseudoUnaryV_V<m>,
-              SchedUnary<"WriteVIALUV", "ReadVIALUV", mx,
-                         forceMergeOpRead=true>;
+              SchedUnary<"WriteVBREV", "ReadVBREV", mx, forceMergeOpRead=true>;
+  }
+}
+
+multiclass VPseudoVCLZ {
+  foreach m = MxList in {
+    defvar mx = m.MX;
+    defm "" : VPseudoUnaryV_V<m>,
+              SchedUnary<"WriteVCLZ", "ReadVCLZ", mx, forceMergeOpRead=true>;
+  }
+}
+
+multiclass VPseudoVCTZ {
+  foreach m = MxList in {
+    defvar mx = m.MX;
+    defm "" : VPseudoUnaryV_V<m>,
+              SchedUnary<"WriteVCTZ", "ReadVCTZ", mx, forceMergeOpRead=true>;
+  }
+}
+
+multiclass VPseudoVCPOP {
+  foreach m = MxList in {
+    defvar mx = m.MX;
+    defm "" : VPseudoUnaryV_V<m>,
+              SchedUnary<"WriteVCPOP", "ReadVCPOP", mx, forceMergeOpRead=true>;
   }
 }
 
 multiclass VPseudoVWALU_VV_VX_VI<Operand ImmType> : VPseudoVWALU_VV_VX {
   foreach m = MxListW in {
     defm "" : VPseudoBinaryW_VI<ImmType, m>,
-              SchedUnary<"WriteVIWALUV", "ReadVIWALUV", m.MX,
+              SchedUnary<"WriteVWSLL", "ReadVWSLL", m.MX,
                          forceMergeOpRead=true>;
   }
 }
 
+multiclass VPseudoVANDN {
+ foreach m = MxList in {
+    defm "" : VPseudoBinaryV_VV<m>,
+            SchedBinary<"WriteVANDN", "ReadVANDN", "ReadVANDN", m.MX,
+                        forceMergeOpRead=true>;
+    defm "" : VPseudoBinaryV_VX<m>,
+            SchedBinary<"WriteVANDN", "ReadVANDN", "ReadVANDN", m.MX,
+                        forceMergeOpRead=true>;
+  }
+}
+
+multiclass VPseudoVBREV8 {
+  foreach m = MxList in {
+    defvar mx = m.MX;
+    defm "" : VPseudoUnaryV_V<m>,
+              SchedUnary<"WriteVBREV8", "ReadVBREV8", mx, forceMergeOpRead=true>;
+  }
+}
+
+multiclass VPseudoVREV8 {
+  foreach m = MxList in {
+    defvar mx = m.MX;
+    defm "" : VPseudoUnaryV_V<m>,
+              SchedUnary<"WriteVREV8", "ReadVREV8", mx, forceMergeOpRead=true>;
+  }
+}
+
+multiclass VPseudoVROL {
+ foreach m = MxList in {
+    defm "" : VPseudoBinaryV_VV<m>,
+            SchedBinary<"WriteVROL", "ReadVROL", "ReadVROL", m.MX,
+                        forceMergeOpRead=true>;
+    defm "" : VPseudoBinaryV_VX<m>,
+            SchedBinary<"WriteVROL", "ReadVROL", "ReadVROL", m.MX,
+                        forceMergeOpRead=true>;
+  }
+}
+
+multiclass VPseudoVROR<Operand ImmType> {
+  defvar Constraint = "";
+  foreach m = MxList in {
+    defvar mx = m.MX;
+    defm "" : VPseudoBinaryV_VV<m>,
+            SchedBinary<"WriteVROR", "ReadVROR", "ReadVROR", mx,
+                        forceMergeOpRead=true>;
+    defm "" : VPseudoBinaryV_VX<m>,
+            SchedBinary<"WriteVROR", "ReadVROR", "ReadVROR", mx,
+                        forceMergeOpRead=true>;
+    defm "" : VPseudoBinaryV_VI<ImmType, m>,
+            SchedUnary<"WriteVROR", "ReadVROR", mx, forceMergeOpRead=true>;
+  }
+}
+
 let Predicates = [HasStdExtZvbb] in {
-  defm PseudoVBREV  : VPseudoVALU_V;
-  defm PseudoVCLZ   : VPseudoVALU_V;
-  defm PseudoVCTZ   : VPseudoVALU_V;
-  defm PseudoVCPOP  : VPseudoVALU_V;
+  defm PseudoVBREV  : VPseudoVBREV;
+  defm PseudoVCLZ   : VPseudoVCLZ;
+  defm PseudoVCTZ   : VPseudoVCTZ;
+  defm PseudoVCPOP  : VPseudoVCPOP;
   defm PseudoVWSLL : VPseudoVWALU_VV_VX_VI<uimm5>;
 } // Predicates = [HasStdExtZvbb]
 
@@ -371,10 +461,10 @@ let Predicates = [HasStdExtZvbc] in {
 } // Predicates = [HasStdExtZvbc]
 
 let Predicates = [HasStdExtZvkb] in {
-  defm PseudoVANDN  : VPseudoVALU_VV_VX;
-  defm PseudoVBREV8 : VPseudoVALU_V;
-  defm PseudoVREV8  : VPseudoVALU_V;
-  defm PseudoVROL   : VPseudoVALU_VV_VX;
+  defm PseudoVANDN  : VPseudoVANDN;
+  defm PseudoVBREV8 : VPseudoVBREV8;
+  defm PseudoVREV8  : VPseudoVREV8;
+  defm PseudoVROL   : VPseudoVROL;
   defm PseudoVROR   : VPseudoVALU_VV_VX_VI<uimm6>;
 } // Predicates = [HasStdExtZvkb]
 
diff --git a/llvm/lib/Target/RISCV/RISCVSchedRocket.td b/llvm/lib/Target/RISCV/RISCVSchedRocket.td
index 65494e73758d63..9ddc4281092dd1 100644
--- a/llvm/lib/Target/RISCV/RISCVSchedRocket.td
+++ b/llvm/lib/Target/RISCV/RISCVSchedRocket.td
@@ -262,4 +262,5 @@ defm : UnsupportedSchedZfa;
 defm : UnsupportedSchedZfh;
 defm : UnsupportedSchedSFB;
 defm : UnsupportedSchedXsfvcp;
+defm : UnsupportedSchedZvk;
 }
diff --git a/llvm/lib/Target/RISCV/RISCVSchedSiFive7.td b/llvm/lib/Target/RISCV/RISCVSchedSiFive7.td
index a532066b3a1c83..e67da839bdb876 100644
--- a/llvm/lib/Target/RISCV/RISCVSchedSiFive7.td
+++ b/llvm/lib/Target/RISCV/RISCVSchedSiFive7.td
@@ -1298,4 +1298,5 @@ defm : UnsupportedSchedZbc;
 defm : UnsupportedSchedZbkb;
 defm : UnsupportedSchedZbkx;
 defm : UnsupportedSchedZfa;
+defm : UnsupportedSchedZvk;
 }
diff --git a/llvm/lib/Target/RISCV/RISCVSchedSiFiveP400.td b/llvm/lib/Target/RISCV/RISCVSchedSiFiveP400.td
index fccdd7e4f3ec2e..a37958826e028a 100644
--- a/llvm/lib/Target/RISCV/RISCVSchedSiFiveP400.td
+++ b/llvm/lib/Target/RISCV/RISCVSchedSiFiveP400.td
@@ -367,4 +367,5 @@ defm : UnsupportedSchedSFB;
 defm : UnsupportedSchedZfa;
 defm : UnsupportedSchedV;
 defm : UnsupportedSchedXsfvcp;
+defm : UnsupportedSchedZvk;
 }
diff --git a/llvm/lib/Target/RISCV/RISCVSchedSiFiveP600.td b/llvm/lib/Target/RISCV/RISCVSchedSiFiveP600.td
index 6e4fb19361f553..cf81101401af84 100644
--- a/llvm/lib/Target/RISCV/RISCVSchedSiFiveP600.td
+++ b/llvm/lib/Target/RISCV/RISCVSchedSiFiveP600.td
@@ -748,6 +748,60 @@ foreach mx = SchedMxList in {
   }
 }
 
+// Vector Crypto
+foreach mx = SchedMxList in {
+  defvar LMulLat = SiFiveP600GetLMulCycles<mx>.c;
+  defvar IsWorstCase = SiFiveP600IsWorstCaseMX<mx, SchedMxList>.c;
+  // Zvbb
+  let Latency = 2, ReleaseAtCycles = [LMulLat] in {
+    defm "" : LMULWriteResMX<"WriteVBREV",   [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVCLZ",    [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVCPOP",   [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVCTZ",    [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVWSLL",   [SiFiveP600VectorArith], mx, IsWorstCase>;
+  }
+  // Zvbc
+  let Latency = 2, ReleaseAtCycles = [LMulLat] in {
+    defm "" : LMULWriteResMX<"WriteVCLMUL",  [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVCLMULH", [SiFiveP600VectorArith], mx, IsWorstCase>;
+  }
+  // Zvkb
+  let Latency = 1, ReleaseAtCycles = [LMulLat] in
+  defm "" : LMULWriteResMX<"WriteVANDN",   [SiFiveP600VectorArith], mx, IsWorstCase>;
+  let Latency = 2, ReleaseAtCycles = [LMulLat] in {
+    defm "" : LMULWriteResMX<"WriteVBREV8",  [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVREV8",   [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVROL",    [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVROR",    [SiFiveP600VectorArith], mx, IsWorstCase>;
+  }
+  // Zvkg
+  let Latency = 2, ReleaseAtCycles = [LMulLat] in {
+    defm "" : LMULWriteResMX<"WriteVGHSH",   [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVGMUL",   [SiFiveP600VectorArith], mx, IsWorstCase>;
+  }
+  // ZvknhaOrZvknhb
+  let Latency = 3, ReleaseAtCycles = [LMulLat] in {
+    defm "" : LMULWriteResMX<"WriteVSHA2CH", [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVSHA2CL", [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVSHA2MS", [SiFiveP600VectorArith], mx, IsWorstCase>;
+  }
+  // Zvkned
+  let Latency = 2, ReleaseAtCycles = [LMulLat] in {
+    defm "" : LMULWriteResMX<"WriteVAESMV",  [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVAESKF1", [SiFiveP600VectorArith], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVAESKF2", [SiFiveP600VectorArith], mx, IsWorstCase>;
+  }
+  let Latency = 1, ReleaseAtCycles = [LMulLat] in
+  defm "" : LMULWriteResMX<"WriteVAESZ",   [SiFiveP600VectorArith], mx, IsWorstCase>;
+  // Zvksed
+  let Latency = 3, ReleaseAtCycles = [LMulLat] in {
+    defm "" : LMULWriteResMX<"WriteVSM4K",   [SiFiveP600VEXQ0], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVSM4R",   [SiFiveP600VEXQ0], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVSM3C",   [SiFiveP600VEXQ0], mx, IsWorstCase>;
+    defm "" : LMULWriteResMX<"WriteVSM3ME",  [SiFiveP600VEXQ0], mx, IsWorstCase>;
+  }
+}
+
 // Others
 def : WriteRes<WriteCSR, [SiFiveP600SYS]>;
 def : WriteRes<WriteNop, []>;
@@ -1032,6 +1086,33 @@ foreach mx = SchedMxList in {
     def : ReadAdvance<!cast<SchedRead>("ReadVMergeOp_" # mx  # "_E" # sew), 0>;
 }
 
+// Vector Crypto Extensions
+defm "" : LMULReadAdvance<"ReadVBREV", 0>;
+defm "" : LMULReadAdvance<"ReadVCLZ", 0>;
+defm "" : LMULReadAdvance<"ReadVCPOP", 0>;
+defm "" : LMULReadAdvance<"ReadVCTZ", 0>;
+defm "" : LMULReadAdvance<"ReadVWSLL", 0>;
+defm "" : LMULReadAdvance<"ReadVCLMUL", 0>;
+defm "" : LMULReadAdvance<"ReadVCLMULH", 0>;
+defm "" : LMULReadAdvance<"ReadVANDN", 0>;
+defm "" : LMULReadAdvance<"ReadVBREV8", 0>;
+defm "" : LMULReadAdvance<"ReadVREV8", 0>;
+defm "" : LMULReadAdvance<"ReadVROL", 0>;
+defm "" : LMULReadAdvance<"ReadVROR", 0>;
+defm "" : LMULReadAdvance<"ReadVGHSH", 0>;
+defm "" : LMULReadAdvance<"ReadVGMUL", 0>;
+defm "" : LMULReadAdvance<"ReadVSHA2CH", 0>;
+defm "" : LMULReadAdvance<"ReadVSHA2CL", 0>;
+defm "" : LMULReadAdvance<"ReadVSHA2MS", 0>;
+defm "" : LMULReadAdvance<"ReadVAESMV", 0>;
+defm "" : LMULReadAdvance<"ReadVAESKF1", 0>;
+defm "" : LMULReadAdvance<"ReadVAESKF2", 0>;
+defm "" : LMULReadAdvance<"ReadVAESZ", 0>;
+defm "" : LMULReadAdvance<"ReadVSM4K", 0>;
+defm "" : LMULReadAdvance<"ReadVSM4R", 0>;
+defm "" : LMULReadAdvance<"ReadVSM3C", 0>;
+defm "" : LMULReadAdvance<"ReadVSM3ME", 0>;
+
 //===----------------------------------------------------------------------===//
 // Unsupported extensions
 defm : UnsupportedSchedZabha;
diff --git a/llvm/lib/Target/RISCV/RISCVSchedSyntacoreSCR1.td b/llvm/lib/Target/RISCV/RISCVSchedSyntacoreSCR1.td
index 0885e325f24e68..31112d140cde7a 100644
--- a/llvm/lib/Target/RISCV/RISCVSchedSyntacoreSCR1.td
+++ b/llvm/lib/Target/RISCV/RISCVSchedSyntacoreSCR1.td
@@ -213,4 +213,5 @@ defm : UnsupportedSchedZbkx;
 defm : UnsupportedSchedZfa;
 defm : UnsupportedSchedZfh;
 defm : UnsupportedSchedXsfvcp;
+defm : UnsupportedSchedZvk;
 }
diff --git a/llvm/lib/Target/RISCV/RISCVSchedXiangShanNanHu.td b/llvm/lib/Target/RISCV/RISCVSchedXiangShanNanHu.td
index e0f1fab1d6b409..dcd1a938a91472 100644
--- a/llvm/lib/Target/RISCV/RISCVSchedXiangShanNanHu.td
+++ b/llvm/lib/Target/RISCV/RISCVSchedXiangShanNanHu.td
@@ -312,4 +312,5 @@ defm : UnsupportedSchedZfh;
 defm : UnsupportedSchedSFB;
 defm : UnsupportedSchedZabha;
 defm : UnsupportedSchedXsfvcp;
+defm : UnsupportedSchedZvk;
 }
diff --git a/llvm/lib/Target/RISCV/RISCVSchedule.td b/llvm/lib/Target/RISCV/RISCVSchedule.td
index 0086557a41fe7c..d9a2e38c0e9d73 100644
--- a/llvm/lib/Target/RISCV/RISCVSchedule.td
+++ b/llvm/lib/Target/RISCV/RISCVSchedule.td
@@ -297,3 +297,4 @@ def : ReadAdvance<ReadAtomicHD, 0>;
 include "RISCVScheduleZb.td"
 include "RISCVScheduleV.td"
 include "RISCVScheduleXSf.td"
+include "RISCVScheduleZvk.td"
diff --git a/llvm/lib/Target/RISCV/RISCVScheduleZvk.td b/llvm/lib/Target/RISCV/RISCVScheduleZvk.td
new file mode 100644
index 00000000000000..3957bd34b875e7
--- /dev/null
+++ b/llvm/lib/Target/RISCV/RISCVScheduleZvk.td
@@ -0,0 +1,202 @@
+//===- RISCVScheduleB.td - RISC-V Scheduling Definitions B -*- tablegen -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+/// Define scheduler resources associated with def operands.
+
+/// Zvbb extension
+defm "" : LMULSchedWrites<"WriteVBREV">;
+defm "" : LMULSchedWrites<"WriteVCLZ">;
+defm "" : LMULSchedWrites<"WriteVCPOP">;
+defm "" : LMULSchedWrites<"WriteVCTZ">;
+defm "" : LMULSchedWrites<"WriteVWSLL">;
+
+/// Zvbc extension
+defm "" : LMULSchedWrites<"WriteVCLMUL">;
+defm "" : LMULSchedWrites<"WriteVCLMULH">;
+
+/// Zvkb extension
+defm "" : LMULSchedWrites<"WriteVANDN">;
+defm "" : LMULSchedWrites<"WriteVBREV8">;
+defm "" : LMULSchedWrites<"WriteVREV8">;
+defm "" : LMULSchedWrites<"WriteVROL">;
+defm "" : LMULSchedWrites<"WriteVROR">;
+
+/// Zvkg extension
+defm "" : LMULSchedWrites<"WriteVGHSH">;
+defm "" : LMULSchedWrites<"WriteVGMUL">;
+
+/// Zvknha or Zvknhb extensions
+defm "" : LMULSchedWrites<"WriteVSHA2CH">;
+defm "" : LMULSchedWrites<"WriteVSHA2CL">;
+defm "" : LMULSchedWrites<"WriteVSHA2MS">;
+
+/// Zvkned extension
+defm "" : LMULSchedWrites<"WriteVAESMV">;
+defm "" : LMULSchedWrites<"WriteVAESKF1">;
+defm "" : LMULSchedWrites<"WriteVAESKF2">;
+defm "" : LMULSchedWrites<"WriteVAESZ">;
+
+/// Zvksed extension
+defm "" : LMULSchedWrites<"WriteVSM4K">;
+defm "" : LMULSchedWrites<"WriteVSM4R">;
+
+/// Zvksh extension
+defm "" : LMULSchedWrites<"WriteVSM3C">;
+defm "" : LMULSchedWrites<"WriteVSM3ME">;
+
+/// Define scheduler resources associated with use operands.
+/// Zvbb extension
+defm "" : LMULSchedReads<"ReadVBREV">;
+defm "" : LMULSchedReads<"ReadVCLZ">;
+defm "" : LMULSchedReads<"ReadVCPOP">;
+defm "" : LMULSchedReads<"ReadVCTZ">;
+defm "" : LMULSchedReads<"ReadVWSLL">;
+
+/// Zvbc extension
+defm "" : LMULSchedReads<"ReadVCLMUL">;
+defm "" : LMULSchedReads<"ReadVCLMULH">;
+
+/// Zvkb extension
+defm "" : LMULSchedReads<"ReadVANDN">;
+defm "" : LMULSchedReads<"ReadVBREV8">;
+defm "" : LMULSchedReads<"ReadVREV8">;
+defm "" : LMULSchedReads<"ReadVROL">;
+defm "" : LMULSchedReads<"ReadVROR">;
+
+/// Zvkg extension
+defm "" : LMULSchedReads<"ReadVGHSH">;
+defm "" : LMULSchedReads<"R...
[truncated]

topperc
topperc reviewed Apr 25, 2024
View reviewed changes
llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
@@ -57,8 +55,7 @@ multiclass VROR_IV_V_X_I<string opcodestr, bits<6> funct6>
def I : RVInstIVI_VROR<funct6, (outs VR:$vd),
(ins VR:$vs2, uimm6:$imm, VMaskOp:$vm),
opcodestr # ".vi", "$vd, $vs2, $imm$vm">,
Sched<[WriteVIALUI_WorstCase, ReadVIALUV_WorstCase,
ReadVMask]>;
SchedUnaryMC<"WriteVROR", "ReadVROR">;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Established convention is a different Write based on the operand type. See WriteVIALUV, WriteVIALUX, and WriteVIALUI

llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
def X : VALUVX<funct6, OPMVX, opcodestr # "." # "vx">,
Sched<[WriteVIALUX_WorstCase, ReadVIALUV_WorstCase,
ReadVIALUX_WorstCase, ReadVMask]>;
SchedBinaryMC<"WriteVCLMULH", "ReadVCLMULH", "ReadVCLMULH">;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the entry for vclmul.vx and vclmulh.vx. Why using WriteVCLMULH?

llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
@@ -24,11 +24,9 @@ def tuimm5 : RISCVOp, TImmLeaf<XLenVT, [{return isUInt<5>(Imm);}]>;
let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in {
multiclass VCLMUL_MV_V_X<string opcodestr, bits<6> funct6> {
def V : VALUVV<funct6, OPMVV, opcodestr # "." # "vv">,
Sched<[WriteVIALUV_WorstCase, ReadVIALUV_WorstCase,
ReadVIALUV_WorstCase, ReadVMask]>;
SchedBinaryMC<"WriteVCLMUL", "ReadVCLMUL", "ReadVCLMUL">;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for vclmul.vv and vclmulh.vv.

llvm/lib/Target/RISCV/RISCVScheduleZvk.td Outdated
defm "" : LMULSchedWrites<"WriteVANDN">;
defm "" : LMULSchedWrites<"WriteVBREV8">;
defm "" : LMULSchedWrites<"WriteVREV8">;
defm "" : LMULSchedWrites<"WriteVROL">;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need separate VROL and VROR. But I do think we need RotateV, RotateX, and RotateI.

topperc
topperc reviewed Apr 25, 2024
View reviewed changes
llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
}
}

multiclass VPseudoVWALU_VV_VX_VI<Operand ImmType> : VPseudoVWALU_VV_VX {
foreach m = MxListW in {
defm "" : VPseudoBinaryW_VI<ImmType, m>,
SchedUnary<"WriteVIWALUV", "ReadVIWALUV", m.MX,
SchedUnary<"WriteVWSLLV", "ReadVWSLLV", m.MX,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be WriteVWSLLI. The non-VI instructions are getting a scheduler class from the inheriting of VPseudoVWALU_VV_VX. That seems odd.

topperc
topperc reviewed Apr 25, 2024
View reviewed changes
llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
multiclass VPseudoVANDN {
foreach m = MxList in {
defm "" : VPseudoBinaryV_VV<m>,
SchedBinary<"WriteVANDNV", "ReadVANDNV", "ReadVANDNV", m.MX,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use the existing VIALUV

topperc
topperc approved these changes Apr 25, 2024
View reviewed changes
Copy link
Collaborator

@topperc topperc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

wangpc-pp
wangpc-pp reviewed Apr 25, 2024
View reviewed changes
llvm/lib/Target/RISCV/RISCVScheduleZvk.td Outdated
@@ -0,0 +1,208 @@
//===- RISCVScheduleB.td - RISC-V Scheduling Definitions B -*- tablegen -*-===//
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy paste mistake here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

wangpc-pp
wangpc-pp reviewed Apr 26, 2024
View reviewed changes
llvm/lib/Target/RISCV/RISCVScheduleZvk.td Outdated
@@ -0,0 +1,208 @@
//===- RISCVScheduleB.td - RISC-V Scheduling Definitions Zvk -*- tablegen -===//
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file name is still incorrect.

wangpc-pp
wangpc-pp reviewed Apr 26, 2024
View reviewed changes
llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
multiclass VPseudoVANDN {
foreach m = MxList in {
defm "" : VPseudoBinaryV_VV<m>,
SchedBinary<"WriteVIALUV", "ReadVIALUV", "ReadVIALUV", m.MX,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indent here.

wangpc-pp
wangpc-pp reviewed Apr 26, 2024
View reviewed changes
llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
SchedBinary<"WriteVIALUV", "ReadVIALUV", "ReadVIALUV", m.MX,
forceMergeOpRead=true>;
defm "" : VPseudoBinaryV_VX<m>,
SchedBinary<"WriteVIALUX", "ReadVIALUV", "ReadVIALUX", m.MX,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto.

llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
multiclass VPseudoVROL {
foreach m = MxList in {
defm "" : VPseudoBinaryV_VV<m>,
SchedBinary<"WriteVRotV", "ReadVRotV", "ReadVRotV", m.MX,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
SchedBinary<"WriteVRotV", "ReadVRotV", "ReadVRotV", m.MX,
forceMergeOpRead=true>;
defm "" : VPseudoBinaryV_VX<m>,
SchedBinary<"WriteVRotX", "ReadVRotV", "ReadVRotX", m.MX,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

llvm/lib/Target/RISCV/RISCVInstrInfoZvk.td Outdated
foreach m = MxList in {
defvar mx = m.MX;
defm "" : VPseudoBinaryV_VV<m>,
SchedBinary<"WriteVRotV", "ReadVRotV", "ReadVRotV", mx,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

llvm/lib/Target/RISCV/RISCVScheduleZvk.td
defm "" : LMULSchedWrites<"WriteVSM3CV">;
defm "" : LMULSchedWrites<"WriteVSM3MEV">;

/// Define scheduler resources associated with use operands.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this line?

Copy link
Contributor Author

@michaelmaitland michaelmaitland Apr 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took the line from RISCVScheduleZb.td. At top of file we have line for def. This is line for use. Do you want me to remove both?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sorry, please just keep it.

wangpc-pp
wangpc-pp approved these changes Apr 26, 2024
View reviewed changes
Copy link
Contributor

@wangpc-pp wangpc-pp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

llvm/lib/Target/RISCV/RISCVScheduleZvk.td
defm "" : LMULSchedWrites<"WriteVSM3CV">;
defm "" : LMULSchedWrites<"WriteVSM3MEV">;

/// Define scheduler resources associated with use operands.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh sorry, please just keep it.

@michaelmaitland
[RISCV] Add Sched classes for vector crypto instructions
219fb86 
The vector crypto instructions may have different scheduling behavior
compared to VALU operations. Instead of using scheduling resources that
describe VALU operations, we give these instructions their own
scheduling resources. This is similar to what we did for Zb* instructions.

The sifive-p670 has vector crypto, so we model behavior for these instructions
in the P600SchedModel. The numbers are based off of measurements collected
internally. These numbers are a bit old and new measurments show that they may
not be fully accurate. It is likley that we will refine these numbers in a
follow up patch(s) based on new measurments.
@michaelmaitland
fixup! fix vandn and vwsll
b583311
@michaelmaitland
fixup! fix copy paste mistake
a8e3f22
@michaelmaitland
fixup! fix filename
179ea08
@michaelmaitland
fixup! indentation
0181004
@michaelmaitland michaelmaitland merged commit 56b8bd7 into llvm:main May 3, 2024
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@topperc topperc topperc approved these changes

@wangpc-pp wangpc-pp wangpc-pp approved these changes

@4vtomat 4vtomat Awaiting requested review from 4vtomat

Assignees
No one assigned
Labels
backend:RISC-V
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@michaelmaitland @llvmbot @topperc @wangpc-pp