DAG: Use sincos vector libcalls through RuntimeLibcalls

[arsenm]

Copy new process from sincospi.

[arsenm]

• PowerPC: Add vec_malloc functions to AIX in RuntimeLibcalls #167089
• XCore: Add iprintf to RuntimeLibcalls system library #167088
• RuntimeLibcalls: Add small_printf functions to emscripten #167087
• RuntimeLibcalls: Add macos unlocked IO functions to systems #167084
• RuntimeLibcalls: Add memset_pattern* calls to darwin systems #167083
• RuntimeLibcalls: Add more function entries from TargetLibraryInfo #167082
• RuntimeLibcalls: Add malloc and free entries #167081
• RuntimeLibcalls: Add mustprogress to common function attributes #167080
• RuntimeLibcalls: Add __memcpy_chk, __memmove_chk, __memset_chk #167053
• RuntimeLibcalls: Add a few libm entries from TargetLibraryInfo #167049
• RuntimeLibcalls: Add definitions for vector math functions #167026
• DAG: Move expandMultipleResultFPLibCall to TargetLowering (NFC) #166988
• DAG: Stop using TargetLibraryInfo for multi-result FP intrinsic codegen #166987
• DAG: Use modf vector libcalls through RuntimeLibcalls #166986
• RuntimeLibcalls: Add libcall entries for sleef and armpl modf functions #166985
• DAG: Use sincos vector libcalls through RuntimeLibcalls #166984 👈 (View in Graphite)
• RuntimeLibcalls: Add call entries for sincos sleef and armpl libcalls #166983
• RuntimeLibcalls: Remove incorrect sincospi from most targets #166982
• RuntimeLibcalls: Add entries for vector sincospi functions #166981
• TargetLibraryInfo: Split off VectorLibrary enum and flag #166980
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[llvmbot]

@llvm/pr-subscribers-llvm-selectiondag

Author: Matt Arsenault (arsenm)

Changes

Copy new process from sincospi.

---

Full diff: https://github.com/llvm/llvm-project/pull/166984.diff

3 Files Affected:

• (modified) llvm/include/llvm/CodeGen/BasicTTIImpl.h (+6-1)
• (modified) llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp (+4-10)
• (modified) llvm/lib/CodeGen/TargetLoweringBase.cpp (+18)

diff --git a/llvm/include/llvm/CodeGen/BasicTTIImpl.h b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
index 1c167af4b0478..a52ad41d0f1b3 100644
--- a/llvm/include/llvm/CodeGen/BasicTTIImpl.h
+++ b/llvm/include/llvm/CodeGen/BasicTTIImpl.h
@@ -334,7 +334,12 @@ class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
 
       break;
     case Intrinsic::sincos:
-      LC = RTLIB::getSINCOS(ScalarVT);
+      LC = RTLIB::getSINCOS(VT);
+      if (LC == RTLIB::UNKNOWN_LIBCALL)
+        LC = RTLIB::getSINCOS(ScalarVT);
+      else if (VT.isVector())
+        IsVectorCall = true;
+
       break;
     default:
       return std::nullopt;
diff --git a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
index f5a54497c8a98..78d8ea0676dd7 100644
--- a/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/LegalizeVectorOps.cpp
@@ -1268,10 +1268,12 @@ void VectorLegalizer::Expand(SDNode *Node, SmallVectorImpl<SDValue> &Results) {
       return;
 
     break;
-
+  case ISD::FSINCOS:
   case ISD::FSINCOSPI: {
     EVT VT = Node->getValueType(0);
-    RTLIB::Libcall LC = RTLIB::getSINCOSPI(VT);
+    RTLIB::Libcall LC = Node->getOpcode() == ISD::FSINCOS
+                            ? RTLIB::getSINCOS(VT)
+                            : RTLIB::getSINCOSPI(VT);
     if (LC != RTLIB::UNKNOWN_LIBCALL &&
         DAG.expandMultipleResultFPLibCall(LC, Node, Results, VT))
       return;
@@ -1280,14 +1282,6 @@ void VectorLegalizer::Expand(SDNode *Node, SmallVectorImpl<SDValue> &Results) {
     // scalarizing.
     break;
   }
-  case ISD::FSINCOS: {
-    // FIXME: Try to directly match vector case like fsincospi
-    EVT VT = Node->getValueType(0).getVectorElementType();
-    RTLIB::Libcall LC = RTLIB::getSINCOS(VT);
-    if (DAG.expandMultipleResultFPLibCall(LC, Node, Results, VT))
-      return;
-    break;
-  }
   case ISD::FMODF: {
     EVT VT = Node->getValueType(0).getVectorElementType();
     RTLIB::Libcall LC = RTLIB::getMODF(VT);
diff --git a/llvm/lib/CodeGen/TargetLoweringBase.cpp b/llvm/lib/CodeGen/TargetLoweringBase.cpp
index 814b4b57a0b9b..b4eb6c357e10e 100644
--- a/llvm/lib/CodeGen/TargetLoweringBase.cpp
+++ b/llvm/lib/CodeGen/TargetLoweringBase.cpp
@@ -425,6 +425,24 @@ RTLIB::Libcall RTLIB::getCOS(EVT RetVT) {
 }
 
 RTLIB::Libcall RTLIB::getSINCOS(EVT RetVT) {
+  // TODO: Tablegen should generate this function
+  if (RetVT.isVector()) {
+    if (!RetVT.isSimple())
+      return RTLIB::UNKNOWN_LIBCALL;
+    switch (RetVT.getSimpleVT().SimpleTy) {
+    case MVT::v4f32:
+      return RTLIB::SINCOS_V4F32;
+    case MVT::v2f64:
+      return RTLIB::SINCOS_V2F64;
+    case MVT::nxv4f32:
+      return RTLIB::SINCOS_NXV4F32;
+    case MVT::nxv2f64:
+      return RTLIB::SINCOS_NXV2F64;
+    default:
+      return RTLIB::UNKNOWN_LIBCALL;
+    }
+  }
+
   return getFPLibCall(RetVT, SINCOS_F32, SINCOS_F64, SINCOS_F80, SINCOS_F128,
                       SINCOS_PPCF128);
 }

[Andarwinux]

Can this handle SINCOS_V8F64? AMDLIBM has this.

[arsenm]

It could but I'm only handling cases that are tested

[RKSimon]

@Andarwinux please raise an issue if you can find AMDLIBM methods llvm doesn't currently handle

[Andarwinux]

@Andarwinux please raise an issue if you can find AMDLIBM methods llvm doesn't currently handle

No problems. It looks like AMDLIBM is still handling by TLI for now. I thought it would also switch to RuntimeLibcalls soon.

But veclib does indeed have issues on x86, see #164642.

[RKSimon]

LGTM