[AArch64] Unify lowering logic for fixed-length vectors. #89393
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In preparation of decoupling codegen for SME from SVE, this patch does a bit of cleanup to unify the logic around calling 'addTypeForFixedLengthSVE'. We only want to call this function when: * We have access to both SVE and NEON, but we prefer to use SVE. * We have access to SVE, but there is no access to NEON. Inside 'addTypeForFixedLengthSVE', we normally use Custom lowering for all operations so they can be converted to/from scalable vector operations. However, there are some exceptions: * For 64/128bit vector loads/stores we prefer the AdvSIMD LDR/STR D/Q-reg instructions, since these are available in Streaming-SVE mode. * For some operations like gather/scatter, we can only use SVE if the full set of SVE instructions is available (as opposed to the streaming[-compatible] subset). Otherwise, these operations need to expand (scalarise)
|
@llvm/pr-subscribers-backend-aarch64 Author: Sander de Smalen (sdesmalen-arm) ChangesIn preparation of decoupling codegen for SME from SVE, this patch does a bit of cleanup to unify the logic around calling We only want to call this function when:
Inside 'addTypeForFixedLengthSVE', we normally use Custom lowering for all operations so they can be converted to/from scalable vector operations. However, there are some exceptions:
Patch is 32.72 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/89393.diff 16 Files Affected:
diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
index eee67a0f823c19..f3275d4d2b55fc 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -1603,39 +1603,19 @@ AArch64TargetLowering::AArch64TargetLowering(const TargetMachine &TM,
setOperationAction(ISD::VECREDUCE_SEQ_FADD, VT, Custom);
}
- if (!Subtarget->isNeonAvailable()) {
- setTruncStoreAction(MVT::v2f32, MVT::v2bf16, Custom);
- setTruncStoreAction(MVT::v4f32, MVT::v4bf16, Custom);
- setTruncStoreAction(MVT::v8f32, MVT::v8bf16, Custom);
- setTruncStoreAction(MVT::v2f64, MVT::v2bf16, Custom);
- setTruncStoreAction(MVT::v4f64, MVT::v4bf16, Custom);
- setTruncStoreAction(MVT::v2f32, MVT::v2f16, Custom);
- setTruncStoreAction(MVT::v4f32, MVT::v4f16, Custom);
- setTruncStoreAction(MVT::v8f32, MVT::v8f16, Custom);
- setTruncStoreAction(MVT::v1f64, MVT::v1f16, Custom);
- setTruncStoreAction(MVT::v2f64, MVT::v2f16, Custom);
- setTruncStoreAction(MVT::v4f64, MVT::v4f16, Custom);
- setTruncStoreAction(MVT::v1f64, MVT::v1f32, Custom);
- setTruncStoreAction(MVT::v2f64, MVT::v2f32, Custom);
- setTruncStoreAction(MVT::v4f64, MVT::v4f32, Custom);
- for (MVT VT : {MVT::v8i8, MVT::v16i8, MVT::v4i16, MVT::v8i16, MVT::v2i32,
- MVT::v4i32, MVT::v1i64, MVT::v2i64})
- addTypeForFixedLengthSVE(VT, /*StreamingSVE=*/ true);
-
- for (MVT VT :
- {MVT::v4f16, MVT::v8f16, MVT::v2f32, MVT::v4f32, MVT::v2f64})
- addTypeForFixedLengthSVE(VT, /*StreamingSVE=*/ true);
- }
-
// NOTE: Currently this has to happen after computeRegisterProperties rather
// than the preferred option of combining it with the addRegisterClass call.
if (Subtarget->useSVEForFixedLengthVectors()) {
- for (MVT VT : MVT::integer_fixedlen_vector_valuetypes())
- if (useSVEForFixedLengthVectorVT(VT))
- addTypeForFixedLengthSVE(VT, /*StreamingSVE=*/ false);
- for (MVT VT : MVT::fp_fixedlen_vector_valuetypes())
- if (useSVEForFixedLengthVectorVT(VT))
- addTypeForFixedLengthSVE(VT, /*StreamingSVE=*/ false);
+ for (MVT VT : MVT::integer_fixedlen_vector_valuetypes()) {
+ if (useSVEForFixedLengthVectorVT(
+ VT, /*OverrideNEON=*/!Subtarget->isNeonAvailable()))
+ addTypeForFixedLengthSVE(VT);
+ }
+ for (MVT VT : MVT::fp_fixedlen_vector_valuetypes()) {
+ if (useSVEForFixedLengthVectorVT(
+ VT, /*OverrideNEON=*/!Subtarget->isNeonAvailable()))
+ addTypeForFixedLengthSVE(VT);
+ }
// 64bit results can mean a bigger than NEON input.
for (auto VT : {MVT::v8i8, MVT::v4i16})
@@ -1869,8 +1849,7 @@ bool AArch64TargetLowering::shouldExpandCttzElements(EVT VT) const {
return !Subtarget->hasSVEorSME() || VT != MVT::nxv16i1;
}
-void AArch64TargetLowering::addTypeForFixedLengthSVE(MVT VT,
- bool StreamingSVE) {
+void AArch64TargetLowering::addTypeForFixedLengthSVE(MVT VT) {
assert(VT.isFixedLengthVector() && "Expected fixed length vector type!");
// By default everything must be expanded.
@@ -1889,13 +1868,17 @@ void AArch64TargetLowering::addTypeForFixedLengthSVE(MVT VT,
setCondCodeAction(ISD::SETONE, VT, Expand);
}
+ TargetLoweringBase::LegalizeAction Default =
+ VT == MVT::v1f64 ? Expand : Custom;
+
// Mark integer truncating stores/extending loads as having custom lowering
if (VT.isInteger()) {
MVT InnerVT = VT.changeVectorElementType(MVT::i8);
while (InnerVT != VT) {
- setTruncStoreAction(VT, InnerVT, Custom);
- setLoadExtAction(ISD::ZEXTLOAD, VT, InnerVT, Custom);
- setLoadExtAction(ISD::SEXTLOAD, VT, InnerVT, Custom);
+ setTruncStoreAction(VT, InnerVT, Default);
+ setLoadExtAction(ISD::ZEXTLOAD, VT, InnerVT, Default);
+ setLoadExtAction(ISD::SEXTLOAD, VT, InnerVT, Default);
+ setLoadExtAction(ISD::EXTLOAD, VT, InnerVT, Default);
InnerVT = InnerVT.changeVectorElementType(
MVT::getIntegerVT(2 * InnerVT.getScalarSizeInBits()));
}
@@ -1907,101 +1890,104 @@ void AArch64TargetLowering::addTypeForFixedLengthSVE(MVT VT,
MVT InnerVT = VT.changeVectorElementType(MVT::f16);
while (InnerVT != VT) {
setTruncStoreAction(VT, InnerVT, Custom);
- setLoadExtAction(ISD::EXTLOAD, VT, InnerVT, Custom);
+ setLoadExtAction(ISD::EXTLOAD, VT, InnerVT, Default);
InnerVT = InnerVT.changeVectorElementType(
MVT::getFloatingPointVT(2 * InnerVT.getScalarSizeInBits()));
}
}
+ bool PreferNEON = VT.is64BitVector() || VT.is128BitVector();
+ bool PreferSVE = !PreferNEON && Subtarget->isSVEAvailable();
+
// Lower fixed length vector operations to scalable equivalents.
- setOperationAction(ISD::ABS, VT, Custom);
- setOperationAction(ISD::ADD, VT, Custom);
- setOperationAction(ISD::AND, VT, Custom);
- setOperationAction(ISD::ANY_EXTEND, VT, Custom);
- setOperationAction(ISD::BITCAST, VT, StreamingSVE ? Legal : Custom);
- setOperationAction(ISD::BITREVERSE, VT, Custom);
- setOperationAction(ISD::BSWAP, VT, Custom);
- setOperationAction(ISD::BUILD_VECTOR, VT, Custom);
- setOperationAction(ISD::CONCAT_VECTORS, VT, Custom);
- setOperationAction(ISD::CTLZ, VT, Custom);
- setOperationAction(ISD::CTPOP, VT, Custom);
- setOperationAction(ISD::CTTZ, VT, Custom);
- setOperationAction(ISD::EXTRACT_SUBVECTOR, VT, Custom);
- setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT, Custom);
- setOperationAction(ISD::FABS, VT, Custom);
- setOperationAction(ISD::FADD, VT, Custom);
- setOperationAction(ISD::FCEIL, VT, Custom);
- setOperationAction(ISD::FCOPYSIGN, VT, Custom);
- setOperationAction(ISD::FDIV, VT, Custom);
- setOperationAction(ISD::FFLOOR, VT, Custom);
- setOperationAction(ISD::FMA, VT, Custom);
- setOperationAction(ISD::FMAXIMUM, VT, Custom);
- setOperationAction(ISD::FMAXNUM, VT, Custom);
- setOperationAction(ISD::FMINIMUM, VT, Custom);
- setOperationAction(ISD::FMINNUM, VT, Custom);
- setOperationAction(ISD::FMUL, VT, Custom);
- setOperationAction(ISD::FNEARBYINT, VT, Custom);
- setOperationAction(ISD::FNEG, VT, Custom);
- setOperationAction(ISD::FP_EXTEND, VT, Custom);
- setOperationAction(ISD::FP_ROUND, VT, Custom);
- setOperationAction(ISD::FP_TO_SINT, VT, Custom);
- setOperationAction(ISD::FP_TO_UINT, VT, Custom);
- setOperationAction(ISD::FRINT, VT, Custom);
- setOperationAction(ISD::FROUND, VT, Custom);
- setOperationAction(ISD::FROUNDEVEN, VT, Custom);
- setOperationAction(ISD::FSQRT, VT, Custom);
- setOperationAction(ISD::FSUB, VT, Custom);
- setOperationAction(ISD::FTRUNC, VT, Custom);
- setOperationAction(ISD::INSERT_VECTOR_ELT, VT, Custom);
- setOperationAction(ISD::LOAD, VT, StreamingSVE ? Legal : Custom);
- setOperationAction(ISD::MGATHER, VT, StreamingSVE ? Expand : Custom);
- setOperationAction(ISD::MLOAD, VT, Custom);
- setOperationAction(ISD::MSCATTER, VT, StreamingSVE ? Expand : Custom);
- setOperationAction(ISD::MSTORE, VT, Custom);
- setOperationAction(ISD::MUL, VT, Custom);
- setOperationAction(ISD::MULHS, VT, Custom);
- setOperationAction(ISD::MULHU, VT, Custom);
- setOperationAction(ISD::OR, VT, Custom);
- setOperationAction(ISD::SCALAR_TO_VECTOR, VT, StreamingSVE ? Legal : Expand);
- setOperationAction(ISD::SDIV, VT, Custom);
- setOperationAction(ISD::SELECT, VT, Custom);
- setOperationAction(ISD::SETCC, VT, Custom);
- setOperationAction(ISD::SHL, VT, Custom);
- setOperationAction(ISD::SIGN_EXTEND, VT, Custom);
- setOperationAction(ISD::SIGN_EXTEND_INREG, VT, Custom);
- setOperationAction(ISD::SINT_TO_FP, VT, Custom);
- setOperationAction(ISD::SMAX, VT, Custom);
- setOperationAction(ISD::SMIN, VT, Custom);
- setOperationAction(ISD::SPLAT_VECTOR, VT, Custom);
- setOperationAction(ISD::SRA, VT, Custom);
- setOperationAction(ISD::SRL, VT, Custom);
- setOperationAction(ISD::STORE, VT, StreamingSVE ? Legal : Custom);
- setOperationAction(ISD::SUB, VT, Custom);
- setOperationAction(ISD::TRUNCATE, VT, Custom);
- setOperationAction(ISD::UDIV, VT, Custom);
- setOperationAction(ISD::UINT_TO_FP, VT, Custom);
- setOperationAction(ISD::UMAX, VT, Custom);
- setOperationAction(ISD::UMIN, VT, Custom);
- setOperationAction(ISD::VECREDUCE_ADD, VT, Custom);
- setOperationAction(ISD::VECREDUCE_AND, VT, Custom);
- setOperationAction(ISD::VECREDUCE_FADD, VT, Custom);
- setOperationAction(ISD::VECREDUCE_FMAX, VT, Custom);
- setOperationAction(ISD::VECREDUCE_FMIN, VT, Custom);
- setOperationAction(ISD::VECREDUCE_FMAXIMUM, VT, Custom);
- setOperationAction(ISD::VECREDUCE_FMINIMUM, VT, Custom);
- setOperationAction(ISD::VECREDUCE_OR, VT, Custom);
- setOperationAction(ISD::VECREDUCE_SEQ_FADD, VT,
- StreamingSVE ? Expand : Custom);
- setOperationAction(ISD::VECREDUCE_SMAX, VT, Custom);
- setOperationAction(ISD::VECREDUCE_SMIN, VT, Custom);
- setOperationAction(ISD::VECREDUCE_UMAX, VT, Custom);
- setOperationAction(ISD::VECREDUCE_UMIN, VT, Custom);
- setOperationAction(ISD::VECREDUCE_XOR, VT, Custom);
- setOperationAction(ISD::VECTOR_SHUFFLE, VT, Custom);
- setOperationAction(ISD::VECTOR_SPLICE, VT, Custom);
- setOperationAction(ISD::VSELECT, VT, Custom);
- setOperationAction(ISD::XOR, VT, Custom);
- setOperationAction(ISD::ZERO_EXTEND, VT, Custom);
+ setOperationAction(ISD::ABS, VT, Default);
+ setOperationAction(ISD::ADD, VT, Default);
+ setOperationAction(ISD::AND, VT, Default);
+ setOperationAction(ISD::ANY_EXTEND, VT, Default);
+ setOperationAction(ISD::BITCAST, VT, PreferNEON ? Legal : Default);
+ setOperationAction(ISD::BITREVERSE, VT, Default);
+ setOperationAction(ISD::BSWAP, VT, Default);
+ setOperationAction(ISD::BUILD_VECTOR, VT, Default);
+ setOperationAction(ISD::CONCAT_VECTORS, VT, Default);
+ setOperationAction(ISD::CTLZ, VT, Default);
+ setOperationAction(ISD::CTPOP, VT, Default);
+ setOperationAction(ISD::CTTZ, VT, Default);
+ setOperationAction(ISD::EXTRACT_SUBVECTOR, VT, Default);
+ setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT, Default);
+ setOperationAction(ISD::FABS, VT, Default);
+ setOperationAction(ISD::FADD, VT, Default);
+ setOperationAction(ISD::FCEIL, VT, Default);
+ setOperationAction(ISD::FCOPYSIGN, VT, Default);
+ setOperationAction(ISD::FDIV, VT, Default);
+ setOperationAction(ISD::FFLOOR, VT, Default);
+ setOperationAction(ISD::FMA, VT, Default);
+ setOperationAction(ISD::FMAXIMUM, VT, Default);
+ setOperationAction(ISD::FMAXNUM, VT, Default);
+ setOperationAction(ISD::FMINIMUM, VT, Default);
+ setOperationAction(ISD::FMINNUM, VT, Default);
+ setOperationAction(ISD::FMUL, VT, Default);
+ setOperationAction(ISD::FNEARBYINT, VT, Default);
+ setOperationAction(ISD::FNEG, VT, Default);
+ setOperationAction(ISD::FP_EXTEND, VT, Default);
+ setOperationAction(ISD::FP_ROUND, VT, Default);
+ setOperationAction(ISD::FP_TO_SINT, VT, Default);
+ setOperationAction(ISD::FP_TO_UINT, VT, Default);
+ setOperationAction(ISD::FRINT, VT, Default);
+ setOperationAction(ISD::FROUND, VT, Default);
+ setOperationAction(ISD::FROUNDEVEN, VT, Default);
+ setOperationAction(ISD::FSQRT, VT, Default);
+ setOperationAction(ISD::FSUB, VT, Default);
+ setOperationAction(ISD::FTRUNC, VT, Default);
+ setOperationAction(ISD::INSERT_VECTOR_ELT, VT, Default);
+ setOperationAction(ISD::LOAD, VT, PreferNEON ? Legal : Default);
+ setOperationAction(ISD::MGATHER, VT, PreferSVE ? Default : Expand);
+ setOperationAction(ISD::MLOAD, VT, Default);
+ setOperationAction(ISD::MSCATTER, VT, PreferSVE ? Default : Expand);
+ setOperationAction(ISD::MSTORE, VT, Default);
+ setOperationAction(ISD::MUL, VT, Default);
+ setOperationAction(ISD::MULHS, VT, Default);
+ setOperationAction(ISD::MULHU, VT, Default);
+ setOperationAction(ISD::OR, VT, Default);
+ setOperationAction(ISD::SCALAR_TO_VECTOR, VT,
+ isTypeLegal(VT) ? Legal : Expand);
+ setOperationAction(ISD::SDIV, VT, Default);
+ setOperationAction(ISD::SELECT, VT, Default);
+ setOperationAction(ISD::SETCC, VT, Default);
+ setOperationAction(ISD::SHL, VT, Default);
+ setOperationAction(ISD::SIGN_EXTEND, VT, Default);
+ setOperationAction(ISD::SIGN_EXTEND_INREG, VT, Default);
+ setOperationAction(ISD::SINT_TO_FP, VT, Default);
+ setOperationAction(ISD::SMAX, VT, Default);
+ setOperationAction(ISD::SMIN, VT, Default);
+ setOperationAction(ISD::SPLAT_VECTOR, VT, Default);
+ setOperationAction(ISD::SRA, VT, Default);
+ setOperationAction(ISD::SRL, VT, Default);
+ setOperationAction(ISD::STORE, VT, PreferNEON ? Legal : Default);
+ setOperationAction(ISD::SUB, VT, Default);
+ setOperationAction(ISD::TRUNCATE, VT, Default);
+ setOperationAction(ISD::UDIV, VT, Default);
+ setOperationAction(ISD::UINT_TO_FP, VT, Default);
+ setOperationAction(ISD::UMAX, VT, Default);
+ setOperationAction(ISD::UMIN, VT, Default);
+ setOperationAction(ISD::VECREDUCE_ADD, VT, Default);
+ setOperationAction(ISD::VECREDUCE_AND, VT, Default);
+ setOperationAction(ISD::VECREDUCE_FADD, VT, Default);
+ setOperationAction(ISD::VECREDUCE_FMAX, VT, Default);
+ setOperationAction(ISD::VECREDUCE_FMIN, VT, Default);
+ setOperationAction(ISD::VECREDUCE_FMAXIMUM, VT, Default);
+ setOperationAction(ISD::VECREDUCE_FMINIMUM, VT, Default);
+ setOperationAction(ISD::VECREDUCE_OR, VT, Default);
+ setOperationAction(ISD::VECREDUCE_SEQ_FADD, VT, PreferSVE ? Default : Expand);
+ setOperationAction(ISD::VECREDUCE_SMAX, VT, Default);
+ setOperationAction(ISD::VECREDUCE_SMIN, VT, Default);
+ setOperationAction(ISD::VECREDUCE_UMAX, VT, Default);
+ setOperationAction(ISD::VECREDUCE_UMIN, VT, Default);
+ setOperationAction(ISD::VECREDUCE_XOR, VT, Default);
+ setOperationAction(ISD::VECTOR_SHUFFLE, VT, Default);
+ setOperationAction(ISD::VECTOR_SPLICE, VT, Default);
+ setOperationAction(ISD::VSELECT, VT, Default);
+ setOperationAction(ISD::XOR, VT, Default);
+ setOperationAction(ISD::ZERO_EXTEND, VT, Default);
}
void AArch64TargetLowering::addDRTypeForNEON(MVT VT) {
diff --git a/llvm/lib/Target/AArch64/AArch64ISelLowering.h b/llvm/lib/Target/AArch64/AArch64ISelLowering.h
index db6e8a00d2fb5e..400368a5e1303d 100644
--- a/llvm/lib/Target/AArch64/AArch64ISelLowering.h
+++ b/llvm/lib/Target/AArch64/AArch64ISelLowering.h
@@ -1013,7 +1013,7 @@ class AArch64TargetLowering : public TargetLowering {
bool isExtFreeImpl(const Instruction *Ext) const override;
void addTypeForNEON(MVT VT);
- void addTypeForFixedLengthSVE(MVT VT, bool StreamingSVE);
+ void addTypeForFixedLengthSVE(MVT VT);
void addDRTypeForNEON(MVT VT);
void addQRTypeForNEON(MVT VT);
diff --git a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-bitcast.ll b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-bitcast.ll
index dd72c2b2bd0109..e3cc74f766ee0e 100644
--- a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-bitcast.ll
+++ b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-bitcast.ll
@@ -60,11 +60,8 @@ define void @bitcast_v2i16(ptr %a, ptr %b) {
; CHECK: // %bb.0:
; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa_offset 16
-; CHECK-NEXT: ldrh w8, [x0, #2]
-; CHECK-NEXT: str w8, [sp, #4]
-; CHECK-NEXT: ldrh w8, [x0]
-; CHECK-NEXT: str w8, [sp]
-; CHECK-NEXT: ldr d0, [sp]
+; CHECK-NEXT: ptrue p0.s, vl2
+; CHECK-NEXT: ld1h { z0.s }, p0/z, [x0]
; CHECK-NEXT: mov z1.s, z0.s[1]
; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: strh w8, [sp, #8]
diff --git a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-extract-vector-elt.ll b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-extract-vector-elt.ll
index 1b9bb42c8582e1..a752e119b2fb2a 100644
--- a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-extract-vector-elt.ll
+++ b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-extract-vector-elt.ll
@@ -90,8 +90,6 @@ define float @extractelement_v8f32(ptr %a) {
define double @extractelement_v1f64(<1 x double> %op1) {
; CHECK-LABEL: extractelement_v1f64:
; CHECK: // %bb.0:
-; CHECK-NEXT: // kill: def $d0 killed $d0 def $z0
-; CHECK-NEXT: // kill: def $d0 killed $d0 killed $z0
; CHECK-NEXT: ret
%r = extractelement <1 x double> %op1, i64 0
ret double %r
diff --git a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-compares.ll b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-compares.ll
index aad078f035f7d6..624d1823abd187 100644
--- a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-compares.ll
+++ b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-compares.ll
@@ -127,11 +127,9 @@ define void @fcmp_oeq_v8f32(ptr %a, ptr %b, ptr %c) {
define <1 x i64> @fcmp_oeq_v1f64(<1 x double> %op1, <1 x double> %op2) {
; CHECK-LABEL: fcmp_oeq_v1f64:
; CHECK: // %bb.0:
-; CHECK-NEXT: ptrue p0.d, vl1
-; CHECK-NEXT: // kill: def $d1 killed $d1 def $z1
-; CHECK-NEXT: // kill: def $d0 killed $d0 def $z0
-; CHECK-NEXT: fcmeq p0.d, p0/z, z0.d, z1.d
-; CHECK-NEXT: mov z0.d, p0/z, #-1 // =0xffffffffffffffff
+; CHECK-NEXT: fcmp d0, d1
+; CHECK-NEXT: csetm x8, eq
+; CHECK-NEXT: mov z0.d, x8
; CHECK-NEXT: // kill: def $d0 killed $d0 killed $z0
; CHECK-NEXT: ret
%cmp = fcmp oeq <1 x double> %op1, %op2
diff --git a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-fma.ll b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-fma.ll
index b5df97f767c13b..c51cedb493fa27 100644
--- a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-fma.ll
+++ b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-fma.ll
@@ -112,9 +112,6 @@ define void @fma_v8f32(ptr %a, ptr %b, ptr %c) {
define <1 x double> @fma_v1f64(<1 x double> %op1, <1 x double> %op2, <1 x double> %op3) {
; CHECK-LABEL: fma_v1f64:
; CHECK: // %bb.0:
-; CHECK-NEXT: // kill: def $d0 killed $d0 def $z0
-; CHECK-NEXT: // kill: def $d2 killed $d2 def $z2
-; CHECK-NEXT: // kill: def $d1 killed $d1 def $z1
; CHECK-NEXT: fmadd d0, d0, d1, d2
; CHECK-NEXT: ret
%mul = fmul contract <1 x double> %op1, %op2
diff --git a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-minmax.ll b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-minmax.ll
index 07a67e26502909..4d583c4e06ccaf 100644
--- a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-minmax.ll
+++ b/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-minmax.ll
@@ -99,8 +99,6 @@ define void @fmaxnm_v8f32(ptr %a, ptr %b) {
define <1 x double> @fmaxnm_v1f64(<1 x double> %op1, <1 x double> %op2) {
; CHECK-LABEL: fmaxnm_v1f64:
; CHECK: // %bb.0:
-; CHECK-NEXT: // kill: def $d0 killed $d0 def $z0
-; CHECK-NEXT: // kill: def $d1 killed $d1 def $z1
; CHECK-NEXT: fmaxnm d0, d0, d1
; CHECK-NEXT: ret
%res = call <1 x double> @llvm.maxnum.v1f64(<1 x double> %op1, <1 x double> %op2)
@@ -233,8 +231,6 @@ define void @fminnm_v8f32(ptr %a, ptr %b) {
define <1 x double> @fminnm_v1f64(<1 x double> %op1, <1 x double> %op2) {
; CHECK-LABEL: fminnm_v1f64:
; CHECK: // %bb.0:
-; CHECK-NEXT: // kill: def $d0 killed $d0 def $z0
-; CHECK-NEXT: // kill: def $d1 killed $d1 def $z1
; CHECK-NEXT: fminnm d0, d0, d1
; CHECK-NEXT: ret
%res = call <1 x double> @llvm.minnum.v1f64(<1 x double> %op1, <1 x double> %op2)
@@ -367,8 +363,6 @@ define void @fmax_v8f32(ptr %a, ptr %b) {
define <1 x double> @fmax_v1f64(<1 x double> %op1, <1 x double> %op2) {
; CHECK-LABEL: fmax_v1f64:
; CHECK: // %bb.0:
-; CHECK-NEXT: // kill: def $d0 killed $d0 def $z0
-; CHECK-NEXT: // kill: def $d1 killed $d1 def $z1
; CHECK-NEXT: fmax d0, d0, d1
; CHECK-NEXT: ret
%res = call <1 x double> @llvm.maximum.v1f64(<1 x double> %op1, <1 x double> %op2)
@@ -501,8 +495,6 @@ define void @fmin_v8f32(ptr %a, ptr %b) {
define <1 x double> @fmin_v1f64(<1 x double> %op1, <1 x double> %op2) {
; CHECK-LABEL: fmin_v1f64:
; CHECK: // %bb.0:
-; CHECK-NEXT: // kill: def $d0 killed $d0 def $z0
-; CHECK-NEXT: // kill: def $d1 killed $d1 def $z1
; CHECK-NEXT: fmin d0, d0, d1
; CHECK-NEXT: ret
%res = call <1 x double> @llvm.minimum.v1f64(<1 x double> %op1, <1 x double> %op2)
diff --git a/llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-reduce.l...
[truncated]
|
aemerson
approved these changes
Apr 23, 2024
| setOperationAction(ISD::MULHU, VT, Default); | ||
| setOperationAction(ISD::OR, VT, Default); | ||
| setOperationAction(ISD::SCALAR_TO_VECTOR, VT, | ||
| isTypeLegal(VT) ? Legal : Expand); |
There was a problem hiding this comment.
This condition will always be true? but we won't have isel patterns for the larger than NEON vector types. Is there a reason not to use PreferNEON?
There was a problem hiding this comment.
Thanks for pointing out. Fixed now!
paulwalker-arm
approved these changes
Apr 24, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In preparation of decoupling codegen for SME from SVE, this patch does a bit of cleanup to unify the logic around calling
'addTypeForFixedLengthSVE'.
We only want to call this function when:
Inside 'addTypeForFixedLengthSVE', we normally use Custom lowering for all operations so they can be converted to/from scalable vector operations.
However, there are some exceptions:
For 64/128bit vector loads/stores we prefer the AdvSIMD LDR/STR D/Q-reg instructions, since these are available in Streaming-SVE mode.
For some operations like gather/scatter, we can only use SVE if the full set of SVE instructions is available (as opposed to the streaming[-compatible] subset). Otherwise, these operations need to expand (scalarise)