[CostModel][X86] Update logic vector reduction costs to match llvm-mca estimates - #214191
Open
RKSimon wants to merge 3 commits into
Open
[CostModel][X86] Update logic vector reduction costs to match llvm-mca estimates#214191RKSimon wants to merge 3 commits into
RKSimon wants to merge 3 commits into
Conversation
…a estimates Replaces the olds costs which were just instruction counts Still working on the bool reduction costs - these will be handled in a followup
llvmorg-github-actions
Bot
added
backend:X86
llvm:analysis
|
@llvm/pr-subscribers-llvm-transforms @llvm/pr-subscribers-llvm-analysis Author: Simon Pilgrim (RKSimon) ChangesReplaces the olds costs which were just instruction counts Still working on the bool reduction costs - these will be handled in a followup Patch is 180.57 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/214191.diff 16 Files Affected:
diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index a421a02fd15db..4f424517e8d08 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -5696,8 +5696,9 @@ X86TTIImpl::getArithmeticReductionCost(unsigned Opcode, VectorType *ValTy,
if (TTI::requiresOrderedReduction(FMF))
return BaseT::getArithmeticReductionCost(Opcode, ValTy, FMF, CostKind);
+ // We use llvm-mca across all supported CPUs to measure the logic cost stats.
// We use the Intel Architecture Code Analyzer(IACA) to measure the throughput
- // and make it as the cost.
+ // and make it as the cost. TODO: Update old IACA numbers to llvm-mca.
static const CostKindTblEntry SLMCostTbl[] = {
{ ISD::FADD, MVT::v2f64, {3, 3, 3, 3} },
@@ -5718,6 +5719,19 @@ X86TTIImpl::getArithmeticReductionCost(unsigned Opcode, VectorType *ValTy,
{ ISD::ADD, MVT::v4i8, {2, 2, 2, 2} },
{ ISD::ADD, MVT::v8i8, {2, 2, 2, 2} },
{ ISD::ADD, MVT::v16i8, {3, 3, 3, 3} },
+
+ { ISD::AND, MVT::v2i64, {2, 2, 3, 3} },
+ { ISD::AND, MVT::v4i32, {3, 4, 5, 5} },
+ { ISD::AND, MVT::v8i16, {4, 7, 8, 8} },
+ { ISD::AND, MVT::v16i8, {5,10,11,11} },
+ { ISD::OR, MVT::v2i64, {2, 2, 3, 3} },
+ { ISD::OR, MVT::v4i32, {3, 4, 5, 5} },
+ { ISD::OR, MVT::v8i16, {4, 7, 8, 8} },
+ { ISD::OR, MVT::v16i8, {5,10,11,11} },
+ { ISD::XOR, MVT::v2i64, {2, 2, 3, 3} },
+ { ISD::XOR, MVT::v4i32, {3, 4, 5, 5} },
+ { ISD::XOR, MVT::v8i16, {4, 7, 8, 8} },
+ { ISD::XOR, MVT::v16i8, {5,10,11,11} },
};
static const CostKindTblEntry AVX1CostTbl[] = {
@@ -5729,6 +5743,52 @@ X86TTIImpl::getArithmeticReductionCost(unsigned Opcode, VectorType *ValTy,
{ ISD::ADD, MVT::v8i32, {5, 5, 5, 5} },
{ ISD::ADD, MVT::v16i16, {5, 5, 5, 5} },
{ ISD::ADD, MVT::v32i8, {4, 4, 4, 4} },
+
+ { ISD::AND, MVT::v4i64, {3, 7, 5, 5} },
+ { ISD::AND, MVT::v8i32, {4, 9, 7, 7} },
+ { ISD::AND, MVT::v16i16, {5,11, 9, 9} },
+ { ISD::AND, MVT::v8i16, {4, 7, 7, 7} },
+ { ISD::AND, MVT::v32i8, {6,13,11,11} },
+ { ISD::AND, MVT::v16i8, {5,10, 9, 9} },
+ { ISD::OR, MVT::v4i64, {3, 7, 5, 5} },
+ { ISD::OR, MVT::v8i32, {4, 9, 7, 7} },
+ { ISD::OR, MVT::v16i16, {5,11, 9, 9} },
+ { ISD::OR, MVT::v8i16, {4, 7, 7, 7} },
+ { ISD::OR, MVT::v32i8, {6,13,11,11} },
+ { ISD::OR, MVT::v16i8, {5,10, 9, 9} },
+ { ISD::XOR, MVT::v4i64, {3, 7, 5, 5} },
+ { ISD::XOR, MVT::v8i32, {4, 9, 7, 7} },
+ { ISD::XOR, MVT::v16i16, {5,11, 9, 9} },
+ { ISD::XOR, MVT::v8i16, {4, 7, 7, 7} },
+ { ISD::XOR, MVT::v32i8, {6,13,11,11} },
+ { ISD::XOR, MVT::v16i8, {5,10, 9, 9} },
+ };
+
+ static const CostKindTblEntry AVX2CostTbl[] = {
+ { ISD::AND, MVT::v4i64, {2, 7, 5, 5} },
+ { ISD::AND, MVT::v2i64, {1, 2, 3, 3} },
+ { ISD::AND, MVT::v8i32, {3, 9, 7, 7} },
+ { ISD::AND, MVT::v4i32, {2, 4, 5, 5} },
+ { ISD::AND, MVT::v16i16, {3,11, 9, 9} },
+ { ISD::AND, MVT::v8i16, {2, 6, 7, 7} },
+ { ISD::AND, MVT::v32i8, {3,13,11,11} },
+ { ISD::AND, MVT::v16i8, {3, 8, 9, 9} },
+ { ISD::OR, MVT::v4i64, {2, 7, 5, 5} },
+ { ISD::OR, MVT::v2i64, {1, 2, 3, 3} },
+ { ISD::OR, MVT::v8i32, {3, 9, 7, 7} },
+ { ISD::OR, MVT::v4i32, {2, 4, 5, 5} },
+ { ISD::OR, MVT::v16i16, {3,11, 9, 9} },
+ { ISD::OR, MVT::v8i16, {2, 6, 7, 7} },
+ { ISD::OR, MVT::v32i8, {3,13,11,11} },
+ { ISD::OR, MVT::v16i8, {3, 8, 9, 9} },
+ { ISD::XOR, MVT::v4i64, {2, 7, 5, 5} },
+ { ISD::XOR, MVT::v2i64, {1, 2, 3, 3} },
+ { ISD::XOR, MVT::v8i32, {3, 9, 7, 7} },
+ { ISD::XOR, MVT::v4i32, {2, 4, 5, 5} },
+ { ISD::XOR, MVT::v16i16, {3,11, 9, 9} },
+ { ISD::XOR, MVT::v8i16, {2, 6, 7, 7} },
+ { ISD::XOR, MVT::v32i8, {3,13,11,11} },
+ { ISD::XOR, MVT::v16i8, {3, 8, 9, 9} },
};
static const CostKindTblEntry AVX512FCostTbl[] = {
@@ -5736,6 +5796,22 @@ X86TTIImpl::getArithmeticReductionCost(unsigned Opcode, VectorType *ValTy,
{ ISD::FADD, MVT::v16f32, {5, 5, 5, 5} },
{ ISD::ADD, MVT::v8i64, {4, 4, 4, 4} },
{ ISD::ADD, MVT::v16i32, {6, 6, 6, 6} },
+
+ { ISD::AND, MVT::v8i64, {3,10, 7, 7} },
+ { ISD::AND, MVT::v16i32, {4,12, 9, 9} },
+ { ISD::AND, MVT::v32i16, {4,14,11,11} },
+ { ISD::AND, MVT::v64i8, {4,16,13,13} },
+ { ISD::AND, MVT::v16i8, {2, 8, 9, 9} },
+ { ISD::OR, MVT::v8i64, {3,10, 7, 7} },
+ { ISD::OR, MVT::v16i32, {4,12, 9, 9} },
+ { ISD::OR, MVT::v32i16, {4,14,11,11} },
+ { ISD::OR, MVT::v64i8, {4,16,13,13} },
+ { ISD::OR, MVT::v16i8, {2, 8, 9, 9} },
+ { ISD::XOR, MVT::v8i64, {3,10, 7, 7} },
+ { ISD::XOR, MVT::v16i32, {4,12, 9, 9} },
+ { ISD::XOR, MVT::v32i16, {4,14,11,11} },
+ { ISD::XOR, MVT::v64i8, {4,16,13,13} },
+ { ISD::XOR, MVT::v16i8, {2, 8, 9, 9} },
};
static const CostKindTblEntry AVX512BWCostTbl[] = {
@@ -5767,6 +5843,11 @@ X86TTIImpl::getArithmeticReductionCost(unsigned Opcode, VectorType *ValTy,
if (auto KindCost = Entry->Cost[CostKind])
return *KindCost;
+ if (ST->hasAVX2())
+ if (const auto *Entry = CostTableLookup(AVX2CostTbl, ISD, MTy))
+ if (auto KindCost = Entry->Cost[CostKind])
+ return *KindCost;
+
if (ST->hasAVX())
if (const auto *Entry = CostTableLookup(AVX1CostTbl, ISD, MTy))
if (auto KindCost = Entry->Cost[CostKind])
@@ -5898,6 +5979,11 @@ X86TTIImpl::getArithmeticReductionCost(unsigned Opcode, VectorType *ValTy,
if (auto KindCost = Entry->Cost[CostKind])
return ArithmeticCost + *KindCost;
+ if (ST->hasAVX2())
+ if (const auto *Entry = CostTableLookup(AVX2CostTbl, ISD, MTy))
+ if (auto KindCost = Entry->Cost[CostKind])
+ return ArithmeticCost + *KindCost;
+
if (ST->hasAVX())
if (const auto *Entry = CostTableLookup(AVX1CostTbl, ISD, MTy))
if (auto KindCost = Entry->Cost[CostKind])
diff --git a/llvm/test/Analysis/CostModel/X86/reduce-and.ll b/llvm/test/Analysis/CostModel/X86/reduce-and.ll
index 21338a12381fc..72e10f07c0039 100644
--- a/llvm/test/Analysis/CostModel/X86/reduce-and.ll
+++ b/llvm/test/Analysis/CostModel/X86/reduce-and.ll
@@ -11,26 +11,34 @@
define i32 @reduce_i64(i32 %arg) {
; SSE-LABEL: 'reduce_i64'
; SSE-NEXT: Cost Model: Found costs of 0 for: %V1 = call i64 @llvm.vector.reduce.and.v1i64(<1 x i64> undef)
-; SSE-NEXT: Cost Model: Found costs of 3 for: %V2 = call i64 @llvm.vector.reduce.and.v2i64(<2 x i64> undef)
-; SSE-NEXT: Cost Model: Found costs of 4 for: %V4 = call i64 @llvm.vector.reduce.and.v4i64(<4 x i64> undef)
-; SSE-NEXT: Cost Model: Found costs of 6 for: %V8 = call i64 @llvm.vector.reduce.and.v8i64(<8 x i64> undef)
-; SSE-NEXT: Cost Model: Found costs of 10 for: %V16 = call i64 @llvm.vector.reduce.and.v16i64(<16 x i64> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:2 CodeSize:3 Lat:2 SizeLat:3 for: %V2 = call i64 @llvm.vector.reduce.and.v2i64(<2 x i64> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:3 CodeSize:4 Lat:3 SizeLat:4 for: %V4 = call i64 @llvm.vector.reduce.and.v4i64(<4 x i64> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:5 CodeSize:6 Lat:5 SizeLat:6 for: %V8 = call i64 @llvm.vector.reduce.and.v8i64(<8 x i64> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:9 CodeSize:10 Lat:9 SizeLat:10 for: %V16 = call i64 @llvm.vector.reduce.and.v16i64(<16 x i64> undef)
; SSE-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
;
-; AVX-LABEL: 'reduce_i64'
-; AVX-NEXT: Cost Model: Found costs of 0 for: %V1 = call i64 @llvm.vector.reduce.and.v1i64(<1 x i64> undef)
-; AVX-NEXT: Cost Model: Found costs of 3 for: %V2 = call i64 @llvm.vector.reduce.and.v2i64(<2 x i64> undef)
-; AVX-NEXT: Cost Model: Found costs of 5 for: %V4 = call i64 @llvm.vector.reduce.and.v4i64(<4 x i64> undef)
-; AVX-NEXT: Cost Model: Found costs of RThru:6 CodeSize:6 Lat:6 SizeLat:7 for: %V8 = call i64 @llvm.vector.reduce.and.v8i64(<8 x i64> undef)
-; AVX-NEXT: Cost Model: Found costs of RThru:8 CodeSize:8 Lat:8 SizeLat:11 for: %V16 = call i64 @llvm.vector.reduce.and.v16i64(<16 x i64> undef)
-; AVX-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
+; AVX1-LABEL: 'reduce_i64'
+; AVX1-NEXT: Cost Model: Found costs of 0 for: %V1 = call i64 @llvm.vector.reduce.and.v1i64(<1 x i64> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:2 CodeSize:3 Lat:2 SizeLat:3 for: %V2 = call i64 @llvm.vector.reduce.and.v2i64(<2 x i64> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:3 CodeSize:5 Lat:7 SizeLat:5 for: %V4 = call i64 @llvm.vector.reduce.and.v4i64(<4 x i64> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:4 CodeSize:6 Lat:8 SizeLat:7 for: %V8 = call i64 @llvm.vector.reduce.and.v8i64(<8 x i64> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:6 CodeSize:8 Lat:10 SizeLat:11 for: %V16 = call i64 @llvm.vector.reduce.and.v16i64(<16 x i64> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
+;
+; AVX2-LABEL: 'reduce_i64'
+; AVX2-NEXT: Cost Model: Found costs of 0 for: %V1 = call i64 @llvm.vector.reduce.and.v1i64(<1 x i64> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:1 CodeSize:3 Lat:2 SizeLat:3 for: %V2 = call i64 @llvm.vector.reduce.and.v2i64(<2 x i64> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:2 CodeSize:5 Lat:7 SizeLat:5 for: %V4 = call i64 @llvm.vector.reduce.and.v4i64(<4 x i64> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:3 CodeSize:6 Lat:8 SizeLat:7 for: %V8 = call i64 @llvm.vector.reduce.and.v8i64(<8 x i64> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:5 CodeSize:8 Lat:10 SizeLat:11 for: %V16 = call i64 @llvm.vector.reduce.and.v16i64(<16 x i64> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
;
; AVX512-LABEL: 'reduce_i64'
; AVX512-NEXT: Cost Model: Found costs of 0 for: %V1 = call i64 @llvm.vector.reduce.and.v1i64(<1 x i64> undef)
-; AVX512-NEXT: Cost Model: Found costs of RThru:3 CodeSize:3 Lat:5 SizeLat:3 for: %V2 = call i64 @llvm.vector.reduce.and.v2i64(<2 x i64> undef)
-; AVX512-NEXT: Cost Model: Found costs of RThru:5 CodeSize:5 Lat:7 SizeLat:5 for: %V4 = call i64 @llvm.vector.reduce.and.v4i64(<4 x i64> undef)
-; AVX512-NEXT: Cost Model: Found costs of RThru:7 CodeSize:7 Lat:9 SizeLat:7 for: %V8 = call i64 @llvm.vector.reduce.and.v8i64(<8 x i64> undef)
-; AVX512-NEXT: Cost Model: Found costs of RThru:8 CodeSize:8 Lat:10 SizeLat:8 for: %V16 = call i64 @llvm.vector.reduce.and.v16i64(<16 x i64> undef)
+; AVX512-NEXT: Cost Model: Found costs of RThru:1 CodeSize:3 Lat:2 SizeLat:3 for: %V2 = call i64 @llvm.vector.reduce.and.v2i64(<2 x i64> undef)
+; AVX512-NEXT: Cost Model: Found costs of RThru:2 CodeSize:5 Lat:7 SizeLat:5 for: %V4 = call i64 @llvm.vector.reduce.and.v4i64(<4 x i64> undef)
+; AVX512-NEXT: Cost Model: Found costs of RThru:3 CodeSize:7 Lat:10 SizeLat:7 for: %V8 = call i64 @llvm.vector.reduce.and.v8i64(<8 x i64> undef)
+; AVX512-NEXT: Cost Model: Found costs of RThru:4 CodeSize:8 Lat:11 SizeLat:8 for: %V16 = call i64 @llvm.vector.reduce.and.v16i64(<16 x i64> undef)
; AVX512-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
;
%V1 = call i64 @llvm.vector.reduce.and.v1i64(<1 x i64> undef)
@@ -43,27 +51,35 @@ define i32 @reduce_i64(i32 %arg) {
define i32 @reduce_i32(i32 %arg) {
; SSE-LABEL: 'reduce_i32'
-; SSE-NEXT: Cost Model: Found costs of 3 for: %V2 = call i32 @llvm.vector.reduce.and.v2i32(<2 x i32> undef)
-; SSE-NEXT: Cost Model: Found costs of 5 for: %V4 = call i32 @llvm.vector.reduce.and.v4i32(<4 x i32> undef)
-; SSE-NEXT: Cost Model: Found costs of 6 for: %V8 = call i32 @llvm.vector.reduce.and.v8i32(<8 x i32> undef)
-; SSE-NEXT: Cost Model: Found costs of 8 for: %V16 = call i32 @llvm.vector.reduce.and.v16i32(<16 x i32> undef)
-; SSE-NEXT: Cost Model: Found costs of 12 for: %V32 = call i32 @llvm.vector.reduce.and.v32i32(<32 x i32> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:3 CodeSize:5 Lat:4 SizeLat:5 for: %V2 = call i32 @llvm.vector.reduce.and.v2i32(<2 x i32> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:3 CodeSize:5 Lat:4 SizeLat:5 for: %V4 = call i32 @llvm.vector.reduce.and.v4i32(<4 x i32> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:4 CodeSize:6 Lat:5 SizeLat:6 for: %V8 = call i32 @llvm.vector.reduce.and.v8i32(<8 x i32> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:6 CodeSize:8 Lat:7 SizeLat:8 for: %V16 = call i32 @llvm.vector.reduce.and.v16i32(<16 x i32> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:10 CodeSize:12 Lat:11 SizeLat:12 for: %V32 = call i32 @llvm.vector.reduce.and.v32i32(<32 x i32> undef)
; SSE-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
;
-; AVX-LABEL: 'reduce_i32'
-; AVX-NEXT: Cost Model: Found costs of 3 for: %V2 = call i32 @llvm.vector.reduce.and.v2i32(<2 x i32> undef)
-; AVX-NEXT: Cost Model: Found costs of 5 for: %V4 = call i32 @llvm.vector.reduce.and.v4i32(<4 x i32> undef)
-; AVX-NEXT: Cost Model: Found costs of 7 for: %V8 = call i32 @llvm.vector.reduce.and.v8i32(<8 x i32> undef)
-; AVX-NEXT: Cost Model: Found costs of RThru:8 CodeSize:8 Lat:8 SizeLat:9 for: %V16 = call i32 @llvm.vector.reduce.and.v16i32(<16 x i32> undef)
-; AVX-NEXT: Cost Model: Found costs of RThru:10 CodeSize:10 Lat:10 SizeLat:13 for: %V32 = call i32 @llvm.vector.reduce.and.v32i32(<32 x i32> undef)
-; AVX-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
+; AVX1-LABEL: 'reduce_i32'
+; AVX1-NEXT: Cost Model: Found costs of RThru:3 CodeSize:5 Lat:4 SizeLat:5 for: %V2 = call i32 @llvm.vector.reduce.and.v2i32(<2 x i32> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:3 CodeSize:5 Lat:4 SizeLat:5 for: %V4 = call i32 @llvm.vector.reduce.and.v4i32(<4 x i32> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:4 CodeSize:7 Lat:9 SizeLat:7 for: %V8 = call i32 @llvm.vector.reduce.and.v8i32(<8 x i32> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:5 CodeSize:8 Lat:10 SizeLat:9 for: %V16 = call i32 @llvm.vector.reduce.and.v16i32(<16 x i32> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:7 CodeSize:10 Lat:12 SizeLat:13 for: %V32 = call i32 @llvm.vector.reduce.and.v32i32(<32 x i32> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
+;
+; AVX2-LABEL: 'reduce_i32'
+; AVX2-NEXT: Cost Model: Found costs of RThru:2 CodeSize:5 Lat:4 SizeLat:5 for: %V2 = call i32 @llvm.vector.reduce.and.v2i32(<2 x i32> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:2 CodeSize:5 Lat:4 SizeLat:5 for: %V4 = call i32 @llvm.vector.reduce.and.v4i32(<4 x i32> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:3 CodeSize:7 Lat:9 SizeLat:7 for: %V8 = call i32 @llvm.vector.reduce.and.v8i32(<8 x i32> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:4 CodeSize:8 Lat:10 SizeLat:9 for: %V16 = call i32 @llvm.vector.reduce.and.v16i32(<16 x i32> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:6 CodeSize:10 Lat:12 SizeLat:13 for: %V32 = call i32 @llvm.vector.reduce.and.v32i32(<32 x i32> undef)
+; AVX2-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
;
; AVX512-LABEL: 'reduce_i32'
-; AVX512-NEXT: Cost Model: Found costs of RThru:3 CodeSize:3 Lat:5 SizeLat:3 for: %V2 = call i32 @llvm.vector.reduce.and.v2i32(<2 x i32> undef)
-; AVX512-NEXT: Cost Model: Found costs of RThru:5 CodeSize:5 Lat:9 SizeLat:5 for: %V4 = call i32 @llvm.vector.reduce.and.v4i32(<4 x i32> undef)
-; AVX512-NEXT: Cost Model: Found costs of RThru:7 CodeSize:7 Lat:11 SizeLat:7 for: %V8 = call i32 @llvm.vector.reduce.and.v8i32(<8 x i32> undef)
-; AVX512-NEXT: Cost Model: Found costs of RThru:9 CodeSize:9 Lat:13 SizeLat:9 for: %V16 = call i32 @llvm.vector.reduce.and.v16i32(<16 x i32> undef)
-; AVX512-NEXT: Cost Model: Found costs of RThru:10 CodeSize:10 Lat:14 SizeLat:10 for: %V32 = call i32 @llvm.vector.reduce.and.v32i32(<32 x i32> undef)
+; AVX512-NEXT: Cost Model: Found costs of RThru:2 CodeSize:5 Lat:4 SizeLat:5 for: %V2 = call i32 @llvm.vector.reduce.and.v2i32(<2 x i32> undef)
+; AVX512-NEXT: Cost Model: Found costs of RThru:2 CodeSize:5 Lat:4 SizeLat:5 for: %V4 = call i32 @llvm.vector.reduce.and.v4i32(<4 x i32> undef)
+; AVX512-NEXT: Cost Model: Found costs of RThru:3 CodeSize:7 Lat:9 SizeLat:7 for: %V8 = call i32 @llvm.vector.reduce.and.v8i32(<8 x i32> undef)
+; AVX512-NEXT: Cost Model: Found costs of RThru:4 CodeSize:9 Lat:12 SizeLat:9 for: %V16 = call i32 @llvm.vector.reduce.and.v16i32(<16 x i32> undef)
+; AVX512-NEXT: Cost Model: Found costs of RThru:5 CodeSize:10 Lat:13 SizeLat:10 for: %V32 = call i32 @llvm.vector.reduce.and.v32i32(<32 x i32> undef)
; AVX512-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
;
%V2 = call i32 @llvm.vector.reduce.and.v2i32(<2 x i32> undef)
@@ -76,39 +92,39 @@ define i32 @reduce_i32(i32 %arg) {
define i32 @reduce_i16(i32 %arg) {
; SSE-LABEL: 'reduce_i16'
-; SSE-NEXT: Cost Model: Found costs of 3 for: %V2 = call i16 @llvm.vector.reduce.and.v2i16(<2 x i16> undef)
-; SSE-NEXT: Cost Model: Found costs of 5 for: %V4 = call i16 @llvm.vector.reduce.and.v4i16(<4 x i16> undef)
-; SSE-NEXT: Cost Model: Found costs of 7 for: %V8 = call i16 @llvm.vector.reduce.and.v8i16(<8 x i16> undef)
-; SSE-NEXT: Cost Model: Found costs of 8 for: %V16 = call i16 @llvm.vector.reduce.and.v16i16(<16 x i16> undef)
-; SSE-NEXT: Cost Model: Found costs of 10 for: %V32 = call i16 @llvm.vector.reduce.and.v32i16(<32 x i16> undef)
-; SSE-NEXT: Cost Model: Found costs of 14 for: %V64 = call i16 @llvm.vector.reduce.and.v64i16(<64 x i16> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:4 CodeSize:8 Lat:7 SizeLat:8 for: %V2 = call i16 @llvm.vector.reduce.and.v2i16(<2 x i16> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:4 CodeSize:8 Lat:7 SizeLat:8 for: %V4 = call i16 @llvm.vector.reduce.and.v4i16(<4 x i16> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:4 CodeSize:8 Lat:7 SizeLat:8 for: %V8 = call i16 @llvm.vector.reduce.and.v8i16(<8 x i16> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:5 CodeSize:9 Lat:8 SizeLat:9 for: %V16 = call i16 @llvm.vector.reduce.and.v16i16(<16 x i16> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:7 CodeSize:11 Lat:10 SizeLat:11 for: %V32 = call i16 @llvm.vector.reduce.and.v32i16(<32 x i16> undef)
+; SSE-NEXT: Cost Model: Found costs of RThru:11 CodeSize:15 Lat:14 SizeLat:15 for: %V64 = call i16 @llvm.vector.reduce.and.v64i16(<64 x i16> undef)
; SSE-NEXT: Cost Model: Found costs of RThru:0 CodeSize:1 Lat:1 SizeLat:1 for: ret i32 undef
;
; AVX1-LABEL: 'reduce_i16'
-; AVX1-NEXT: Cost Model: Found costs of RThru:3 CodeSize:3 Lat:4 SizeLat:3 for: %V2 = call i16 @llvm.vector.reduce.and.v2i16(<2 x i16> undef)
-; AVX1-NEXT: Cost Model: Found costs of RThru:5 CodeSize:5 Lat:6 SizeLat:5 for: %V4 = call i16 @llvm.vector.reduce.and.v4i16(<4 x i16> undef)
-; AVX1-NEXT: Cost Model: Found costs of RThru:7 CodeSize:7 Lat:8 SizeLat:7 for: %V8 = call i16 @llvm.vector.reduce.and.v8i16(<8 x i16> undef)
-; AVX1-NEXT: Cost Model: Found costs of RThru:9 CodeSize:9 Lat:10 SizeLat:9 for: %V16 = call i16 @llvm.vector.reduce.and.v16i16(<16 x i16> undef)
-; AVX1-NEXT: Cost Model: Found costs of RThru:10 CodeSize:10 Lat:11 SizeLat:11 for: %V32 = call i16 @llvm.vector.reduce.and.v32i16(<32 x i16> undef)
-; AVX1-NEXT: Cost Model: Found costs of RThru:12 CodeSize:12 Lat:13 SizeLat:15 for: %V64 = call i16 @llvm.vector.reduce.and.v64i16(<64 x i16> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:4 CodeSize:7 Lat:7 SizeLat:7 for: %V2 = call i16 @llvm.vector.reduce.and.v2i16(<2 x i16> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:4 CodeSize:7 Lat:7 SizeLat:7 for: %V4 = call i16 @llvm.vector.reduce.and.v4i16(<4 x i16> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:4 CodeSize:7 Lat:7 SizeLat:7 for: %V8 = call i16 @llvm.vector.reduce.and.v8i16(<8 x i16> undef)
+; AVX1-NEXT: Cost Model: Found costs of RThru:5 CodeSize:9 L...
[truncated]
|
You can test this locally with the following command:git-clang-format --diff origin/main HEAD --extensions cpp -- llvm/lib/Target/X86/X86TargetTransformInfo.cpp --diff_from_common_commit
View the diff from clang-format here.diff --git a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
index 828d79ab0..079e48686 100644
--- a/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
+++ b/llvm/lib/Target/X86/X86TargetTransformInfo.cpp
@@ -5721,112 +5721,126 @@ X86TTIImpl::getArithmeticReductionCost(unsigned Opcode, VectorType *ValTy,
};
static const CostKindTblEntry SSE2CostTbl[] = {
- { ISD::FADD, MVT::v2f64, {2, 2, 2, 2} },
- { ISD::FADD, MVT::v2f32, {2, 2, 2, 2} },
- { ISD::FADD, MVT::v4f32, {4, 4, 4, 4} },
- { ISD::ADD, MVT::v2i64, {2, 2, 2, 2} }, // The data reported by the IACA tool is "1.6".
- { ISD::ADD, MVT::v2i32, {2, 2, 2, 2} }, // FIXME: chosen to be less than v4i32
- { ISD::ADD, MVT::v4i32, {3, 3, 3, 3} }, // The data reported by the IACA tool is "3.3".
- { ISD::ADD, MVT::v2i16, {2, 2, 2, 2} }, // The data reported by the IACA tool is "4.3".
- { ISD::ADD, MVT::v4i16, {3, 3, 3, 3} }, // The data reported by the IACA tool is "4.3".
- { ISD::ADD, MVT::v8i16, {4, 4, 4, 4} }, // The data reported by the IACA tool is "4.3".
- { ISD::ADD, MVT::v2i8, {2, 2, 2, 2} },
- { ISD::ADD, MVT::v4i8, {2, 2, 2, 2} },
- { ISD::ADD, MVT::v8i8, {2, 2, 2, 2} },
- { ISD::ADD, MVT::v16i8, {3, 3, 3, 3} },
-
- { ISD::AND, MVT::v2i64, {2, 2, 3, 3} },
- { ISD::AND, MVT::v4i32, {3, 4, 5, 5} },
- { ISD::AND, MVT::v8i16, {4, 7, 8, 8} },
- { ISD::AND, MVT::v16i8, {6,10,11,11} },
- { ISD::OR, MVT::v2i64, {2, 2, 3, 3} },
- { ISD::OR, MVT::v4i32, {3, 4, 5, 5} },
- { ISD::OR, MVT::v8i16, {4, 7, 8, 8} },
- { ISD::OR, MVT::v16i8, {6,10,11,11} },
- { ISD::XOR, MVT::v2i64, {2, 2, 3, 3} },
- { ISD::XOR, MVT::v4i32, {3, 4, 5, 5} },
- { ISD::XOR, MVT::v8i16, {4, 7, 8, 8} },
- { ISD::XOR, MVT::v16i8, {6,10,11,11} },
+ {ISD::FADD, MVT::v2f64, {2, 2, 2, 2}},
+ {ISD::FADD, MVT::v2f32, {2, 2, 2, 2}},
+ {ISD::FADD, MVT::v4f32, {4, 4, 4, 4}},
+ {ISD::ADD,
+ MVT::v2i64,
+ {2, 2, 2, 2}}, // The data reported by the IACA tool is "1.6".
+ {ISD::ADD,
+ MVT::v2i32,
+ {2, 2, 2, 2}}, // FIXME: chosen to be less than v4i32
+ {ISD::ADD,
+ MVT::v4i32,
+ {3, 3, 3, 3}}, // The data reported by the IACA tool is "3.3".
+ {ISD::ADD,
+ MVT::v2i16,
+ {2, 2, 2, 2}}, // The data reported by the IACA tool is "4.3".
+ {ISD::ADD,
+ MVT::v4i16,
+ {3, 3, 3, 3}}, // The data reported by the IACA tool is "4.3".
+ {ISD::ADD,
+ MVT::v8i16,
+ {4, 4, 4, 4}}, // The data reported by the IACA tool is "4.3".
+ {ISD::ADD, MVT::v2i8, {2, 2, 2, 2}},
+ {ISD::ADD, MVT::v4i8, {2, 2, 2, 2}},
+ {ISD::ADD, MVT::v8i8, {2, 2, 2, 2}},
+ {ISD::ADD, MVT::v16i8, {3, 3, 3, 3}},
+
+ {ISD::AND, MVT::v2i64, {2, 2, 3, 3}},
+ {ISD::AND, MVT::v4i32, {3, 4, 5, 5}},
+ {ISD::AND, MVT::v8i16, {4, 7, 8, 8}},
+ {ISD::AND, MVT::v16i8, {6, 10, 11, 11}},
+ {ISD::OR, MVT::v2i64, {2, 2, 3, 3}},
+ {ISD::OR, MVT::v4i32, {3, 4, 5, 5}},
+ {ISD::OR, MVT::v8i16, {4, 7, 8, 8}},
+ {ISD::OR, MVT::v16i8, {6, 10, 11, 11}},
+ {ISD::XOR, MVT::v2i64, {2, 2, 3, 3}},
+ {ISD::XOR, MVT::v4i32, {3, 4, 5, 5}},
+ {ISD::XOR, MVT::v8i16, {4, 7, 8, 8}},
+ {ISD::XOR, MVT::v16i8, {6, 10, 11, 11}},
};
static const CostKindTblEntry AVX1CostTbl[] = {
- { ISD::FADD, MVT::v4f64, {3, 3, 3, 3} },
- { ISD::FADD, MVT::v4f32, {3, 3, 3, 3} },
- { ISD::FADD, MVT::v8f32, {4, 4, 4, 4} },
- { ISD::ADD, MVT::v2i64, {1, 1, 1, 1} }, // The data reported by the IACA tool is "1.5".
- { ISD::ADD, MVT::v4i64, {3, 3, 3, 3} },
- { ISD::ADD, MVT::v8i32, {5, 5, 5, 5} },
- { ISD::ADD, MVT::v16i16, {5, 5, 5, 5} },
- { ISD::ADD, MVT::v32i8, {4, 4, 4, 4} },
-
- { ISD::AND, MVT::v4i64, {3, 7, 5, 5} },
- { ISD::AND, MVT::v8i32, {4, 9, 7, 7} },
- { ISD::AND, MVT::v16i16, {5,11, 9, 9} },
- { ISD::AND, MVT::v8i16, {4, 7, 7, 7} },
- { ISD::AND, MVT::v32i8, {6,13,11,11} },
- { ISD::AND, MVT::v16i8, {5,10, 9, 9} },
- { ISD::OR, MVT::v4i64, {3, 7, 5, 5} },
- { ISD::OR, MVT::v8i32, {4, 9, 7, 7} },
- { ISD::OR, MVT::v16i16, {5,11, 9, 9} },
- { ISD::OR, MVT::v8i16, {4, 7, 7, 7} },
- { ISD::OR, MVT::v32i8, {6,13,11,11} },
- { ISD::OR, MVT::v16i8, {5,10, 9, 9} },
- { ISD::XOR, MVT::v4i64, {3, 7, 5, 5} },
- { ISD::XOR, MVT::v8i32, {4, 9, 7, 7} },
- { ISD::XOR, MVT::v16i16, {5,11, 9, 9} },
- { ISD::XOR, MVT::v8i16, {4, 7, 7, 7} },
- { ISD::XOR, MVT::v32i8, {6,13,11,11} },
- { ISD::XOR, MVT::v16i8, {5,10, 9, 9} },
+ {ISD::FADD, MVT::v4f64, {3, 3, 3, 3}},
+ {ISD::FADD, MVT::v4f32, {3, 3, 3, 3}},
+ {ISD::FADD, MVT::v8f32, {4, 4, 4, 4}},
+ {ISD::ADD,
+ MVT::v2i64,
+ {1, 1, 1, 1}}, // The data reported by the IACA tool is "1.5".
+ {ISD::ADD, MVT::v4i64, {3, 3, 3, 3}},
+ {ISD::ADD, MVT::v8i32, {5, 5, 5, 5}},
+ {ISD::ADD, MVT::v16i16, {5, 5, 5, 5}},
+ {ISD::ADD, MVT::v32i8, {4, 4, 4, 4}},
+
+ {ISD::AND, MVT::v4i64, {3, 7, 5, 5}},
+ {ISD::AND, MVT::v8i32, {4, 9, 7, 7}},
+ {ISD::AND, MVT::v16i16, {5, 11, 9, 9}},
+ {ISD::AND, MVT::v8i16, {4, 7, 7, 7}},
+ {ISD::AND, MVT::v32i8, {6, 13, 11, 11}},
+ {ISD::AND, MVT::v16i8, {5, 10, 9, 9}},
+ {ISD::OR, MVT::v4i64, {3, 7, 5, 5}},
+ {ISD::OR, MVT::v8i32, {4, 9, 7, 7}},
+ {ISD::OR, MVT::v16i16, {5, 11, 9, 9}},
+ {ISD::OR, MVT::v8i16, {4, 7, 7, 7}},
+ {ISD::OR, MVT::v32i8, {6, 13, 11, 11}},
+ {ISD::OR, MVT::v16i8, {5, 10, 9, 9}},
+ {ISD::XOR, MVT::v4i64, {3, 7, 5, 5}},
+ {ISD::XOR, MVT::v8i32, {4, 9, 7, 7}},
+ {ISD::XOR, MVT::v16i16, {5, 11, 9, 9}},
+ {ISD::XOR, MVT::v8i16, {4, 7, 7, 7}},
+ {ISD::XOR, MVT::v32i8, {6, 13, 11, 11}},
+ {ISD::XOR, MVT::v16i8, {5, 10, 9, 9}},
};
static const CostKindTblEntry AVX2CostTbl[] = {
- { ISD::AND, MVT::v4i64, {2, 7, 5, 5} },
- { ISD::AND, MVT::v2i64, {1, 2, 3, 3} },
- { ISD::AND, MVT::v8i32, {3, 9, 7, 7} },
- { ISD::AND, MVT::v4i32, {2, 4, 5, 5} },
- { ISD::AND, MVT::v16i16, {3,11, 9, 9} },
- { ISD::AND, MVT::v8i16, {2, 6, 7, 7} },
- { ISD::AND, MVT::v32i8, {3,13,11,11} },
- { ISD::AND, MVT::v16i8, {3, 8, 9, 9} },
- { ISD::OR, MVT::v4i64, {2, 7, 5, 5} },
- { ISD::OR, MVT::v2i64, {1, 2, 3, 3} },
- { ISD::OR, MVT::v8i32, {3, 9, 7, 7} },
- { ISD::OR, MVT::v4i32, {2, 4, 5, 5} },
- { ISD::OR, MVT::v16i16, {3,11, 9, 9} },
- { ISD::OR, MVT::v8i16, {2, 6, 7, 7} },
- { ISD::OR, MVT::v32i8, {3,13,11,11} },
- { ISD::OR, MVT::v16i8, {3, 8, 9, 9} },
- { ISD::XOR, MVT::v4i64, {2, 7, 5, 5} },
- { ISD::XOR, MVT::v2i64, {1, 2, 3, 3} },
- { ISD::XOR, MVT::v8i32, {3, 9, 7, 7} },
- { ISD::XOR, MVT::v4i32, {2, 4, 5, 5} },
- { ISD::XOR, MVT::v16i16, {3,11, 9, 9} },
- { ISD::XOR, MVT::v8i16, {2, 6, 7, 7} },
- { ISD::XOR, MVT::v32i8, {3,13,11,11} },
- { ISD::XOR, MVT::v16i8, {3, 8, 9, 9} },
+ {ISD::AND, MVT::v4i64, {2, 7, 5, 5}},
+ {ISD::AND, MVT::v2i64, {1, 2, 3, 3}},
+ {ISD::AND, MVT::v8i32, {3, 9, 7, 7}},
+ {ISD::AND, MVT::v4i32, {2, 4, 5, 5}},
+ {ISD::AND, MVT::v16i16, {3, 11, 9, 9}},
+ {ISD::AND, MVT::v8i16, {2, 6, 7, 7}},
+ {ISD::AND, MVT::v32i8, {3, 13, 11, 11}},
+ {ISD::AND, MVT::v16i8, {3, 8, 9, 9}},
+ {ISD::OR, MVT::v4i64, {2, 7, 5, 5}},
+ {ISD::OR, MVT::v2i64, {1, 2, 3, 3}},
+ {ISD::OR, MVT::v8i32, {3, 9, 7, 7}},
+ {ISD::OR, MVT::v4i32, {2, 4, 5, 5}},
+ {ISD::OR, MVT::v16i16, {3, 11, 9, 9}},
+ {ISD::OR, MVT::v8i16, {2, 6, 7, 7}},
+ {ISD::OR, MVT::v32i8, {3, 13, 11, 11}},
+ {ISD::OR, MVT::v16i8, {3, 8, 9, 9}},
+ {ISD::XOR, MVT::v4i64, {2, 7, 5, 5}},
+ {ISD::XOR, MVT::v2i64, {1, 2, 3, 3}},
+ {ISD::XOR, MVT::v8i32, {3, 9, 7, 7}},
+ {ISD::XOR, MVT::v4i32, {2, 4, 5, 5}},
+ {ISD::XOR, MVT::v16i16, {3, 11, 9, 9}},
+ {ISD::XOR, MVT::v8i16, {2, 6, 7, 7}},
+ {ISD::XOR, MVT::v32i8, {3, 13, 11, 11}},
+ {ISD::XOR, MVT::v16i8, {3, 8, 9, 9}},
};
static const CostKindTblEntry AVX512FCostTbl[] = {
- { ISD::FADD, MVT::v8f64, {4, 4, 4, 4} },
- { ISD::FADD, MVT::v16f32, {5, 5, 5, 5} },
- { ISD::ADD, MVT::v8i64, {4, 4, 4, 4} },
- { ISD::ADD, MVT::v16i32, {6, 6, 6, 6} },
-
- { ISD::AND, MVT::v8i64, {3,10, 7, 7} },
- { ISD::AND, MVT::v16i32, {4,12, 9, 9} },
- { ISD::AND, MVT::v32i16, {4,14,11,11} },
- { ISD::AND, MVT::v64i8, {4,16,13,13} },
- { ISD::AND, MVT::v16i8, {2, 8, 9, 9} },
- { ISD::OR, MVT::v8i64, {3,10, 7, 7} },
- { ISD::OR, MVT::v16i32, {4,12, 9, 9} },
- { ISD::OR, MVT::v32i16, {4,14,11,11} },
- { ISD::OR, MVT::v64i8, {4,16,13,13} },
- { ISD::OR, MVT::v16i8, {2, 8, 9, 9} },
- { ISD::XOR, MVT::v8i64, {3,10, 7, 7} },
- { ISD::XOR, MVT::v16i32, {4,12, 9, 9} },
- { ISD::XOR, MVT::v32i16, {4,14,11,11} },
- { ISD::XOR, MVT::v64i8, {4,16,13,13} },
- { ISD::XOR, MVT::v16i8, {2, 8, 9, 9} },
+ {ISD::FADD, MVT::v8f64, {4, 4, 4, 4}},
+ {ISD::FADD, MVT::v16f32, {5, 5, 5, 5}},
+ {ISD::ADD, MVT::v8i64, {4, 4, 4, 4}},
+ {ISD::ADD, MVT::v16i32, {6, 6, 6, 6}},
+
+ {ISD::AND, MVT::v8i64, {3, 10, 7, 7}},
+ {ISD::AND, MVT::v16i32, {4, 12, 9, 9}},
+ {ISD::AND, MVT::v32i16, {4, 14, 11, 11}},
+ {ISD::AND, MVT::v64i8, {4, 16, 13, 13}},
+ {ISD::AND, MVT::v16i8, {2, 8, 9, 9}},
+ {ISD::OR, MVT::v8i64, {3, 10, 7, 7}},
+ {ISD::OR, MVT::v16i32, {4, 12, 9, 9}},
+ {ISD::OR, MVT::v32i16, {4, 14, 11, 11}},
+ {ISD::OR, MVT::v64i8, {4, 16, 13, 13}},
+ {ISD::OR, MVT::v16i8, {2, 8, 9, 9}},
+ {ISD::XOR, MVT::v8i64, {3, 10, 7, 7}},
+ {ISD::XOR, MVT::v16i32, {4, 12, 9, 9}},
+ {ISD::XOR, MVT::v32i16, {4, 14, 11, 11}},
+ {ISD::XOR, MVT::v64i8, {4, 16, 13, 13}},
+ {ISD::XOR, MVT::v16i8, {2, 8, 9, 9}},
};
static const CostKindTblEntry AVX512BWCostTbl[] = {
|
You can test this locally with the following command:git diff -U0 --pickaxe-regex -S '([^a-zA-Z0-9#_-]undef([^a-zA-Z0-9_-]|$)|UndefValue::get)' 'HEAD~1' HEAD llvm/lib/Target/X86/X86TargetTransformInfo.cpp llvm/test/Analysis/CostModel/X86/reduce-and.ll llvm/test/Analysis/CostModel/X86/reduce-or.ll llvm/test/Analysis/CostModel/X86/reduce-xor.ll llvm/test/Transforms/PhaseOrdering/X86/avg.ll llvm/test/Transforms/SLPVectorizer/X86/bool-mask.ll llvm/test/Transforms/SLPVectorizer/X86/extractelemets-extended-by-poison.ll llvm/test/Transforms/SLPVectorizer/X86/multi-use-bitcasted-reduction.ll llvm/test/Transforms/SLPVectorizer/X86/reduced-val-extracted-and-externally-used.ll llvm/test/Transforms/SLPVectorizer/X86/replaced-external-in-reduction.ll llvm/test/Transforms/SLPVectorizer/X86/same-values-sub-node-with-poisons.ll llvm/test/Transforms/VectorCombine/X86/fold-equivalent-reduction-cmp.ll llvm/test/Transforms/VectorCombine/X86/fold-reduction-zero-test.ll llvm/test/Transforms/VectorCombine/X86/fold-signbit-reduction-cmp-codesize.ll llvm/test/Transforms/VectorCombine/X86/fold-signbit-reduction-cmp.ll llvm/test/Transforms/VectorCombine/X86/icmp-vector-reduce.llThe following files introduce new uses of undef:
Undef is now deprecated and should only be used in the rare cases where no replacement is possible. For example, a load of uninitialized memory yields In tests, avoid using For example, this is considered a bad practice: define void @fn() {
...
br i1 undef, ...
}Please use the following instead: define void @fn(i1 %cond) {
...
br i1 %cond, ...
}Please refer to the Undefined Behavior Manual for more information. |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
ParkHanbum
reviewed
Aug 6, 2026
| { ISD::AND, MVT::v16i16, {5,11, 9, 9} }, | ||
| { ISD::AND, MVT::v8i16, {4, 7, 7, 7} }, | ||
| { ISD::AND, MVT::v32i8, {6,13,11,11} }, | ||
| { ISD::AND, MVT::v16i8, {5,10, 9, 9} }, |
Contributor
There was a problem hiding this comment.
I compared all the values, and this only happens for SSE2 and AVX512F v16i8: why is 5.5 rounded down to 5, while 1.5 is rounded up to 2?
Contributor
Author
There was a problem hiding this comment.
do you have an example?
my script should always be rounding up: https://github.com/RKSimon/llvm-scripts/blob/main/check_cost_tables.py#L101-L117
Contributor
There was a problem hiding this comment.
SSE case
% cat x86-reduce-mca-results/SSE2_AND_v16i8/silvermont/reduce_and_v16i8.mca.s
# LLVM-MCA-BEGIN reduce_and_v16i8
pshufd $238, %xmm0, %xmm1
pand %xmm0, %xmm1
pshufd $85, %xmm1, %xmm0
pand %xmm1, %xmm0
movdqa %xmm0, %xmm1
psrld $16, %xmm1
pand %xmm0, %xmm1
movdqa %xmm1, %xmm0
psrlw $8, %xmm0
pand %xmm1, %xmm0
movd %xmm0, %eax
# LLVM-MCA-END reduce_and_v16i8
% /Users/b2sy/work/build/bin/llvm-mca /Users/b2sy/work/llvm-x86-reduce-mca-verify/x86-reduce-mca-results/SSE2_AND_v16i8/silvermont/reduce_and_v16i8.mca.s -mtriple=x86_64-unknown-linux-gnu -mcpu=silvermont -iterations=100 -instruction-info -resource-pressure
[0] Code Region - reduce_and_v16i8
Iterations: 100
Instructions: 1100
Total Cycles: 1004
Total uOps: 1100
Dispatch Width: 2
uOps Per Cycle: 1.10
IPC: 1.10
Block RThroughput: 5.5
AVX512f case
b2sy@mac llvm-x86-reduce-mca-verify % cat /Users/b2sy/work/llvm-x86-reduce-mca-verify/x86-reduce-mca-results/AVX512F_AND_v16i8/skylake-avx512/reduce_and_v16i8.mca.s
# LLVM-MCA-BEGIN reduce_and_v16i8
vpshufd $238, %xmm0, %xmm1
vpand %xmm1, %xmm0, %xmm0
vpsrlq $32, %xmm0, %xmm1
vpand %xmm1, %xmm0, %xmm0
vpsrld $16, %xmm0, %xmm1
vpand %xmm1, %xmm0, %xmm0
vpsrlw $8, %xmm0, %xmm1
vpand %xmm1, %xmm0, %xmm0
vmovd %xmm0, %eax
# LLVM-MCA-END reduce_and_v16i8
b2sy@mac llvm-x86-reduce-mca-verify % /Users/b2sy/work/build/bin/llvm-mca /Users/b2sy/work/llvm-x86-reduce-mca-verify/x86-reduce-mca-results/AVX512F_AND_v16i8/skylake-avx512/reduce_and_v16i8.mca.s -mtriple=x86_64-unknown-linux-gnu -mcpu=skylake-avx512 -iterations=100 -instruction-info -resource-pressure
[0] Code Region - reduce_and_v16i8
Iterations: 100
Instructions: 900
Total Cycles: 805
Total uOps: 900
Dispatch Width: 6
uOps Per Cycle: 1.12
IPC: 1.12
Block RThroughput: 1.5
what do I wrong?
Contributor
There was a problem hiding this comment.
llc option : https://compiler-explorer.com/z/MjabKEdn8
Contributor
Author
There was a problem hiding this comment.
nice catch - I'd disabled atom cpus due to their terrible i64 handling when I was working on vecreduce_add, and failed to re-enable it
Contributor
Author
There was a problem hiding this comment.
Fixed for silvermont and above - the bonnell model has some very dodgy numbers for gpr<->vec moves that makes the llvm-mca stats very suspicious
RKSimon
added a commit
that referenced
this pull request
Aug 6, 2026
These had matching high latency and throughput numbers, which suggest they hadn't been updated since the itinerary conversion. Confirmed with Agner, uops.info and instlatx64 which report a consistent throughput of 2cy, latency seems to be about 4cy (uops.info just says <= 6cy) Noticed while trying to confirm #214191 vector reduction costs for atom type cpus, and llvm-mca was reporting some very odd numbers for bonnell.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the olds costs which were just instruction counts
Still working on the bool reduction costs - these will be handled in a followup