[CIR] Implement builtin extractf #170427
Open
[CIR] Implement builtin extractf #170427
+275
−7
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
badumbatish
requested review from
andykaylor,
bcardosolopes,
lanza and
xlauko
as code owners
llvmbot
added
clang
Contributor
Author
|
merge conflict, will fix soon |
Member
|
@llvm/pr-subscribers-clang Author: Jasmine Tang (badumbatish) ChangesImplement builtin extractf, tests are from clang/test/CodeGen/X86/avx512f-builtins.c. I'm not sure why the OG tests are very succinct but i'm porting the same testing format over from OG. I added a new type constraint "element or vector of element" since LLVMIR also has said constraint. The new getBoolMaskValue is because the existing SelectOp already accepts only a boolean condition; it'd make more sense for it to accept a vector of boolean instead of a vector of i32. Full diff: https://github.com/llvm/llvm-project/pull/170427.diff 4 Files Affected:
diff --git a/clang/include/clang/CIR/Dialect/IR/CIROps.td b/clang/include/clang/CIR/Dialect/IR/CIROps.td
index 4b64fc56c57ad..40203e21c8f18 100644
--- a/clang/include/clang/CIR/Dialect/IR/CIROps.td
+++ b/clang/include/clang/CIR/Dialect/IR/CIROps.td
@@ -1841,8 +1841,8 @@ def CIR_SelectOp : CIR_Op<"select", [
let summary = "Yield one of two values based on a boolean value";
let description = [{
The `cir.select` operation takes three operands. The first operand
- `condition` is a boolean value of type `!cir.bool`. The second and the third
- operand can be of any CIR types, but their types must be the same. If the
+ `condition` is either a boolean value of type `!cir.bool` or a boolean vector of type `!cir.bool`.
+ The second and the third operand can be of any CIR types, but their types must be the same. If the
first operand is `true`, the operation yields its second operand. Otherwise,
the operation yields its third operand.
@@ -1856,7 +1856,7 @@ def CIR_SelectOp : CIR_Op<"select", [
```
}];
- let arguments = (ins CIR_BoolType:$condition, CIR_AnyType:$true_value,
+ let arguments = (ins CIR_ScalarOrVectorOf<CIR_BoolType>:$condition, CIR_AnyType:$true_value,
CIR_AnyType:$false_value);
let results = (outs CIR_AnyType:$result);
diff --git a/clang/include/clang/CIR/Dialect/IR/CIRTypeConstraints.td b/clang/include/clang/CIR/Dialect/IR/CIRTypeConstraints.td
index ddca98eac93ab..dd514d755ce24 100644
--- a/clang/include/clang/CIR/Dialect/IR/CIRTypeConstraints.td
+++ b/clang/include/clang/CIR/Dialect/IR/CIRTypeConstraints.td
@@ -250,8 +250,8 @@ def CIR_PtrToArray : CIR_PtrToType<CIR_AnyArrayType>;
def CIR_AnyVectorType : CIR_TypeBase<"::cir::VectorType", "vector type">;
-def CIR_VectorElementType : AnyTypeOf<[CIR_AnyIntOrFloatType, CIR_AnyPtrType],
- "any cir integer, floating point or pointer type"
+def CIR_VectorElementType : AnyTypeOf<[CIR_AnyBoolType, CIR_AnyIntOrFloatType, CIR_AnyPtrType],
+ "any cir boolean, integer, floating point or pointer type"
> {
let cppFunctionName = "isValidVectorTypeElementType";
}
@@ -266,6 +266,16 @@ class CIR_VectorTypeOf<list<Type> types, string summary = "">
"vector of " # CIR_TypeSummaries<types>.value,
summary)>;
+class CIR_VectorOf<Type T> : CIR_ConfinedType<
+ CIR_AnyVectorType,
+ [CIR_ElementTypePred<T.predicate>],
+ "CIR vector of " # T.summary>;
+
+// Type constraint accepting a either a type T or a vector of type T
+// Mimicking LLVMIR's LLVM_ScalarOrVectorOf
+class CIR_ScalarOrVectorOf<Type T> :
+ AnyTypeOf<[T, CIR_VectorOf<T>]>;
+
// Vector of integral type
def IntegerVector : Type<
And<[
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index 224a182ed17d1..97e25136ba3eb 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -85,6 +85,71 @@ static mlir::Value getMaskVecValue(CIRGenBuilderTy &builder, mlir::Location loc,
return maskVec;
}
+static mlir::Value getBoolMaskVecValue(CIRGenBuilderTy &builder,
+ mlir::Location loc, mlir::Value mask,
+ unsigned numElems) {
+
+ cir::BoolType boolTy = builder.getBoolTy();
+ auto maskTy = cir::VectorType::get(
+ boolTy, cast<cir::IntType>(mask.getType()).getWidth());
+ mlir::Value maskVec = builder.createBitcast(mask, maskTy);
+
+ if (numElems < 8) {
+ SmallVector<mlir::Attribute, 4> indices;
+ mlir::Type i32Ty = builder.getSInt32Ty();
+ for (auto i : llvm::seq<unsigned>(0, numElems))
+ indices.push_back(cir::IntAttr::get(i32Ty, i));
+
+ maskVec = builder.createVecShuffle(loc, maskVec, maskVec, indices);
+ }
+ return maskVec;
+}
+
+// Helper function mirroring OG's bool Constant::isAllOnesValue()
+static bool isAllOnesValue(mlir::Value value) {
+ auto constOp = mlir::dyn_cast_or_null<cir::ConstantOp>(value.getDefiningOp());
+ if (!constOp)
+ return false;
+
+ // Check for -1 integers
+ if (auto intAttr = constOp.getValueAttr<cir::IntAttr>()) {
+ return intAttr.getValue().isAllOnes();
+ }
+
+ // Check for FP which are bitcasted from -1 integers
+ if (auto fpAttr = constOp.getValueAttr<cir::FPAttr>()) {
+ return fpAttr.getValue().bitcastToAPInt().isAllOnes();
+ }
+
+ // Check for constant vectors with splat values
+ if (cir::VectorType v = dyn_cast<cir::VectorType>(constOp.getType())) {
+ if (auto vecAttr = constOp.getValueAttr<mlir::DenseElementsAttr>()) {
+ if (vecAttr.isSplat()) {
+ auto splatAttr = vecAttr.getSplatValue<mlir::Attribute>();
+ if (auto splatInt = mlir::dyn_cast<cir::IntAttr>(splatAttr)) {
+ return splatInt.getValue().isAllOnes();
+ }
+ }
+ }
+ }
+
+ return false;
+}
+
+static mlir::Value emitX86Select(CIRGenBuilderTy &builder, mlir::Location loc,
+ mlir::Value mask, mlir::Value op0,
+ mlir::Value op1) {
+
+ // If the mask is all ones just return first argument.
+ if (isAllOnesValue(mask))
+ return op0;
+
+ mask = getBoolMaskVecValue(builder, loc, mask,
+ cast<cir::VectorType>(op0.getType()).getSize());
+
+ return builder.createSelect(loc, mask, op0, op1);
+}
+
static mlir::Value emitX86MaskAddLogic(CIRGenBuilderTy &builder,
mlir::Location loc,
const std::string &intrinsicName,
@@ -701,7 +766,31 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned builtinID,
case X86::BI__builtin_ia32_extractf64x2_256_mask:
case X86::BI__builtin_ia32_extracti64x2_256_mask:
case X86::BI__builtin_ia32_extractf64x2_512_mask:
- case X86::BI__builtin_ia32_extracti64x2_512_mask:
+ case X86::BI__builtin_ia32_extracti64x2_512_mask: {
+ mlir::Location loc = getLoc(expr->getExprLoc());
+ cir::VectorType dstTy = cast<cir::VectorType>(convertType(expr->getType()));
+ unsigned numElts = dstTy.getSize();
+ unsigned srcNumElts = cast<cir::VectorType>(ops[0].getType()).getSize();
+ unsigned subVectors = srcNumElts / numElts;
+ unsigned index =
+ ops[1].getDefiningOp<cir::ConstantOp>().getIntValue().getZExtValue();
+
+ index &= subVectors - 1; // Remove any extra bits.
+ index *= numElts;
+
+ int64_t indices[16];
+ for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i + index;
+
+ mlir::Value zero = builder.getNullValue(ops[0].getType(), loc);
+ mlir::Value res =
+ builder.createVecShuffle(loc, ops[0], zero, ArrayRef(indices, numElts));
+ if (ops.size() == 4) {
+ res = emitX86Select(builder, loc, ops[3], res, ops[2]);
+ }
+
+ return res;
+ }
case X86::BI__builtin_ia32_vinsertf128_pd256:
case X86::BI__builtin_ia32_vinsertf128_ps256:
case X86::BI__builtin_ia32_vinsertf128_si256:
diff --git a/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c b/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c
index e03109510a931..d5d209f9c4417 100644
--- a/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c
+++ b/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c
@@ -446,3 +446,181 @@ __m512i test_mm512_mask_i32gather_epi64(__m512i __v1_old, __mmask8 __mask, __m25
// OGCG: call <8 x i64> @llvm.x86.avx512.mask.gather.dpq.512
return _mm512_mask_i32gather_epi64(__v1_old, __mask, __index, __addr, 2);
}
+
+__m256d test_mm512_extractf64x4_pd(__m512d a)
+{
+ // CIR-LABEL: test_mm512_extractf64x4_pd
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !cir.double>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !cir.double>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.double>
+
+ // LLVM-LABEL: test_mm512_extractf64x4_pd
+ // LLVM: shufflevector <8 x double> %{{.*}}, <8 x double> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+
+ // OGCG-LABEL: test_mm512_extractf64x4_pd
+ // OGCG: shufflevector <8 x double> %{{.*}}, <8 x double> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ return _mm512_extractf64x4_pd(a, 1);
+}
+
+__m256d test_mm512_mask_extractf64x4_pd(__m256d __W,__mmask8 __U,__m512d __A){
+ // CIR-LABEL: test_mm512_mask_extractf64x4_pd
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !cir.double>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !cir.double>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.double>
+ // CIR: cir.select if %{{.*}} then %{{.*}} else %{{.*}} : (!cir.vector<4 x !cir.bool>, !cir.vector<4 x !cir.double>, !cir.vector<4 x !cir.double>) -> !cir.vector<4 x !cir.double>
+
+ // LLVM-LABEL: test_mm512_mask_extractf64x4_pd
+ // LLVM: shufflevector <8 x double> %{{.*}}, <8 x double> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x double> %{{.*}}, <4 x double> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_mask_extractf64x4_pd
+ // OGCG: shufflevector <8 x double> %{{.*}}, <8 x double> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x double> %{{.*}}, <4 x double> %{{.*}}
+ return _mm512_mask_extractf64x4_pd( __W, __U, __A, 1);
+}
+
+__m256d test_mm512_maskz_extractf64x4_pd(__mmask8 __U,__m512d __A){
+ // CIR-LABEL: test_mm512_maskz_extractf64x4_pd
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !cir.double>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !cir.double>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.double>
+ // CIR: cir.select if %{{.*}} then %{{.*}} else %{{.*}} : (!cir.vector<4 x !cir.bool>, !cir.vector<4 x !cir.double>, !cir.vector<4 x !cir.double>) -> !cir.vector<4 x !cir.double>
+
+ // LLVM-LABEL: test_mm512_maskz_extractf64x4_pd
+ // LLVM: shufflevector <8 x double> %{{.*}}, <8 x double> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x double> %{{.*}}, <4 x double> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_maskz_extractf64x4_pd
+ // OGCG: shufflevector <8 x double> %{{.*}}, <8 x double> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x double> %{{.*}}, <4 x double> %{{.*}}
+ return _mm512_maskz_extractf64x4_pd( __U, __A, 1);
+}
+
+__m128 test_mm512_extractf32x4_ps(__m512 a)
+{
+ // CIR-LABEL: test_mm512_extractf32x4_ps
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !cir.float>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !cir.float>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.float>
+
+ // LLVM-LABEL: test_mm512_extractf32x4_ps
+ // LLVM: shufflevector <16 x float> %{{.*}}, <16 x float> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+
+ // OGCG-LABEL: test_mm512_extractf32x4_ps
+ // OGCG: shufflevector <16 x float> %{{.*}}, <16 x float> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ return _mm512_extractf32x4_ps(a, 1);
+}
+
+__m128 test_mm512_mask_extractf32x4_ps(__m128 __W, __mmask8 __U,__m512 __A){
+ // CIR-LABEL: test_mm512_mask_extractf32x4_ps
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !cir.float>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !cir.float>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.float>
+
+ // LLVM-LABEL: test_mm512_mask_extractf32x4_ps
+ // LLVM: shufflevector <16 x float> %{{.*}}, <16 x float> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x float> %{{.*}}, <4 x float> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_mask_extractf32x4_ps
+ // OGCG: shufflevector <16 x float> %{{.*}}, <16 x float> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x float> %{{.*}}, <4 x float> %{{.*}}
+ return _mm512_mask_extractf32x4_ps( __W, __U, __A, 1);
+}
+
+__m128 test_mm512_maskz_extractf32x4_ps( __mmask8 __U,__m512 __A){
+ // CIR-LABEL: test_mm512_maskz_extractf32x4_ps
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !cir.float>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !cir.float>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.float>
+
+ // LLVM-LABEL: test_mm512_maskz_extractf32x4_ps
+ // LLVM: shufflevector <16 x float> %{{.*}}, <16 x float> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x float> %{{.*}}, <4 x float> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_maskz_extractf32x4_ps
+ // OGCG: shufflevector <16 x float> %{{.*}}, <16 x float> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x float> %{{.*}}, <4 x float> %{{.*}}
+ return _mm512_maskz_extractf32x4_ps(__U, __A, 1);
+}
+
+__m128i test_mm512_extracti32x4_epi32(__m512i __A) {
+ // CIR-LABEL: test_mm512_extracti32x4_epi32
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !s32i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !s32i>) [#cir.int<12> : !s32i, #cir.int<13> : !s32i, #cir.int<14> : !s32i, #cir.int<15> : !s32i] : !cir.vector<4 x !s32i>
+
+ // LLVM-LABEL: test_mm512_extracti32x4_epi32
+ // LLVM: shufflevector <16 x i32> %{{.*}}, <16 x i32> zeroinitializer, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+
+ // OGCG-LABEL: test_mm512_extracti32x4_epi32
+ // OGCG: shufflevector <16 x i32> %{{.*}}, <16 x i32> poison, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ return _mm512_extracti32x4_epi32(__A, 3);
+}
+
+__m128i test_mm512_mask_extracti32x4_epi32(__m128i __W, __mmask8 __U, __m512i __A) {
+ // CIR-LABEL: test_mm512_mask_extracti32x4_epi32
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !s32i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !s32i>) [#cir.int<12> : !s32i, #cir.int<13> : !s32i, #cir.int<14> : !s32i, #cir.int<15> : !s32i] : !cir.vector<4 x !s32i>
+ // CIR: cir.select if %{{.*}} then %{{.*}} else %{{.*}} : (!cir.vector<4 x !cir.bool>, !cir.vector<4 x !s32i>, !cir.vector<4 x !s32i>) -> !cir.vector<4 x !s32i>
+
+ // LLVM-LABEL: test_mm512_mask_extracti32x4_epi32
+ // LLVM: shufflevector <16 x i32> %{{.*}}, <16 x i32> zeroinitializer, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_mask_extracti32x4_epi32
+ // OGCG: shufflevector <16 x i32> %{{.*}}, <16 x i32> poison, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}}
+ return _mm512_mask_extracti32x4_epi32(__W, __U, __A, 3);
+}
+
+__m128i test_mm512_maskz_extracti32x4_epi32(__mmask8 __U, __m512i __A) {
+ // CIR-LABEL: test_mm512_maskz_extracti32x4_epi32
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !s32i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !s32i>) [#cir.int<12> : !s32i, #cir.int<13> : !s32i, #cir.int<14> : !s32i, #cir.int<15> : !s32i] : !cir.vector<4 x !s32i>
+ // CIR: cir.select if %{{.*}} then %{{.*}} else %{{.*}} : (!cir.vector<4 x !cir.bool>, !cir.vector<4 x !s32i>, !cir.vector<4 x !s32i>) -> !cir.vector<4 x !s32i>
+
+ // LLVM-LABEL: test_mm512_maskz_extracti32x4_epi32
+ // LLVM: shufflevector <16 x i32> %{{.*}}, <16 x i32> zeroinitializer, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_maskz_extracti32x4_epi32
+ // OGCG: shufflevector <16 x i32> %{{.*}}, <16 x i32> poison, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}}
+ return _mm512_maskz_extracti32x4_epi32(__U, __A, 3);
+}
+
+__m256i test_mm512_extracti64x4_epi64(__m512i __A) {
+ // CIR-LABEL: test_mm512_extracti64x4_epi64
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !s64i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !s64i>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !s64i>
+
+ // LLVM-LABEL: test_mm512_extracti64x4_epi64
+ // LLVM: shufflevector <8 x i64> %{{.*}}, <8 x i64> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+
+ // OGCG-LABEL: test_mm512_extracti64x4_epi64
+ // OGCG: shufflevector <8 x i64> %{{.*}}, <8 x i64> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ return _mm512_extracti64x4_epi64(__A, 1);
+}
+
+__m256i test_mm512_mask_extracti64x4_epi64(__m256i __W, __mmask8 __U, __m512i __A) {
+ // CIR-LABEL: test_mm512_mask_extracti64x4_epi64
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !s64i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !s64i>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !s64i>
+
+ // LLVM-LABEL: test_mm512_mask_extracti64x4_epi64
+ // LLVM: shufflevector <8 x i64> %{{.*}}, <8 x i64> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_mask_extracti64x4_epi64
+ // OGCG: shufflevector <8 x i64> %{{.*}}, <8 x i64> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
+ return _mm512_mask_extracti64x4_epi64(__W, __U, __A, 1);
+}
+
+__m256i test_mm512_maskz_extracti64x4_epi64(__mmask8 __U, __m512i __A) {
+ // CIR-LABEL: test_mm512_maskz_extracti64x4_epi64
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !s64i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !s64i>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !s64i>
+
+ // LLVM-LABEL: test_mm512_maskz_extracti64x4_epi64
+ // LLVM: shufflevector <8 x i64> %{{.*}}, <8 x i64> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_maskz_extracti64x4_epi64
+ // OGCG: shufflevector <8 x i64> %{{.*}}, <8 x i64> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
+ return _mm512_maskz_extracti64x4_epi64(__U, __A, 1);
+}
|
Member
|
@llvm/pr-subscribers-clangir Author: Jasmine Tang (badumbatish) ChangesImplement builtin extractf, tests are from clang/test/CodeGen/X86/avx512f-builtins.c. I'm not sure why the OG tests are very succinct but i'm porting the same testing format over from OG. I added a new type constraint "element or vector of element" since LLVMIR also has said constraint. The new getBoolMaskValue is because the existing SelectOp already accepts only a boolean condition; it'd make more sense for it to accept a vector of boolean instead of a vector of i32. Full diff: https://github.com/llvm/llvm-project/pull/170427.diff 4 Files Affected:
diff --git a/clang/include/clang/CIR/Dialect/IR/CIROps.td b/clang/include/clang/CIR/Dialect/IR/CIROps.td
index 4b64fc56c57ad..40203e21c8f18 100644
--- a/clang/include/clang/CIR/Dialect/IR/CIROps.td
+++ b/clang/include/clang/CIR/Dialect/IR/CIROps.td
@@ -1841,8 +1841,8 @@ def CIR_SelectOp : CIR_Op<"select", [
let summary = "Yield one of two values based on a boolean value";
let description = [{
The `cir.select` operation takes three operands. The first operand
- `condition` is a boolean value of type `!cir.bool`. The second and the third
- operand can be of any CIR types, but their types must be the same. If the
+ `condition` is either a boolean value of type `!cir.bool` or a boolean vector of type `!cir.bool`.
+ The second and the third operand can be of any CIR types, but their types must be the same. If the
first operand is `true`, the operation yields its second operand. Otherwise,
the operation yields its third operand.
@@ -1856,7 +1856,7 @@ def CIR_SelectOp : CIR_Op<"select", [
```
}];
- let arguments = (ins CIR_BoolType:$condition, CIR_AnyType:$true_value,
+ let arguments = (ins CIR_ScalarOrVectorOf<CIR_BoolType>:$condition, CIR_AnyType:$true_value,
CIR_AnyType:$false_value);
let results = (outs CIR_AnyType:$result);
diff --git a/clang/include/clang/CIR/Dialect/IR/CIRTypeConstraints.td b/clang/include/clang/CIR/Dialect/IR/CIRTypeConstraints.td
index ddca98eac93ab..dd514d755ce24 100644
--- a/clang/include/clang/CIR/Dialect/IR/CIRTypeConstraints.td
+++ b/clang/include/clang/CIR/Dialect/IR/CIRTypeConstraints.td
@@ -250,8 +250,8 @@ def CIR_PtrToArray : CIR_PtrToType<CIR_AnyArrayType>;
def CIR_AnyVectorType : CIR_TypeBase<"::cir::VectorType", "vector type">;
-def CIR_VectorElementType : AnyTypeOf<[CIR_AnyIntOrFloatType, CIR_AnyPtrType],
- "any cir integer, floating point or pointer type"
+def CIR_VectorElementType : AnyTypeOf<[CIR_AnyBoolType, CIR_AnyIntOrFloatType, CIR_AnyPtrType],
+ "any cir boolean, integer, floating point or pointer type"
> {
let cppFunctionName = "isValidVectorTypeElementType";
}
@@ -266,6 +266,16 @@ class CIR_VectorTypeOf<list<Type> types, string summary = "">
"vector of " # CIR_TypeSummaries<types>.value,
summary)>;
+class CIR_VectorOf<Type T> : CIR_ConfinedType<
+ CIR_AnyVectorType,
+ [CIR_ElementTypePred<T.predicate>],
+ "CIR vector of " # T.summary>;
+
+// Type constraint accepting a either a type T or a vector of type T
+// Mimicking LLVMIR's LLVM_ScalarOrVectorOf
+class CIR_ScalarOrVectorOf<Type T> :
+ AnyTypeOf<[T, CIR_VectorOf<T>]>;
+
// Vector of integral type
def IntegerVector : Type<
And<[
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index 224a182ed17d1..97e25136ba3eb 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -85,6 +85,71 @@ static mlir::Value getMaskVecValue(CIRGenBuilderTy &builder, mlir::Location loc,
return maskVec;
}
+static mlir::Value getBoolMaskVecValue(CIRGenBuilderTy &builder,
+ mlir::Location loc, mlir::Value mask,
+ unsigned numElems) {
+
+ cir::BoolType boolTy = builder.getBoolTy();
+ auto maskTy = cir::VectorType::get(
+ boolTy, cast<cir::IntType>(mask.getType()).getWidth());
+ mlir::Value maskVec = builder.createBitcast(mask, maskTy);
+
+ if (numElems < 8) {
+ SmallVector<mlir::Attribute, 4> indices;
+ mlir::Type i32Ty = builder.getSInt32Ty();
+ for (auto i : llvm::seq<unsigned>(0, numElems))
+ indices.push_back(cir::IntAttr::get(i32Ty, i));
+
+ maskVec = builder.createVecShuffle(loc, maskVec, maskVec, indices);
+ }
+ return maskVec;
+}
+
+// Helper function mirroring OG's bool Constant::isAllOnesValue()
+static bool isAllOnesValue(mlir::Value value) {
+ auto constOp = mlir::dyn_cast_or_null<cir::ConstantOp>(value.getDefiningOp());
+ if (!constOp)
+ return false;
+
+ // Check for -1 integers
+ if (auto intAttr = constOp.getValueAttr<cir::IntAttr>()) {
+ return intAttr.getValue().isAllOnes();
+ }
+
+ // Check for FP which are bitcasted from -1 integers
+ if (auto fpAttr = constOp.getValueAttr<cir::FPAttr>()) {
+ return fpAttr.getValue().bitcastToAPInt().isAllOnes();
+ }
+
+ // Check for constant vectors with splat values
+ if (cir::VectorType v = dyn_cast<cir::VectorType>(constOp.getType())) {
+ if (auto vecAttr = constOp.getValueAttr<mlir::DenseElementsAttr>()) {
+ if (vecAttr.isSplat()) {
+ auto splatAttr = vecAttr.getSplatValue<mlir::Attribute>();
+ if (auto splatInt = mlir::dyn_cast<cir::IntAttr>(splatAttr)) {
+ return splatInt.getValue().isAllOnes();
+ }
+ }
+ }
+ }
+
+ return false;
+}
+
+static mlir::Value emitX86Select(CIRGenBuilderTy &builder, mlir::Location loc,
+ mlir::Value mask, mlir::Value op0,
+ mlir::Value op1) {
+
+ // If the mask is all ones just return first argument.
+ if (isAllOnesValue(mask))
+ return op0;
+
+ mask = getBoolMaskVecValue(builder, loc, mask,
+ cast<cir::VectorType>(op0.getType()).getSize());
+
+ return builder.createSelect(loc, mask, op0, op1);
+}
+
static mlir::Value emitX86MaskAddLogic(CIRGenBuilderTy &builder,
mlir::Location loc,
const std::string &intrinsicName,
@@ -701,7 +766,31 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned builtinID,
case X86::BI__builtin_ia32_extractf64x2_256_mask:
case X86::BI__builtin_ia32_extracti64x2_256_mask:
case X86::BI__builtin_ia32_extractf64x2_512_mask:
- case X86::BI__builtin_ia32_extracti64x2_512_mask:
+ case X86::BI__builtin_ia32_extracti64x2_512_mask: {
+ mlir::Location loc = getLoc(expr->getExprLoc());
+ cir::VectorType dstTy = cast<cir::VectorType>(convertType(expr->getType()));
+ unsigned numElts = dstTy.getSize();
+ unsigned srcNumElts = cast<cir::VectorType>(ops[0].getType()).getSize();
+ unsigned subVectors = srcNumElts / numElts;
+ unsigned index =
+ ops[1].getDefiningOp<cir::ConstantOp>().getIntValue().getZExtValue();
+
+ index &= subVectors - 1; // Remove any extra bits.
+ index *= numElts;
+
+ int64_t indices[16];
+ for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i + index;
+
+ mlir::Value zero = builder.getNullValue(ops[0].getType(), loc);
+ mlir::Value res =
+ builder.createVecShuffle(loc, ops[0], zero, ArrayRef(indices, numElts));
+ if (ops.size() == 4) {
+ res = emitX86Select(builder, loc, ops[3], res, ops[2]);
+ }
+
+ return res;
+ }
case X86::BI__builtin_ia32_vinsertf128_pd256:
case X86::BI__builtin_ia32_vinsertf128_ps256:
case X86::BI__builtin_ia32_vinsertf128_si256:
diff --git a/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c b/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c
index e03109510a931..d5d209f9c4417 100644
--- a/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c
+++ b/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c
@@ -446,3 +446,181 @@ __m512i test_mm512_mask_i32gather_epi64(__m512i __v1_old, __mmask8 __mask, __m25
// OGCG: call <8 x i64> @llvm.x86.avx512.mask.gather.dpq.512
return _mm512_mask_i32gather_epi64(__v1_old, __mask, __index, __addr, 2);
}
+
+__m256d test_mm512_extractf64x4_pd(__m512d a)
+{
+ // CIR-LABEL: test_mm512_extractf64x4_pd
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !cir.double>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !cir.double>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.double>
+
+ // LLVM-LABEL: test_mm512_extractf64x4_pd
+ // LLVM: shufflevector <8 x double> %{{.*}}, <8 x double> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+
+ // OGCG-LABEL: test_mm512_extractf64x4_pd
+ // OGCG: shufflevector <8 x double> %{{.*}}, <8 x double> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ return _mm512_extractf64x4_pd(a, 1);
+}
+
+__m256d test_mm512_mask_extractf64x4_pd(__m256d __W,__mmask8 __U,__m512d __A){
+ // CIR-LABEL: test_mm512_mask_extractf64x4_pd
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !cir.double>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !cir.double>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.double>
+ // CIR: cir.select if %{{.*}} then %{{.*}} else %{{.*}} : (!cir.vector<4 x !cir.bool>, !cir.vector<4 x !cir.double>, !cir.vector<4 x !cir.double>) -> !cir.vector<4 x !cir.double>
+
+ // LLVM-LABEL: test_mm512_mask_extractf64x4_pd
+ // LLVM: shufflevector <8 x double> %{{.*}}, <8 x double> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x double> %{{.*}}, <4 x double> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_mask_extractf64x4_pd
+ // OGCG: shufflevector <8 x double> %{{.*}}, <8 x double> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x double> %{{.*}}, <4 x double> %{{.*}}
+ return _mm512_mask_extractf64x4_pd( __W, __U, __A, 1);
+}
+
+__m256d test_mm512_maskz_extractf64x4_pd(__mmask8 __U,__m512d __A){
+ // CIR-LABEL: test_mm512_maskz_extractf64x4_pd
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !cir.double>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !cir.double>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.double>
+ // CIR: cir.select if %{{.*}} then %{{.*}} else %{{.*}} : (!cir.vector<4 x !cir.bool>, !cir.vector<4 x !cir.double>, !cir.vector<4 x !cir.double>) -> !cir.vector<4 x !cir.double>
+
+ // LLVM-LABEL: test_mm512_maskz_extractf64x4_pd
+ // LLVM: shufflevector <8 x double> %{{.*}}, <8 x double> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x double> %{{.*}}, <4 x double> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_maskz_extractf64x4_pd
+ // OGCG: shufflevector <8 x double> %{{.*}}, <8 x double> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x double> %{{.*}}, <4 x double> %{{.*}}
+ return _mm512_maskz_extractf64x4_pd( __U, __A, 1);
+}
+
+__m128 test_mm512_extractf32x4_ps(__m512 a)
+{
+ // CIR-LABEL: test_mm512_extractf32x4_ps
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !cir.float>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !cir.float>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.float>
+
+ // LLVM-LABEL: test_mm512_extractf32x4_ps
+ // LLVM: shufflevector <16 x float> %{{.*}}, <16 x float> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+
+ // OGCG-LABEL: test_mm512_extractf32x4_ps
+ // OGCG: shufflevector <16 x float> %{{.*}}, <16 x float> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ return _mm512_extractf32x4_ps(a, 1);
+}
+
+__m128 test_mm512_mask_extractf32x4_ps(__m128 __W, __mmask8 __U,__m512 __A){
+ // CIR-LABEL: test_mm512_mask_extractf32x4_ps
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !cir.float>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !cir.float>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.float>
+
+ // LLVM-LABEL: test_mm512_mask_extractf32x4_ps
+ // LLVM: shufflevector <16 x float> %{{.*}}, <16 x float> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x float> %{{.*}}, <4 x float> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_mask_extractf32x4_ps
+ // OGCG: shufflevector <16 x float> %{{.*}}, <16 x float> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x float> %{{.*}}, <4 x float> %{{.*}}
+ return _mm512_mask_extractf32x4_ps( __W, __U, __A, 1);
+}
+
+__m128 test_mm512_maskz_extractf32x4_ps( __mmask8 __U,__m512 __A){
+ // CIR-LABEL: test_mm512_maskz_extractf32x4_ps
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !cir.float>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !cir.float>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !cir.float>
+
+ // LLVM-LABEL: test_mm512_maskz_extractf32x4_ps
+ // LLVM: shufflevector <16 x float> %{{.*}}, <16 x float> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x float> %{{.*}}, <4 x float> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_maskz_extractf32x4_ps
+ // OGCG: shufflevector <16 x float> %{{.*}}, <16 x float> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x float> %{{.*}}, <4 x float> %{{.*}}
+ return _mm512_maskz_extractf32x4_ps(__U, __A, 1);
+}
+
+__m128i test_mm512_extracti32x4_epi32(__m512i __A) {
+ // CIR-LABEL: test_mm512_extracti32x4_epi32
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !s32i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !s32i>) [#cir.int<12> : !s32i, #cir.int<13> : !s32i, #cir.int<14> : !s32i, #cir.int<15> : !s32i] : !cir.vector<4 x !s32i>
+
+ // LLVM-LABEL: test_mm512_extracti32x4_epi32
+ // LLVM: shufflevector <16 x i32> %{{.*}}, <16 x i32> zeroinitializer, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+
+ // OGCG-LABEL: test_mm512_extracti32x4_epi32
+ // OGCG: shufflevector <16 x i32> %{{.*}}, <16 x i32> poison, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ return _mm512_extracti32x4_epi32(__A, 3);
+}
+
+__m128i test_mm512_mask_extracti32x4_epi32(__m128i __W, __mmask8 __U, __m512i __A) {
+ // CIR-LABEL: test_mm512_mask_extracti32x4_epi32
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !s32i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !s32i>) [#cir.int<12> : !s32i, #cir.int<13> : !s32i, #cir.int<14> : !s32i, #cir.int<15> : !s32i] : !cir.vector<4 x !s32i>
+ // CIR: cir.select if %{{.*}} then %{{.*}} else %{{.*}} : (!cir.vector<4 x !cir.bool>, !cir.vector<4 x !s32i>, !cir.vector<4 x !s32i>) -> !cir.vector<4 x !s32i>
+
+ // LLVM-LABEL: test_mm512_mask_extracti32x4_epi32
+ // LLVM: shufflevector <16 x i32> %{{.*}}, <16 x i32> zeroinitializer, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_mask_extracti32x4_epi32
+ // OGCG: shufflevector <16 x i32> %{{.*}}, <16 x i32> poison, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}}
+ return _mm512_mask_extracti32x4_epi32(__W, __U, __A, 3);
+}
+
+__m128i test_mm512_maskz_extracti32x4_epi32(__mmask8 __U, __m512i __A) {
+ // CIR-LABEL: test_mm512_maskz_extracti32x4_epi32
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<16 x !s32i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<16 x !s32i>) [#cir.int<12> : !s32i, #cir.int<13> : !s32i, #cir.int<14> : !s32i, #cir.int<15> : !s32i] : !cir.vector<4 x !s32i>
+ // CIR: cir.select if %{{.*}} then %{{.*}} else %{{.*}} : (!cir.vector<4 x !cir.bool>, !cir.vector<4 x !s32i>, !cir.vector<4 x !s32i>) -> !cir.vector<4 x !s32i>
+
+ // LLVM-LABEL: test_mm512_maskz_extracti32x4_epi32
+ // LLVM: shufflevector <16 x i32> %{{.*}}, <16 x i32> zeroinitializer, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_maskz_extracti32x4_epi32
+ // OGCG: shufflevector <16 x i32> %{{.*}}, <16 x i32> poison, <4 x i32> <i32 12, i32 13, i32 14, i32 15>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x i32> %{{.*}}, <4 x i32> %{{.*}}
+ return _mm512_maskz_extracti32x4_epi32(__U, __A, 3);
+}
+
+__m256i test_mm512_extracti64x4_epi64(__m512i __A) {
+ // CIR-LABEL: test_mm512_extracti64x4_epi64
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !s64i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !s64i>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !s64i>
+
+ // LLVM-LABEL: test_mm512_extracti64x4_epi64
+ // LLVM: shufflevector <8 x i64> %{{.*}}, <8 x i64> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+
+ // OGCG-LABEL: test_mm512_extracti64x4_epi64
+ // OGCG: shufflevector <8 x i64> %{{.*}}, <8 x i64> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ return _mm512_extracti64x4_epi64(__A, 1);
+}
+
+__m256i test_mm512_mask_extracti64x4_epi64(__m256i __W, __mmask8 __U, __m512i __A) {
+ // CIR-LABEL: test_mm512_mask_extracti64x4_epi64
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !s64i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !s64i>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !s64i>
+
+ // LLVM-LABEL: test_mm512_mask_extracti64x4_epi64
+ // LLVM: shufflevector <8 x i64> %{{.*}}, <8 x i64> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_mask_extracti64x4_epi64
+ // OGCG: shufflevector <8 x i64> %{{.*}}, <8 x i64> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
+ return _mm512_mask_extracti64x4_epi64(__W, __U, __A, 1);
+}
+
+__m256i test_mm512_maskz_extracti64x4_epi64(__mmask8 __U, __m512i __A) {
+ // CIR-LABEL: test_mm512_maskz_extracti64x4_epi64
+ // CIR: [[ZERO:%.*]] = cir.const #cir.zero : !cir.vector<8 x !s64i>
+ // CIR: cir.vec.shuffle(%{{.*}}, [[ZERO]] : !cir.vector<8 x !s64i>) [#cir.int<4> : !s32i, #cir.int<5> : !s32i, #cir.int<6> : !s32i, #cir.int<7> : !s32i] : !cir.vector<4 x !s64i>
+
+ // LLVM-LABEL: test_mm512_maskz_extracti64x4_epi64
+ // LLVM: shufflevector <8 x i64> %{{.*}}, <8 x i64> zeroinitializer, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // LLVM: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
+
+ // OGCG-LABEL: test_mm512_maskz_extracti64x4_epi64
+ // OGCG: shufflevector <8 x i64> %{{.*}}, <8 x i64> poison, <4 x i32> <i32 4, i32 5, i32 6, i32 7>
+ // OGCG: select <4 x i1> %{{.*}}, <4 x i64> %{{.*}}, <4 x i64> %{{.*}}
+ return _mm512_maskz_extracti64x4_epi64(__U, __A, 1);
+}
|
xlauko
reviewed
Dec 3, 2025
badumbatish
force-pushed
the
cir_extract
branch
from
6f8a0e7 to
7de533b
Compare
andykaylor
reviewed
Dec 6, 2025
| // Check for FP which are bitcasted from -1 integers | ||
| if (auto fpAttr = constOp.getValueAttr<cir::FPAttr>()) | ||
| return fpAttr.getValue().bitcastToAPInt().isAllOnes(); | ||
|
|
Contributor
There was a problem hiding this comment.
Can you remove the extra blank line here?
| return boolAttr.getValue(); | ||
| llvm_unreachable("Expected a BoolAttr in ConstantOp"); | ||
| } | ||
| static bool isAllOnesValue(mlir::Value value) { |
Contributor
There was a problem hiding this comment.
This feels weird as a static function. @xlauko is that what you had in mind? I would have expected the dynamic cast to ConstantOp to happen at the callsite with this as a non-static function.
| `condition` is a boolean value of type `!cir.bool`. The second and the third | ||
| operand can be of any CIR types, but their types must be the same. If the | ||
| `condition` is either a boolean value of type `!cir.bool` or a boolean vector of type `!cir.bool`. | ||
| The second and the third operand can be of any CIR types, but their types must be the same. If the |
Contributor
There was a problem hiding this comment.
Please reformat this to fit in 80 columns.
| `condition` is either a boolean value of type `!cir.bool` or a boolean vector of type `!cir.bool`. | ||
| The second and the third operand can be of any CIR types, but their types must be the same. If the | ||
| first operand is `true`, the operation yields its second operand. Otherwise, | ||
| the operation yields its third operand. |
Contributor
There was a problem hiding this comment.
Can you describe the behavior in the case where the first operand is a vector of bool? In that case the second and third arguments must be vectors with the same number of elements as the first argument, right?
| CIR_AnyType:$true_value, | ||
| CIR_AnyType:$false_value | ||
| ); | ||
|
|
Contributor
There was a problem hiding this comment.
This probably needs a verifier for the case where the arguments are vectors.
Comment on lines
+253
to
255
| def CIR_VectorElementType : AnyTypeOf<[CIR_AnyBoolType, CIR_AnyIntOrFloatType, CIR_AnyPtrType], | ||
| "any cir boolean, integer, floating point or pointer type" | ||
| > { |
Contributor
There was a problem hiding this comment.
Suggested change
| def CIR_VectorElementType : AnyTypeOf<[CIR_AnyBoolType, CIR_AnyIntOrFloatType, CIR_AnyPtrType], | |
| "any cir boolean, integer, floating point or pointer type" | |
| > { | |
| def CIR_VectorElementType | |
| def CIR_VectorElementType | |
| : AnyTypeOf<[CIR_AnyBoolType, CIR_AnyIntOrFloatType, CIR_AnyPtrType], | |
| "any boolean, integer, floating point or pointer type"> { |
| mlir::Value maskVec = builder.createBitcast(mask, maskTy); | ||
|
|
||
| if (numElems < 8) { | ||
| SmallVector<mlir::Attribute, 4> indices; |
Contributor
There was a problem hiding this comment.
Suggested change
| SmallVector<mlir::Attribute, 4> indices; | |
| SmallVector<mlir::Attribute> indices; |
| int64_t indices[16]; | ||
| std::iota(indices, indices + numElts, index); | ||
|
|
||
| mlir::Value zero = builder.getNullValue(ops[0].getType(), loc); |
Contributor
There was a problem hiding this comment.
Suggested change
| mlir::Value zero = builder.getNullValue(ops[0].getType(), loc); | |
| mlir::Value | |
| poison = builder.getConstant(loc, cir::PoisonAttr::get(ops[0].getType())); |
|
|
||
| mlir::Value zero = builder.getNullValue(ops[0].getType(), loc); | ||
| mlir::Value res = | ||
| builder.createVecShuffle(loc, ops[0], zero, ArrayRef(indices, numElts)); |
Contributor
There was a problem hiding this comment.
Suggested change
| builder.createVecShuffle(loc, ops[0], zero, ArrayRef(indices, numElts)); | |
| builder.createVecShuffle(loc, ops[0], poison, ArrayRef(indices, numElts)); |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement builtin extractf, tests are from clang/test/CodeGen/X86/avx512f-builtins.c.
I'm not sure why the OG tests are very succinct but i'm porting the same testing format over from OG.
I added a new type constraint "element or vector of element" since LLVMIR also has said constraint. The new getBoolMaskValue is because the existing SelectOp already accepts only a boolean condition; it'd make more sense for it to accept a vector of boolean instead of a vector of i32.