[CIR][X86] Implement lowering for pmuldq / pmuludq builtins #169853
+238
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GeneraluseAI
requested review from
andykaylor,
bcardosolopes,
lanza and
xlauko
as code owners
|
Thank you for submitting a Pull Request (PR) to the LLVM Project! This PR will be automatically labeled and the relevant teams will be notified. If you wish to, you can add reviewers by using the "Reviewers" section on this page. If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers. If you have further questions, they may be answered by the LLVM GitHub User Guide. You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums. |
llvmbot
added
clang
Member
|
@llvm/pr-subscribers-clangir @llvm/pr-subscribers-clang Author: AIT (GeneraluseAI) Changespart of #167765 This patch adds CIR codegen support for X86 pmuldq and pmuludq operations, covering the signed and unsigned variants across all supported vector widths. The builtins now lower to the expected CIR representation matching the semantics of the corresponding LLVM intrinsics. Full diff: https://github.com/llvm/llvm-project/pull/169853.diff 5 Files Affected:
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index 0e43345bad6f1..dd9c3d90b793d 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -85,6 +85,47 @@ static mlir::Value getMaskVecValue(CIRGenBuilderTy &builder, mlir::Location loc,
return maskVec;
}
+static mlir::Value emitX86Muldq(CIRGenFunction &cgf, const CallExpr *expr,
+ bool isSigned,
+ SmallVectorImpl<mlir::Value> &ops) {
+ CIRGenBuilderTy &builder = cgf.getBuilder();
+ mlir::Location loc = cgf.getLoc(expr->getExprLoc());
+ mlir::Type ty = ops[0].getType();
+ unsigned tyPrimitiveSizeInBits =
+ cgf.cgm.getDataLayout().getTypeSizeInBits(ty);
+ mlir::Value lhs, rhs;
+ // in cir, if a shiftOperation is shift right,it will be translated into Ashr
+ // or lShr automatically in match and rewrite stage according to its operand's
+ // type
+ if (isSigned) {
+ ty =
+ cir::VectorType::get(builder.getSInt64Ty(), tyPrimitiveSizeInBits / 64);
+ cir::ConstantOp shiftAmt =
+ builder.getConstant(loc, cir::IntAttr::get(builder.getSInt64Ty(), 32));
+ cir::VecSplatOp shiftSplatVecOp =
+ cir::VecSplatOp::create(builder, loc, ty, shiftAmt.getResult());
+ mlir::Value shiftSplatValue = shiftSplatVecOp.getResult();
+ lhs = builder.createBitcast(loc, ops[0], ty);
+ rhs = builder.createBitcast(loc, ops[1], ty);
+ lhs = builder.createShift(loc, lhs, shiftSplatValue, true);
+ lhs = builder.createShift(loc, lhs, shiftSplatValue, false);
+ rhs = builder.createShift(loc, rhs, shiftSplatValue, true);
+ rhs = builder.createShift(loc, rhs, shiftSplatValue, false);
+ } else {
+ ty =
+ cir::VectorType::get(builder.getSInt64Ty(), tyPrimitiveSizeInBits / 64);
+ cir::ConstantOp maskScalar = builder.getConstant(
+ loc, cir::IntAttr::get(builder.getSInt64Ty(), 0xffffffff));
+ cir::VecSplatOp mask =
+ cir::VecSplatOp::create(builder, loc, ty, maskScalar.getResult());
+ lhs = builder.createBitcast(loc, ops[0], ty);
+ rhs = builder.createBitcast(loc, ops[1], ty);
+ lhs = builder.createAnd(loc, lhs, mask);
+ rhs = builder.createAnd(loc, rhs, mask);
+ }
+ return builder.createMul(loc, lhs, rhs);
+}
+
mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned builtinID,
const CallExpr *expr) {
if (builtinID == Builtin::BI__builtin_cpu_is) {
@@ -784,12 +825,18 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned builtinID,
case X86::BI__builtin_ia32_sqrtph512:
case X86::BI__builtin_ia32_sqrtps512:
case X86::BI__builtin_ia32_sqrtpd512:
+ cgm.errorNYI(expr->getSourceRange(),
+ std::string("unimplemented X86 builtin call: ") +
+ getContext().BuiltinInfo.getName(builtinID));
+ return {};
case X86::BI__builtin_ia32_pmuludq128:
case X86::BI__builtin_ia32_pmuludq256:
case X86::BI__builtin_ia32_pmuludq512:
+ return emitX86Muldq(*this, expr, /*IsSigned*/ false, ops);
case X86::BI__builtin_ia32_pmuldq128:
case X86::BI__builtin_ia32_pmuldq256:
case X86::BI__builtin_ia32_pmuldq512:
+ return emitX86Muldq(*this, expr, /*IsSigned*/ true, ops);
case X86::BI__builtin_ia32_pternlogd512_mask:
case X86::BI__builtin_ia32_pternlogq512_mask:
case X86::BI__builtin_ia32_pternlogd128_mask:
diff --git a/clang/test/CIR/CodeGenBuiltins/X86/avx2-builtins.c b/clang/test/CIR/CodeGenBuiltins/X86/avx2-builtins.c
new file mode 100644
index 0000000000000..bd8d7044da416
--- /dev/null
+++ b/clang/test/CIR/CodeGenBuiltins/X86/avx2-builtins.c
@@ -0,0 +1,68 @@
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx2 -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx2 -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s
+
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx2 -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx2 -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s
+
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-apple-darwin -target-feature +avx2 -emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s --check-prefixes=OGCG
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -fms-extensions -fms-compatibility -ffreestanding %s -triple=x86_64-windows-msvc -target-feature +avx2 -emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s --check-prefixes=OGCG
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-apple-darwin -target-feature +avx2 -emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s --check-prefixes=OGCG
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -fms-extensions -fms-compatibility -ffreestanding %s -triple=x86_64-windows-msvc -target-feature +avx2 -emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s --check-prefixes=OGCG
+
+#include <immintrin.h>
+
+__m256i test_mm256_mul_epu32(__m256i a, __m256i b) {
+// CIR-LABEL: _mm256_mul_epu32
+// CIR: [[MASK_SCALAR:%.*]] = cir.const #cir.int<4294967295> : !s64i
+// CIR: [[MASK_VEC:%.*]] = cir.vec.splat [[MASK_SCALAR]] : !s64i, !cir.vector<4 x !s64i>
+// CIR: [[BC_A:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<4 x !s64i>
+// CIR: [[BC_B:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<4 x !s64i>
+// CIR: [[AND_A:%.*]] = cir.binop(and, [[BC_A]], [[MASK_VEC]])
+// CIR: [[AND_B:%.*]] = cir.binop(and, [[BC_B]], [[MASK_VEC]])
+// CIR: [[MUL:%.*]] = cir.binop(mul, [[AND_A]], [[AND_B]])
+
+// LLVM-LABEL: _mm256_mul_epu32
+// LLVM: and <4 x i64> %{{.*}}, splat (i64 4294967295)
+// LLVM: and <4 x i64> %{{.*}}, splat (i64 4294967295)
+// LLVM: mul <4 x i64> %{{.*}}, %{{.*}}
+
+// OGCG-LABEL: _mm256_mul_epu32
+// OGCG: and <4 x i64> %{{.*}}, splat (i64 4294967295)
+// OGCG: and <4 x i64> %{{.*}}, splat (i64 4294967295)
+// OGCG: mul <4 x i64> %{{.*}}, %{{.*}}
+
+return _mm256_mul_epu32(a, b);
+}
+
+__m256i test_mm256_mul_epi32(__m256i a, __m256i b) {
+ // CIR-LABEL: _mm256_mul_epi32
+ // CIR: [[SC:%.*]] = cir.const #cir.int<32> : !s64i
+ // CIR: [[SV:%.*]] = cir.vec.splat [[SC]] : !s64i, !cir.vector<4 x !s64i>
+ // CIR: [[A64:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<4 x !s64i>
+ // CIR: [[B64:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<4 x !s64i>
+ // CIR: [[SHL_A:%.*]] = cir.shift(left, [[A64]] : !cir.vector<4 x !s64i>, [[SV]] : !cir.vector<4 x !s64i>)
+ // CIR: [[ASHR_A:%.*]] = cir.shift(right, [[SHL_A]] : !cir.vector<4 x !s64i>, [[SV]] : !cir.vector<4 x !s64i>)
+ // CIR: [[SHL_B:%.*]] = cir.shift(left, [[B64]] : !cir.vector<4 x !s64i>, [[SV]] : !cir.vector<4 x !s64i>)
+ // CIR: [[ASHR_B:%.*]] = cir.shift(right, [[SHL_B]] : !cir.vector<4 x !s64i>, [[SV]] : !cir.vector<4 x !s64i>)
+ // CIR: [[MUL:%.*]] = cir.binop(mul, [[ASHR_A]], [[ASHR_B]])
+
+ // LLVM-LABEL: _mm256_mul_epi32
+ // LLVM: shl <4 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: ashr <4 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: shl <4 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: ashr <4 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: mul <4 x i64> %{{.*}}, %{{.*}}
+
+ // OGCG-LABEL: _mm256_mul_epi32
+ // OGCG: shl <4 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: ashr <4 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: shl <4 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: ashr <4 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: mul <4 x i64> %{{.*}}, %{{.*}}
+
+ return _mm256_mul_epi32(a, b);
+}
diff --git a/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c b/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c
old mode 100644
new mode 100755
index dc54a87856a7c..ad29522fca506
--- a/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c
+++ b/clang/test/CIR/CodeGenBuiltins/X86/avx512f-builtins.c
@@ -77,3 +77,56 @@ __m512i test_mm512_undefined_epi32(void) {
// OGCG: ret <8 x i64> zeroinitializer
return _mm512_undefined_epi32();
}
+
+__m512i test_mm512_mul_epi32(__m512i __A, __m512i __B) {
+ // CIR-LABEL: _mm512_mul_epi32
+ // CIR: [[SC:%.*]] = cir.const #cir.int<32> : !s64i
+ // CIR: [[SV:%.*]] = cir.vec.splat [[SC]] : !s64i, !cir.vector<8 x !s64i>
+ // CIR: [[A64:%.*]] = cir.cast bitcast %{{.*}} : !cir.vector<16 x !s32i> -> !cir.vector<8 x !s64i>
+ // CIR: [[B64:%.*]] = cir.cast bitcast %{{.*}} : !cir.vector<16 x !s32i> -> !cir.vector<8 x !s64i>
+ // CIR: [[SHL_A:%.*]] = cir.shift(left, [[A64]] : !cir.vector<8 x !s64i>, [[SV]] : !cir.vector<8 x !s64i>)
+ // CIR: [[ASHR_A:%.*]] = cir.shift(right, [[SHL_A]] : !cir.vector<8 x !s64i>, [[SV]] : !cir.vector<8 x !s64i>)
+ // CIR: [[SHL_B:%.*]] = cir.shift(left, [[B64]] : !cir.vector<8 x !s64i>, [[SV]] : !cir.vector<8 x !s64i>)
+ // CIR: [[ASHR_B:%.*]] = cir.shift(right, [[SHL_B]] : !cir.vector<8 x !s64i>, [[SV]] : !cir.vector<8 x !s64i>)
+ // CIR: [[MUL:%.*]] = cir.binop(mul, [[ASHR_A]], [[ASHR_B]])
+
+ // LLVM-LABEL: _mm512_mul_epi32
+ // LLVM: shl <8 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: ashr <8 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: shl <8 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: ashr <8 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: mul <8 x i64> %{{.*}}, %{{.*}}
+
+ // OGCG-LABEL: _mm512_mul_epi32
+ // OGCG: shl <8 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: ashr <8 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: shl <8 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: ashr <8 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: mul <8 x i64> %{{.*}}, %{{.*}}
+
+ return _mm512_mul_epi32(__A, __B);
+}
+
+
+__m512i test_mm512_mul_epu32(__m512i __A, __m512i __B) {
+ // CIR-LABEL: _mm512_mul_epu32
+ // CIR: [[MASK_SCALAR:%.*]] = cir.const #cir.int<4294967295> : !s64i
+ // CIR: [[MASK_VEC:%.*]] = cir.vec.splat [[MASK_SCALAR]] : !s64i, !cir.vector<8 x !s64i>
+ // CIR: [[BC_A:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<8 x !s64i>
+ // CIR: [[BC_B:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<8 x !s64i>
+ // CIR: [[AND_A:%.*]] = cir.binop(and, [[BC_A]], [[MASK_VEC]])
+ // CIR: [[AND_B:%.*]] = cir.binop(and, [[BC_B]], [[MASK_VEC]])
+ // CIR: [[MUL:%.*]] = cir.binop(mul, [[AND_A]], [[AND_B]])
+
+ // LLVM-LABEL: _mm512_mul_epu32
+ // LLVM: and <8 x i64> %{{.*}}, splat (i64 4294967295)
+ // LLVM: and <8 x i64> %{{.*}}, splat (i64 4294967295)
+ // LLVM: mul <8 x i64> %{{.*}}, %{{.*}}
+
+ // OGCG-LABEL: _mm512_mul_epu32
+ // OGCG: and <8 x i64> %{{.*}}, splat (i64 4294967295)
+ // OGCG: and <8 x i64> %{{.*}}, splat (i64 4294967295)
+ // OGCG: mul <8 x i64> %{{.*}}, %{{.*}}
+
+ return _mm512_mul_epu32(__A, __B);
+}
diff --git a/clang/test/CIR/CodeGenBuiltins/X86/sse2-builtins.c b/clang/test/CIR/CodeGenBuiltins/X86/sse2-builtins.c
index f5e07cdc28ccd..9c87d0fe91327 100644
--- a/clang/test/CIR/CodeGenBuiltins/X86/sse2-builtins.c
+++ b/clang/test/CIR/CodeGenBuiltins/X86/sse2-builtins.c
@@ -108,3 +108,26 @@ void test_mm_pause(void) {
// LLVM: call void @llvm.x86.sse2.pause()
// OGCG: call void @llvm.x86.sse2.pause()
}
+
+__m128i test_mm_mul_epu32(__m128i A, __m128i B) {
+// CIR-LABEL: _mm_mul_epu32
+// CIR: [[MASK_SCALAR:%.*]] = cir.const #cir.int<4294967295> : !s64i
+// CIR: [[MASK_VEC:%.*]] = cir.vec.splat [[MASK_SCALAR]] : !s64i, !cir.vector<2 x !s64i>
+// CIR: [[BC_A:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<2 x !s64i>
+// CIR: [[BC_B:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<2 x !s64i>
+// CIR: [[AND_A:%.*]] = cir.binop(and, [[BC_A]], [[MASK_VEC]])
+// CIR: [[AND_B:%.*]] = cir.binop(and, [[BC_B]], [[MASK_VEC]])
+// CIR: [[MUL:%.*]] = cir.binop(mul, [[AND_A]], [[AND_B]])
+
+// LLVM-LABEL: _mm_mul_epu32
+// LLVM: and <2 x i64> %{{.*}}, splat (i64 4294967295)
+// LLVM: and <2 x i64> %{{.*}}, splat (i64 4294967295)
+// LLVM: mul <2 x i64> %{{.*}}, %{{.*}}
+
+// OGCG-LABEL: _mm_mul_epu32
+// OGCG: and <2 x i64> %{{.*}}, splat (i64 4294967295)
+// OGCG: and <2 x i64> %{{.*}}, splat (i64 4294967295)
+// OGCG: mul <2 x i64> %{{.*}}, %{{.*}}
+
+return _mm_mul_epu32(A, B);
+}
diff --git a/clang/test/CIR/CodeGenBuiltins/X86/sse41-builtins.c b/clang/test/CIR/CodeGenBuiltins/X86/sse41-builtins.c
new file mode 100644
index 0000000000000..472ce3e1e0df0
--- /dev/null
+++ b/clang/test/CIR/CodeGenBuiltins/X86/sse41-builtins.c
@@ -0,0 +1,45 @@
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse4.1 -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse4.1 -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s
+
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse4.1 -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +sse4.1 -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s
+
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-apple-darwin -target-feature +sse4.1 -emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s --check-prefixes=OGCG
+// RUN: %clang_cc1 -x c -flax-vector-conversions=none -fms-extensions -fms-compatibility -ffreestanding %s -triple=x86_64-windows-msvc -target-feature +sse4.1 -emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s --check-prefixes=OGCG
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-apple-darwin -target-feature +sse4.1 -emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s --check-prefixes=OGCG
+// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -fms-extensions -fms-compatibility -ffreestanding %s -triple=x86_64-windows-msvc -target-feature +sse4.1 -emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s --check-prefixes=OGCG
+
+#include <immintrin.h>
+
+__m128i test_mm_mul_epi32(__m128i x, __m128i y) {
+ // CIR-LABEL: _mm_mul_epi32
+ // CIR: [[SC:%.*]] = cir.const #cir.int<32> : !s64i
+ // CIR: [[SV:%.*]] = cir.vec.splat [[SC]] : !s64i, !cir.vector<2 x !s64i>
+ // CIR: [[A64:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<2 x !s64i>
+ // CIR: [[B64:%.*]] = cir.cast bitcast %{{.*}} : {{.*}} -> !cir.vector<2 x !s64i>
+ // CIR: [[SHL_A:%.*]] = cir.shift(left, [[A64]] : !cir.vector<2 x !s64i>, [[SV]] : !cir.vector<2 x !s64i>)
+ // CIR: [[ASHR_A:%.*]] = cir.shift(right, [[SHL_A]] : !cir.vector<2 x !s64i>, [[SV]] : !cir.vector<2 x !s64i>)
+ // CIR: [[SHL_B:%.*]] = cir.shift(left, [[B64]] : !cir.vector<2 x !s64i>, [[SV]] : !cir.vector<2 x !s64i>)
+ // CIR: [[ASHR_B:%.*]] = cir.shift(right, [[SHL_B]] : !cir.vector<2 x !s64i>, [[SV]] : !cir.vector<2 x !s64i>)
+ // CIR: [[MUL:%.*]] = cir.binop(mul, [[ASHR_A]], [[ASHR_B]])
+
+ // LLVM-LABEL: _mm_mul_epi32
+ // LLVM: shl <2 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: ashr <2 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: shl <2 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: ashr <2 x i64> %{{.*}}, splat (i64 32)
+ // LLVM: mul <2 x i64> %{{.*}}, %{{.*}}
+
+ // OGCG-LABEL: _mm_mul_epi32
+ // OGCG: shl <2 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: ashr <2 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: shl <2 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: ashr <2 x i64> %{{.*}}, splat (i64 32)
+ // OGCG: mul <2 x i64> %{{.*}}, %{{.*}}
+
+ return _mm_mul_epi32(x, y);
+}
|
GeneraluseAI
force-pushed
the
cir_x86_avx512_pmul_builtin_lowering
branch
2 times, most recently
from
a24ba5b to
41fa386
Compare
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
GeneraluseAI
force-pushed
the
cir_x86_avx512_pmul_builtin_lowering
branch
from
41fa386 to
cdaefb1
Compare
This patch adds CIR codegen support for X86 pmuldq and pmuludq operations, covering the signed and unsigned variants across all supported vector widths. The builtins now lower to the expected CIR representation matching the semantics of the corresponding LLVM intrinsics.
GeneraluseAI
force-pushed
the
cir_x86_avx512_pmul_builtin_lowering
branch
from
cdaefb1 to
14da804
Compare
andykaylor
reviewed
Dec 1, 2025
| ops[0].getType()); | ||
| } | ||
|
|
||
| static mlir::Value emitX86Muldq(CIRGenFunction &cgf, const CallExpr *expr, |
Contributor
There was a problem hiding this comment.
Suggested change
| static mlir::Value emitX86Muldq(CIRGenFunction &cgf, const CallExpr *expr, | |
| static mlir::Value emitX86Muldq(CIRGenBuilderTy &builder, mlir::Location loc, |
| // type | ||
| if (isSigned) { | ||
| ty = | ||
| cir::VectorType::get(builder.getSInt64Ty(), tyPrimitiveSizeInBits / 64); |
Contributor
There was a problem hiding this comment.
This line appears in both clauses of the if-else expression. It can be hoisted out as it is in classic codegen. Please also retain the comment that is used there.
Comment on lines
+138
to
+139
| lhs = builder.createBitcast(loc, ops[0], ty); | ||
| rhs = builder.createBitcast(loc, ops[1], ty); |
Contributor
There was a problem hiding this comment.
These also should be hoisted out of the if-else.
| mlir::Type ty = ops[0].getType(); | ||
| unsigned tyPrimitiveSizeInBits = | ||
| cgf.cgm.getDataLayout().getTypeSizeInBits(ty); | ||
| mlir::Value lhs, rhs; |
Contributor
There was a problem hiding this comment.
Suggested change
| mlir::Value lhs, rhs; | |
| mlir::Value lhs = builder.createBitcast(loc, ops[0], ty); | |
| mlir::Value rhs = builder.createBitcast(loc, ops[1], ty); |
| mlir::Location loc = cgf.getLoc(expr->getExprLoc()); | ||
| mlir::Type ty = ops[0].getType(); | ||
| unsigned tyPrimitiveSizeInBits = | ||
| cgf.cgm.getDataLayout().getTypeSizeInBits(ty); |
Contributor
There was a problem hiding this comment.
Suggested change
| cgf.cgm.getDataLayout().getTypeSizeInBits(ty); | |
| unsigned tyPrimitiveSizeInBits = | |
| cgf.cgm.getDataLayout().getTypeSizeInBits(ops[0].getType()); | |
| // Arguments have a vXi32 type so cast to vXi64. | |
| mlir::Type ty = cir::VectorType::get(builder.getSInt64Ty(), tyPrimitiveSizeInBits / 64); |
| mlir::Value shiftSplatValue = shiftSplatVecOp.getResult(); | ||
| lhs = builder.createBitcast(loc, ops[0], ty); | ||
| rhs = builder.createBitcast(loc, ops[1], ty); | ||
| lhs = builder.createShift(loc, lhs, shiftSplatValue, true); |
Contributor
There was a problem hiding this comment.
I'd add a comment here explaining the purpose of these shifts. The comment above about arithmetic shift being used automatically really belongs here, but the more relevant information is that the is filling the upper bits with of the vector with the sign-bit of the 32-bit value in each lane.
| cir::VecSplatOp::create(builder, loc, ty, maskScalar.getResult()); | ||
| lhs = builder.createBitcast(loc, ops[0], ty); | ||
| rhs = builder.createBitcast(loc, ops[1], ty); | ||
| lhs = builder.createAnd(loc, lhs, mask); |
Contributor
There was a problem hiding this comment.
Please keep the comment "// Clear the upper bits". It may seem obvious, but it's a useful hint as to the logic of this function.
| case X86::BI__builtin_ia32_pmuludq128: | ||
| case X86::BI__builtin_ia32_pmuludq256: | ||
| case X86::BI__builtin_ia32_pmuludq512: | ||
| return emitX86Muldq(*this, expr, /*IsSigned*/ false, ops); |
Contributor
There was a problem hiding this comment.
Suggested change
| return emitX86Muldq(*this, expr, /*IsSigned*/ false, ops); | |
| return emitX86Muldq(builder, getLoc(expr->getExprLoc()), /*isSigned=*/false, ops); |
This suggestion reflects the change in signature suggested above, but it also aligns the isSigned= comment with clang conventions.
| case X86::BI__builtin_ia32_pmuldq128: | ||
| case X86::BI__builtin_ia32_pmuldq256: | ||
| case X86::BI__builtin_ia32_pmuldq512: | ||
| return emitX86Muldq(*this, expr, /*IsSigned*/ true, ops); |
Contributor
There was a problem hiding this comment.
Suggested change
| return emitX86Muldq(*this, expr, /*IsSigned*/ true, ops); | |
| return emitX86Muldq(builder, getLoc(expr->getExprLoc), /*isSigned=*/true, ops); |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
part of #167765
This patch adds CIR codegen support for X86 pmuldq and pmuludq operations, covering the signed and unsigned variants across all supported vector widths. The builtins now lower to the expected CIR representation matching the semantics of the corresponding LLVM intrinsics.