-
Notifications
You must be signed in to change notification settings - Fork 367
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a new x8-packq
microkernel that packs and per-row dynamically quantizes fp32
to qp8
.
#6424
Conversation
const size_t k_block_len = kr / sr; | ||
|
||
for (size_t row_idx = 0; row_idx < num_rows; ++row_idx) { | ||
float max0 = -FLT_MAX; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
use params or initialize to the first value, like rmax
// Scale the values. | ||
int32_t v0_s32 = (int32_t)(round(src0_0 * scale0)); | ||
|
||
v0_s32 = v0_s32 + nudged_zero_point0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we make this adjustable for vnni to support +128?
const float src0_0 = *(src_ptr + k_idx + k_block_idx); | ||
|
||
// Scale the values. | ||
int32_t v0_s32 = (int32_t)(round(src0_0 * scale0)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
roundf?
v0_s32 = v0_s32 + nudged_zero_point0; | ||
v0_s32 = fmaxf(v0_s32, INT8_MIN); | ||
v0_s32 = fminf(v0_s32, INT8_MAX); | ||
*((int8_t*)(dst_ptr)) = (int8_t)v0_s32; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can this be made to do 8 values from src0 and 8 values from src1 for i8mm?
size_t m_idx_start, // Starting index in `lhs_packed`. | ||
const float* XNN_RESTRICT lhs, // Left-hand operator to pack. | ||
size_t lhs_stride, // Stride in bytes between the rows of `lhs`. | ||
void* XNN_RESTRICT lhs_packed // The quantized and packed output. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can lhs output be standard gemm qd8 int8 with stride and quantization params as (row sum) a different pointer?
ba3ec60
to
9d5e952
Compare
3d53157
to
46834e5
Compare
…uantizes `fp32` to `qp8`. The microkernels themselves are just wrappers for the corresponding KleidiAI kernels. PiperOrigin-RevId: 643324022
46834e5
to
bffaa6f
Compare
Add a new
x8-packq
microkernel that packs and per-row dynamically quantizesfp32
toqp8
.The microkernels themselves are just wrappers for the corresponding KleidiAI kernels.