QS8 / QU8 HSwish Microkernels #8255
Conversation
swamipreksha
commented
Apr 13, 2025
swamipreksha
commented
- Implementations for various ISAs:
- x86 AVX2
- Scalar ISA
- Unit tests
- Implementations for various ISAs: - x86 AVX2 - Scalar ISA - Unit tests Signed-Off-by: Ravi Kumar Soni <ravi.kumar.soni@intel.com> Signed-off-by: Swami, Preksha <preksha.swami@intel.com>
swamipreksha
force-pushed
the
qs8_qu8_vhswish
branch
from
558d05d to
83b449d
Compare
|
@dsharlet Kindly review our hardswish implementation for qs8 and qu8 datatypes. |
|
This is a unary op, where we should be generating a LUT. I'm not sure that this implementation is going to be that much better. It's vectorized, but it needs quite a few instructions, and many of those instructions are 4x wider (floats) than the input/output data (int8/uint8). Can you please benchmark to see if this is the case? What does the following command report on master and with your branch? |
| __m256 va_sum = _mm256_add_ps(va_mul, vthree); | ||
| __m256 va_clamped = _mm256_min_ps(_mm256_max_ps(va_sum, vzero), vsix); | ||
| __m256 vacc = _mm256_mul_ps(_mm256_mul_ps(va_mul, vsixth), va_clamped); | ||
| vacc = _mm256_div_ps(vacc, voutput_scale); |
There was a problem hiding this comment.
can this be a reciprocal mul_ps instead?
| input += 8; | ||
| const __m128i vout_low = _mm256_castsi256_si128(vout); | ||
| const __m128i vout_high = _mm256_extracti128_si256(vout, 1); | ||
| const __m128i vout_packed = _mm_packs_epi32(vout_low, vout_high); |
There was a problem hiding this comment.
instead of 2 packs can you do a cvtepi32_epi8?
fbarchard
left a comment
fbarchard
left a comment
There was a problem hiding this comment.
Thanks for the PR! There is an f32-vhswish/avx.c.in that is a bit simplier than this?
is the math the same?
and yes, to dillon's suggestion of a lut. An example of that is softmax for 8 bit.
As it is, the cost of converting bytes to float and back is a bottleneck, and only 8 values can be processed at a time.
3 participants
