[SPIR-V] Avoid shifting by non-splat amounts #16913
Labels
codegen/spirv
SPIR-V code generation compiler backend
good first issue 🌱
Good for newcomers
help wanted
Extra attention is needed
performance ⚡
Performance/optimization related work across the compiler and runtime
On RDNA gpus, shift vector instructions are much faster when the shift amount is a splat constant. However, we seem to emit non-splat shifts for int4 matvec from LLama2:
Input
Compile command:
The dumps directory contains IR at the level of the spir-v dialect (
dumps/module__main_dispatch_0__main_dispatch_0_generic_4096x32x128_f16.spirv.mlir
), with the following shift ops:We should figure out which pass produces this instruction sequence (
--mlir-print-ir-after-all
) and change it to avoid shifting by non-splat amounts.The text was updated successfully, but these errors were encountered: