v0.5.11

What's Changed

Added

• int8_matmul, an int8 x int8 matmul with per-row (or per-row per K group) activation scales and
  per-channel weight scales on M5 and later GPUs, about 1.7x faster than a Q4_K quantized_matmul
  at 38010 rows on M5 Max with the activation quantize included; lossy. int8_quantize_rows makes
  its codes and scales, for activations and weights.

Changed

• The rows LoRA epilogue of quantized_matmul, quantized_matmul_qmv_bias and gather_qmv_kq adds
  less time at decode widths. In benchmarks/bench_lora_epilogue.py on M3 Max (K 512, N 2048, 8
  gathered rows, rank 1) it adds about 4 us over the base op instead of about 68 us. Rank 1, and
  small ranks at K up to 2048, run one epilogue dispatch; KQ_LORA_SPLIT=1 forces the split kernels.

Full Changelog: v0.5.10...v0.5.11