feat(gemm_a8w8_blockscale_bpreshuffle): route decode-M DeepSeek shapes to triton

[valarLip]

Summary

Add a libtype == "triton" dispatch branch to gemm_a8w8_blockscale_bpreshuffle
so the tuned CSV can route a given (M, N, K) to the triton preshuffle kernel,
and route the decode-M DeepSeek FP8 blockscale attention projections to triton
where it is measurably faster and not less accurate than the current asm/ck
kernels on gfx950 (MI35x).

What changed

• Dispatch (aiter/ops/gemm_op_a8w8.py): new libtype == "triton" case.

  • Handled before the Y output buffer is allocated (triton allocates its own
    output) — no wasted m×n alloc on the triton path.
  • backend is taken from the CSV kernelName only when it is triton/gluon;
    anything else falls back to auto-detect instead of tripping triton's internal
    assert backend in ('triton','gluon').

• Tuned configs (aiter/ops/triton/configs/gemm/*.json): per-shape gfx950
  triton configs for N=2048/16384/65536/8192, K∈{7168,1536}. The pre-existing
  N=2048 file is merged (kept its M_LEQ_256/any large-M buckets).

• Routing CSVs: libtype=triton only on the M segments that win:

  shape (N,K)	routed M	note
  wqkv_a (2048,7168)	1,2,8,16,32,80	KSPLIT=4 deep-K split
  wq_b tp1 (65536,1536)	1,2,4,8
  wq_b tp4 (16384,1536)	1,4,8,16	~1.4×, was untuned ck
  wq_b tp8 (8192,1536)	2,80,96,112,144	M=2 1.74× (was asm+memset)

  Large-M / compute-bound and the remaining tp8 M stay on asm/ck. The N=8192 rows
  live in the shared dsv3 file (V3/V4 same shape), so V3 benefits too.

How the M segments were chosen

Every M row of these shapes in the CSVs was measured with aiter's run_perftest
(same process, asm/ck vs triton) against a torch fp32 reference. A row is routed
to triton only if triton is valid (err < 0.05), ≥3% faster in steady state,
and not less accurate than asm/ck. No row was faster-but-less-accurate — in
the flipped rows triton's err is 0 while asm/ck carry fp8 rounding (~0.003–0.01).

Verification

• Steady-state A/B (10 reps, warmup dropped): all flipped rows 1.09–1.74× faster
  (largest: 16384 ~1.4×, 8192 M=2 1.74×).
• op_tests/test_gemm_a8w8_blockscale.py re-run on the V4 shapes: unified API
  routes to triton for the flipped M, all err == 0, and beats raw asm on every
  measured point.

Test plan

• python op_tests/test_gemm_a8w8_blockscale.py -nk 2048,7168 8192,1536 16384,1536 65536,1536 -m 1 2 4 8 16 32 96 128 --ck_preshuffle 1
• Confirm dispatch routes as expected via AITER_LOG_TUNED_CONFIG=1.

🤖 Generated with Claude Code

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 4202 --add-label <label>

[bingxche]

Hi @valarLip , could you please take a look at this issue #4323? Thanks!