[kernels] hybrid_w4a16: fix bf16 asymmetric dequant slowdown on gfx1151

[mgehre-amd]

Improve bf16 perf in w4a16 triton kernel and add bfloat16 support to benchmark infra.

Native-bf16 asymmetric w4a16 models (RedHatAI/Qwen3-8B-quantized.w4a16, Orion-zhen/Qwen3-1.7B-AWQ, cyankiwi/Qwen3-VL-4B-Instruct-AWQ-4bit, ...) suffered a ~15% prefill TTFT regression on Strix Halo (gfx1151) vs forcing the same models to fp16, even when the kernel autotune config was retuned for bf16.

The bottleneck is the HAS_ZP=True branch of _triton_w4a16_skinny_fmt_kernel:

b_fp = (b.to(scales.dtype) - zp_raw[:, None]) * scales[:, None]

For bf16 this casts the full [BLOCK_N, BLOCK_K] int32 nibble tile to bf16 before subtracting the zero-point. On RDNA3.5 the compiler then emits an inner loop of:

v_cvt_f32_ubyte0 ; int -> fp32
v_cvt_f16_f32 ; explicit narrowing
v_sub_bf16 (slow) ; bf16 ALU goes via fp32
v_pk_mul_bf16

Mirroring the symmetric path -- subtract in INT first (zp values are 0..15, fits exactly in any int width), then cast once after the subtract -- lets LLVM fold the conversion into v_dot2_bf16_bf16, since bf16 is the top 16 bits of fp32 and v_cvt_f32_i32 implicitly produces a usable bf16 representation. The inner loop collapses to:

v_sub_nc_u32 ; int sub (cheap)
v_cvt_f32_i32 ; int -> fp32 (bf16 lives in high half, free)
v_dot2_bf16_bf16 ; fused MAC

Kernel microbench (M=128, gfx1151, Strix Halo, asymmetric Qwen3-8B prefill projection shapes, group_size=128):

QKV (24576x4096): 1519 -> 1271 us (-16.3%)
down ( 4096x12288): 815 -> 685 us (-16.0%)
gate_up(6144x4096): 371 -> 324 us (-12.7%)
o_proj( 4096x4096): 271 -> 234 us (-13.7%)

The fp16 branch is left untouched on purpose: switching fp16 to the int-subtract-first form does NOT collapse to a packed fused MAC (no v_dot2_f16_f16 on gfx1151, and fp16 is not a sub-bit-pattern of fp32 so the v_cvt_f16_f32 narrowing is unavoidable). A unified rewrite regressed fp16 by 6-9% per shape in microbench.

End-to-end on RedHatAI/Qwen3-8B-quantized.w4a16 (input_len=128 output_len=128 num_prompts=10 max_num_seqs=1, Strix Halo):

                          bf16        fp16
  Median TTFT  before  137.6 ms    118.8 ms
  Median TTFT  after   117   ms    119.5 ms  (bf16 -15.0%, fp16 noise)
  Median TPOT          24.01 ms    24.07 ms  (unchanged, decode path)

bf16 is now ~1.5% faster than fp16, so the dtype: float16 pin can be removed from this model in rocm-scripts/regression_config.yaml.

Correctness: tests/kernels/quantization/test_hybrid_w4a16_triton.py (20 cases covering sym/asym x fp16/bf16 x 5 shapes) all pass.

[mgehre-amd]

fyi @marcusr-amd for bf16 perf

[roberteg16]

Are we planning to follow this up with a ticket for the compiler team?

[mgehre-amd]

Are we planning to follow this up with a ticket for the compiler team?

I started to look into this, but my agent run into a dead end. Will need to backtrack and properly report this for further investigation.

[eble-amd]

In my opinion, the merge commit should be removed, this PR should be rebased, and the golden values should be remeasured by running the full set of tests. My concern is that the subsets of the tests that you and Robert added separately might interact and push the job toward intermittency.

[eble-amd]

Friendly reminder: The performance tests currently allow a ridiculously wide tolerance (±80%) for the hybrid_triton_w4a16 kernel.