Skip to content

AITER v0.1.27

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 26 Oct 08:41
v0.1.27
778d74d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

AITER v0.1.27 - weekly release

Scheduled release from release/v0.1.27.

Diff base: v0.1.26

Wheels

Prebuilt manylinux_2_28 wheels, GPU_ARCHS=gfx942;gfx950:

ROCm Python Wheel
ROCm 7.0 cp310 amd_aiter-0.1.27+rocm7.0.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.0 cp312 amd_aiter-0.1.27+rocm7.0.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.1 cp310 amd_aiter-0.1.27+rocm7.1.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.1 cp312 amd_aiter-0.1.27+rocm7.1.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.2 cp310 amd_aiter-0.1.27+rocm7.2.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.2 cp312 amd_aiter-0.1.27+rocm7.2.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
What's Changed

Changes

  • [CI] CI: auto-update split test times by @aiter-gh-app[bot] in #6657
  • [FlyDSL] Add packed D256 PA decode pipeline for gfx950 by @fsx950223 in #6655
  • [Triton/Gluon] benchmark + config + tuning for fused_qk_concat on gfx950 by @nidal567 in #6287
  • [FlyDSL] Fuse add-RMSNorm and MXFP4 quant into INT4 QuickReduce by @mjkvaak-amd in #6369
  • [HIP] Support Gemma (1 + w) weights in fused AR+RMSNorm+MXFP4 quant by @mjkvaak-amd in #6000
  • [Triton/Gluon] [Config] Add gfx1200 and gfx1201 Triton MHA configs by @0xDELUXA in #6374
  • [Triton/Gluon] [JIT] [Bugfix] add support for MI300A and fix XCD count by @aurianer in #6327
  • [HIP] [Bugfix] scaled_silu_and_mul: device-side scale, fp8 saturation, row bounds by @AndreasKaratzas in #6641
  • [JIT] Round custom all-reduce BF16 stores to nearest even by @AndreasKaratzas in #6640
  • [Triton/Gluon] : Remove bad mla_decode.py op by @rahulbatra85 in #6633
  • [Triton/Gluon] [Bugfix] Fix Triton topk for rows containing -inf by @truong-v in #6095
  • [Triton/Gluon] Add fused_indexed_gated_residual for diffusion transformer blocks by @Shan2L in #6416
  • [FlyDSL] Use typed APIs in MoE routing and MX quantization by @coderfeli in #6379
  • [HIP] [FlyDSL] conv3d: kW and kH x kW LDS reuse of the im2col input on gfx950 by @huizzhan in #6421
  • fix: zero-pad XQ to padded weight K for all a8w8 bpreshuffle backends by @yuychang in #6456
  • [FlyDSL] [MegaMoE] fix(dispatch): source-indexed payload for every rank at the indexed MTPR class by @Yaowu-Xiong in #6556
  • [FlyDSL] Flydsl tdm new interface by @xudoyuan in #6252
  • [CK] [FlyDSL] Optimize MI308 MoE Down kernels and add BF16 support (2/3) by @luocheng25 in #5932
  • [FlyDSL] Revert " Flydsl tdm new interface" by @coderfeli in #6716
  • [ASM] [HIP] [FlyDSL] Optimize gfx1250 persistent MLA decode, add BF16 and TP4/TP2 MTP by @xiangM99 in #6563
  • [FlyDSL] [JIT] mega_moe_tp: unified path and big-M optimizations by @binding7012 in #6514
  • [Config] Tune Fused MoE for DSR1 FP4 TP8 by @farlukas in #6544
  • [Triton/Gluon] Add x_EQ_x Config Matcher to Unified Attention by @farlukas in #6161
  • [Triton/Gluon] tune fused_bmm_rope_kv_cache on gfx950, add bench & configs (gfx950/1250) by @nidal567 in #6501
  • [Triton/Gluon] [Bugfix] Fix gfx1250 gemm_a16w16 dropping output rows past 2**31 bytes by @yichiche in #6629
  • [Triton/Gluon] [Config] gfx950 a16w16 N=2880 K=4096: retune M_LEQ_4 to avoid Triton 3.8 VGPR spill (gpt-oss-20b o_proj decode) by @afriedri in #6680
  • [Triton/Gluon] [CI] Fix Flash Attention CI failures by @micmelesse in #5687
  • [Triton/Gluon] : Make use of parser arg consistent in benchmarks by @rahulbatra85 in #6347
  • [Config] [DSV4.1Flash] Fuse the moe stage-2 input quant into the stage-1 epilogue by @heachary in #6611
  • [DSV4.1Flash] Bound the a8w4 MoE activation quant to the live token count by @heachary in #6610
  • [Triton/Gluon] [gfx1250] fp8_mqa_logits: fix epilogue store masks for cu_start != 0 by @lijinpei-amd in #4687
  • [Triton/Gluon] Read (B, next_n) context lengths in the paged MQA logits kernels by @amd-sriram in #6451
  • [Triton/Gluon] gluon_fused_qk_norm for 1250 by @Boss2002n in #6634
  • [FlyDSL] Simplify kernel APIs and remove redundant wrappers by @coderfeli in #6380
  • [FlyDSL] [CI] [Feature] Enable gfx1250 paged attention decode with FP8 and BF16 by @fsx950223 in #6462
  • [ASM] Fix gfx950 GQA16 MTP softmax initialization by @fangche123 in #6606
  • [HIP] [JIT] Enable gfx1250 QuickReduce for TP2 and TP4 by @hubertlu-tw in #5965
  • [HIP] fea: support max tp64 on ag by @TennyWang1223 in #6570
  • [Config] Tune gfx1250 MiniMax-M3 PTPC GEMMs by @XiaobingSuper in #6717
  • [HIP] [Bugfix] paged_attention_ragged: scale FP8 softmax probabilities into the e4m3 normal range by @rbrugaro-amd in #5642
  • [ASM] mla_decode_fwd: wire is_causal through Python and C++ dispatch by @alexioslyrakis-amd in #3993
  • [HIP] [Perf] scaled_silu_and_mul: split short batches across workgroups by @AndreasKaratzas in #6656
  • [CI] Relax Standard Test Results reporting by @gyohuangxin in #6364
  • [ASM] [HIP] [CI] [gfx1250] add 192x256 no-ps & fix dispatch for A4W4 by @dbyoung18 in #6607
  • [FlyDSL] [gfx950] hc_mix kernel for Qwen3.8 hyper-connections by @amd-nprotaso in #5993
  • [Triton/Gluon] [Bugfix] gfx1250 Gluon a8w8 blockscale: NUM_BUFFERS config key, clamped to the split by @Boss2002n in #6679
  • [Triton/Gluon] Fused GDN decode kernel for any value/key head ratio (Qwen3.5/3.8) by @eky-amd in #6488
  • [HIP] [Bugfix] Fix topk_sigmoid pybind conversion by @rjrock in #6592
  • [Triton/Gluon] [Config] Tune gfx1250 Gluon GEMMs for Kimi-K2 Thinking by @Boss2002n in #6725
  • [Triton/Gluon] [Config] Tune gfx1250 Gluon MXFP4 GEMMs for Llama3 70B by @Boss2002n in #6653
  • [Triton/Gluon] [Config] Tune gfx1250 Gluon a16w16 GEMMs for Llama4 Maverick by @Boss2002n in #6650
  • [Triton/Gluon] [Config] Tune gfx1250 Gluon a16w16 GEMMs for Qwen3-235B-A22B by @Boss2002n in #6649
  • [Triton/Gluon] Fix NaN dq in Triton MHA backward when every logit of a row is very n… by @Abdennacer-Badaoui in #6428
  • [Triton/Gluon] [Test] Stop test_extend_attention from changing the default dtype of later tests by @Boss2002n in #6747
  • [Triton/Gluon] [Bugfix] Scale gfx942 Sage probabilities before FP8 conversion by @akshatvishu in #6138
  • [Triton/Gluon] [Config] gemm_a16w16: add gfx1101 (RDNA3) configs by @Ragua1 in #5961
  • [Triton/Gluon] Add fused gated RMSNorm + MXFP4 quant by @mjkvaak-amd in #6038
  • [Triton/Gluon] [Config] Add gfx1151 gemm_a16w16 DEFAULT.json by @klin2024 in #6671
  • [Triton/Gluon] gfx942 a16w4 MoE: tune gpt-oss decode tiles for Triton 3.8 by @afriedri in #6688
  • [Triton/Gluon] bench_pa_decode: report the real KV bandwidth and time with CUDA graphs by @NimitPtl in #6690
  • [Triton/Gluon] [Config] Tune gfx1250 Gluon GEMMs for DeepSeek-R1 by @Boss2002n in #6687
  • [Config] Add Muse Spark Open MXFP4 FMoE tuning for gfx950 by @hongxiayang in #6593
  • [Triton/Gluon] [Tuning] Sweep NUM_BUFFERS 5 and 7, and kernel_type for gfx1250 persistent a16w16 and blockscale preshuffle by @Boss2002n in #6744
  • [Triton/Gluon] [Config] Tune gfx1250 Gluon MXFP4 GEMMs for Llama3 8B by @Boss2002n in #6652
  • [Triton/Gluon] Tune for DSR1 Input Projection by @farlukas in #6726
  • [HIP] Revert " [Bugfix] Fix topk_sigmoid pybind conversion" by @valarLip in #6811
  • [Config] [Perf] Add gfx950 a8w8 bpreshuffle and BF16 GEMM tunings for Qwen3.8 Flash Next by @amd-cgilli in #6769
  • [HIP] [JIT] [Bugfix] Import aiter in a process with no visible GPU by @Arist12 in #6738
  • [Triton/Gluon] [Config] gluon a16w16 configs for dsv4.1 moe router and kv compressor shapes by @ahmed-bsod in #6695
  • [Bugfix] Bind load_hip_runtime to the HIP runtime already loaded by @omirosh in #6398
  • [Triton/Gluon] [Config] Tune gfx1250 Gluon MXFP4 GEMMs for Llama3 405B by @Boss2002n in #6654
  • [Config] [MoE] [gfx950] Qwen3-235B FP8 EP: drop the token=1,2 tuned fmoe rows that lose to the CK 2-stage default by @afriedri in #6681
  • [Config] Add v41 tp2 moe config for shared expert fusion on 8k/16k input by @1am9trash in #6622
  • [CK] [Bugfix] Write grouped MoE tuning results back in place by @jhinpan in #6395
  • [HIP] [MoE] Qwen3.8 TP=8 MXFP4 MoE aux kernel codegen by @johannes-graner in #6468
  • [Bugfix] Fix missing extension module names in pretune builds by @yuzhouo7 in #5991
  • [Config] Qwen3.8-Flash-Next FP8 fused-MoE tuned configs for gfx942 (MI325X, 304 CUs) by @tc3oliver in #6693
  • [ASM] fix(mla): disable spurious [-64,64] output clamp & preserve FP32 deno… by @steamedMantou in #6750
  • [Triton/Gluon] [Perf] Tune gfx950 GEMM configs for N=K=2048 by @heslami in #6061
  • [ASM] [HIP] Fix wrong results in gfx942 fp8 persistent / blockscale fused-MoE dispatch by @AndreasKaratzas in #6639
  • [Triton/Gluon] Fix int32 overflow in _gluon_deepgemm_fp8_paged_mqa_logits_preshuffle_varctx K/scale load offset by @wufann in #5320
  • [Triton/Gluon] Add fused_gemm_a16w16_mul_add for the DiT gated residual by @klin2024 in #6674
  • [ASM] Restore gfx950 PA 2TG vector loads by @fangche123 in #6826
  • [Triton/Gluon] fix(gfx950): add compat shim for get_scaled_upcast_fp4_scale_layout on upstream Triton by @srinivamd in #6773
  • [Triton/Gluon] Flash Attention PR followup by @micmelesse in #6737
  • [Triton/Gluon] [Bugfix] Compute Triton split-K partial offsets in int64 by @siliangchen-amd in #6758
  • [Triton/Gluon] [GFX12] Gluon a8w8 Sparse MLA by @k50112113 in #6116
  • [Triton/Gluon] [HIP] [FlyDSL] fix(topk): graph-safe end=None; tgemm.mm(out=); one amax floor by @valarLip in #6833
  • [Triton/Gluon] feat(triton): add gemm_a16w4 int4 weight-only GEMM (W4A16) for GPTQ/AWQ checkpoints by @TwelveNights in #6778
  • [Triton/Gluon] fix mha UT by @Boss2002n in #6796
  • [Triton/Gluon] fix UT leaks by @Boss2002n in #6795
  • [Triton/Gluon] fix UT leaks by @Boss2002n in #6797
  • [Triton/Gluon] [Bugfix] Mask KV block padding lanes out of the softmax in Triton paged_attention_decode by @truong-v in #6325
  • [OPUS] [Bugfix] gfx1250 RMSNorm: write row tails when N % 8 != 0 and fence the opus block reduction by @jhinpan in #6348
  • [Triton/Gluon] Cleanup test_pa_decode_gluon by @Boss2002n in #6809
  • [Triton/Gluon] [Config] gfx950: tune gemm_a16w16 configs for DeepSeek-V4 GEMMs (K=7168) by @k50112113 in #6601
  • [Triton/Gluon] Fix unified_attention gfx950 wrong output when packed KV cache is 2–4 GiB by @zhentaocc in #6661
  • [Triton/Gluon] Support BF16 state storage in gfx950 KDA by @YukioZzz in #6672
  • [Triton/Gluon] [Config] [Perf] Add gfx942 a8w8 blockscale Triton configs for Qwen3 shapes by @siliangchen-amd in #6757
  • [Triton/Gluon] [FlyDSL] fix cuda and dtype assignments in triton UTs by @Boss2002n in #6783
  • [Triton/Gluon] fix cuda assignments: review follow-up by @Boss2002n in #6846
  • [FlyDSL] [gfx950] Causal conv1d prefill and GDN decode/verify kernels for Qwen3.5/Qwen3.8 by @amd-nprotaso in #5992
  • [FlyDSL] Add fused QSA sparse decode for gfx950 by @amd-nprotaso in #6670
  • [JIT] [hipb] Support scale mode arguments in hipb_mm by @rkayaith in #6582
  • [ASM] Check model_dim for gfx950 fp8 blockscale fused-MoE 1-stage kernels by @AndreasKaratzas in #6691
  • [JIT] [FMHA] Round CK FMHA bf16 conversions to nearest instead of truncating by @AndreasKaratzas in #6748
  • [FlyDSL] [gfx1250] Fix the dropped K tail and the bias column offset in the grouped MoE TDM GEMM by @lalala-sh in #6663
  • [HIP] [Fix] mla_decode_stage1_asm_fwd: reject MLA rows that are not 576 wide by @jin-amd in #6198
  • [HIP] [JIT] [Feature] MLA decode: add get_mla_decode_shape_support, backed by shared C++ head-fold/backend rules by @okorzh-amd in #6107
  • [HIP] Opt biased grouped topk gfx1250 by @yzhou103 in #6257
  • [HIP] fix: gfx1250 LL ar by @TennyWang1223 in #6756
  • [ASM] [HIP] Align persistent workgroup count for fmoe kernels that remap across XCDs by @AndreasKaratzas in #6694
  • [HIP] [JIT] [Feature] Add fused Mega-MHC for DeepSeek V4.1 by @junhaha666 in #6520
  • Stop reporting a fast-kernel failure that never ran in test_topk_row_prefill by @yzhou103 in #6479
  • [CI] Enable MiniMax-M3 MXFP8 2P1D TP8 ATOM DI CI use cases by @avininjamay8 in #5634
  • [HIP] [Bugfix] Fix topk_sigmoid pybind conversion by @rjrock in #6834
  • [HIP] [CI] fix(cpp_itfs/pa): make the C++ paged_attention_ragged entry point usable, and test it by @jiejingzhangamd in #4486
  • [Config] Add gfx950 a8w8 blockscale bpreshuffle GEMM tunings for Qwen3-14B-FP8 by @lishuoshuo-amd in #6862
  • [CI] Increase MI35X Triton test timeout budget by @gyohuangxin in #6878
  • [Triton/Gluon] [Config] [Perf] Tune gfx950 BF16 batched GEMM for Kimi-K2.6 by @lucas-santos-amd in #6780
  • [Triton/Gluon] [Config] [gfx1100] Add RMSNorm large-M/small-N config by @TwelveNights in #6772
  • [Triton/Gluon] Add Triton A8W8 Per-token Preshuffled GEMM by @farlukas in #6850
  • [Triton/Gluon] [Config] [Perf] Retune gfx950 preshuffled AFP4WFP4 decode buckets for Llama-3.3-70B MXFP4 (TP8/TP4) by @afriedri in #6741
  • [Triton/Gluon] [Config] add DSV4 Flash MOE tunned config by @k50112113 in #6890
  • [Triton/Gluon] [gfx1250] unified_attention: fast split-KV reduce at small batch by @vgokhale in #6888
  • [Triton/Gluon] : Add common functions for rotating buffers and do_bench_aiter_triton. by @rahulbatra85 in #6813
  • [FlyDSL] [gfx1250] Paged FP8 MQA logits with 8-token KV pages by @yhl-amd in #6863
  • [Triton/Gluon] Fix gfx950 AFP4WFP4 tiles for Qwen3.8-27B-MXFP4 TP1 by @a-sidorova in #6874
  • [HIP] [FlyDSL] [JIT] [gfx1250] Add MXFP4 a4w4 GEMM with tuned DSR1 configs by @aoli26 in #6621
  • [FlyDSL] fp8_mqa_logits: derive the split target from kernel occupancy by @akii96 in #5921
  • [FlyDSL] [gfx1250] Batched mxfp8_32 GEMM with 32x32 weight scales and K tails by @yhl-amd in #6864
  • [Config] [GFX1250] Kimi-K3 TP8 gemm tune by @zejunchen-zejun in #6763
  • [ASM] [HIP] [FlyDSL] feat: pad-row routing count, custom all_gatherv, MoE quant-sort race fix by @valarLip in #6916
  • [FlyDSL] [gfx950] Fix 8-wave a8w8 GEMM reading LDS before the last tile lands by @yangyuhuiling in #6368
  • [HIP] [GLM-5.3-Flash] MXFP4 fused-MoE tuned configs for MI355X by @amd-sriram in #6435
  • [HIP] [Bugfix] topk_gating kernels: trap when vec_size does not divide num_experts by @truong-v in #6323
  • [FlyDSL] [gfx1250] FMHA prefill: support v_hdim 256 by @vgokhale in #6897
  • [HIP] [CustomAR] Keep CUDA-graph buffer registration in step across ranks by @okorzh-amd in #6819
  • [CI] CI: run extended tests on main pushes by @gyohuangxin in #6931
  • [CI] Enable DeepSeek-V4-Pro 2P1D TP8 ATOM DI smoke cases by @avininjamay8 in #5784
  • [Config] tuning: qwen35 bf16 gemm tuning on gfx1250 by @mqhc2020 in #5999
  • Skip MegaMoE TP multigpu test by @gyohuangxin in #6933

New Contributors

Full Changelog: v0.1.26...v0.1.27

Contributors

rkayaith, vgokhale, and 77 other contributors
Assets 8
Loading