Skip to content

AITER v0.1.29

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 09 Nov 10:52
v0.1.29
a0c4b69
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

AITER v0.1.29 - weekly release

Scheduled release from release/v0.1.29.

Diff base: v0.1.28

Wheels

Prebuilt manylinux_2_28 wheels, GPU_ARCHS=gfx942;gfx950:

ROCm Python Wheel
ROCm 7.2 cp310 amd_aiter-0.1.29+rocm7.2.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.2 cp312 amd_aiter-0.1.29+rocm7.2.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.14 cp310 amd_aiter-0.1.29+rocm7.14.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.14 cp312 amd_aiter-0.1.29+rocm7.14.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 10.0 cp310 amd_aiter-0.1.29+rocm10.0.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 10.0 cp312 amd_aiter-0.1.29+rocm10.0.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 10.1 cp310 amd_aiter-0.1.29+rocm10.1.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 10.1 cp312 amd_aiter-0.1.29+rocm10.1.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
What's Changed

Changes

  • [CI] CI: auto-update split test times by @aiter-gh-app[bot] in #7290
  • [HIP] [FlyDSL] perf(mega_moe): fuse the gfx1250 compact dispatch and GEMM1 into one stage-1 kernel by @GwilliamHu in #6564
  • [ASM] [HIP] [OPUS] Add gfx950 MLA v4.1 a8w4 prefill kernels for an NVFP4 prefix KV by @kaiyang-1 in #7124
  • [Perf] Avoid torch Stream construction across grouped MoE launches by @xytpai in #7219
  • [Bugfix] MLA decode: merge only the splits gfx950 bf16 gqa32 stage 1 writes by @batzor in #7182
  • [Config] [gfx1250] Tune MiniMax-M3 TP4 BF16, indexer-CP QKV and MoE token 1024 by @XiaobingSuper in #7209
  • [Triton/Gluon] Window-aware split-KV for sliding-window verify on gfx950 by @okorzh-amd in #6889
  • [FlyDSL] gfx1201 (RDNA4) FlyDSL BF16 attention optimizations & FP8 attention implementation by @pds-amd in #4188
  • [Triton/Gluon] Enable and tune pipelined unshuffled MLA decode on gfx1250 by @justinrosner in #6545
  • [Triton/Gluon] [Kernel] Add dual-layout H16 MXFP4 quantization by @DaiXindi-AMD in #6458
  • [Triton/Gluon] [Perf] fused_reduce_act_mul_and_mxfp4_quant: wider launch tile, add round_to_input_dtype and transpose_scale by @reger-men in #6024
  • [CI] Run the selected Triton tests on each merge to main by @Boss2002n in #7229
  • [FlyDSL] [gfx1250] mega_moe S1F: reachable stage-1 hook, plan precompile, tuned defaults (on by default) by @yhl-amd in #7315
  • [CI] Reuse FlyDSL AOT cache and reduce A8W8 GEMM test overhead by @yifehuan in #7316
  • [FlyDSL] Fix grouped GEMM C ABI overflow for large EP buffers by @coderfeli in #7299
  • [Triton/Gluon] [HIP] [OPUS] [AITERKER-209] Add gfx1250-strict architecture plumbing by @zufayu in #6767
  • [HIP] [FlyDSL] optimize dsv41 perf for decode shape by @solinzby1 in #7309
  • [HIP] [CK] [JIT] [Tuner] Rank accuracy-qualified kernels by package energy by @rpoornac in #6970
  • [FlyDSL] FMHA gfx1250: per-tensor fp8 descales for the a8w8 kernel by @ruanjm in #7214
  • [ASM] [HIP] [OPUS] Add gfx1250 MLA v4.1 a8w8 sparse prefill kernels by @kaiyang-1 in #7314
  • [Triton/Gluon] [Config] Tune gfx950 gemm_a8w8 for Llama 3.1 8B FP8 decode. by @amd-jrosas in #7347
  • mrope cache-quant test: covering default set, full matrix behind --full by @vorapolsiloai in #7318
  • [FlyDSL] pa_decode: peel the last KV tile so only it carries the V tail mask by @fsx950223 in #7197
  • [FlyDSL] Tune DSv4.1 EP8 A8W4 MegaMoE by @GwilliamHu in #7127
  • [CK] [Bugfix] FMoE tuner: use all sources for shapes FlyDSL v2 cannot tune by @tc3oliver in #6692
  • [OPUS] [JIT] Fall back to regular COs when A0 strict is unsupported by the HIP runtime by @demonsan in #7185
  • [Triton/Gluon] Allow a strided feature dim in conv_state for single-token conv1d update by @AndreasKaratzas in #6925
  • [JIT] fix(jit): keep detached-head advice local to third-party clone by @adenzhou1350 in #5938
  • [HIP] perf(custom_ar): gfx1250 LL all-gather for messages up to 4 MiB a rank by @XiaobingSuper in #7265
  • [FlyDSL] [CI] Conv3d: default to representative Wan correctness, move full matrix to perf mode by @huizzhan in #7331
  • [ASM] [HIP] gfx1250 mha bwd: hd64 kernels and hd128 border mask by @slippedJim in #7160
  • [Test] Use the device fp8 range in test_fused_kv_norm_rope_group_quant by @KHassanQazi in #6886
  • review-pr: add HK20 (submodule upstream check) and A4 (new-feature re… by @yzhou103 in #6865
  • [FlyDSL] Tdm new interface refactor: make_tdm_atom -> make_tiled_tdm_atom by @xudoyuan in #7213
  • [HIP] [Perf] Route gfx950 E=128..255 decode at M>=17 to multi-phase MoE sorting by @lishuoshuo-amd in #7134
  • [Test] Shrink the default shapes of test_topk_per_row.py by @lirui927 in #7323
  • [FlyDSL] [CI] Add FlyDSL fused intranode all-to-all for Ulysses sequence-parallel attention by @johannes-graner in #5815
  • [Triton/Gluon] fix: remove exact-M RMSNorm forward specialization by @ZhuoBohang-AMD in #7311
  • [FlyDSL] Merge gfx1250 PA decode partitions in the tile kernel and speed up dense decode by @zhiding512 in #7079
  • [Triton/Gluon] Add Split-K Variant of A8W8 BMM with Fused Reduce Quant by @farlukas in #6727
  • [Triton/Gluon] Add arch_info.is_sw_emulated_fp4_avail() and enable gemm_afp4wfp4 on gfx1101 by @Ragua1 in #7048
  • Cache the MLA split count, not the split indptr tensor by @peizhang56 in #5194
  • [Triton/Gluon] Add MXFP8 Flash Attention v2 (gfx950 / CDNA4) by @WuLei-AMD in #5578
  • [ASM] [HIP] Mha v4: gfx950 hd256 mxfp4/6 kernels with per-batch seqlens_k by @jcaraban in #7173
  • [Triton/Gluon] [gfx950] unified_attention: split-KV 3D configs for head_size <= 128 keyed on max_seqlen_q by @lishuoshuo-amd in #6983
  • [Triton/Gluon] [Config] MoE and MLA configs for gfx1101, gfx1150 and gfx1151: sparse_mla_dsv4_train, sonicmoe by @Ragua1 in #6158
  • [FlyDSL] [JIT] Build the GEMM, conv, chunk_gdn_h and FP8 FMHA AOT caches by replaying the runtime by @zhiding512 in #6956
  • [HIP] [Bugfix] sample_kernels: partial-vector and prologue-tail defects by @yitingw1 in #7373
  • [CI] ci: avoid build runner for disabled wheel jobs by @gyohuangxin in #7392
  • [Triton/Gluon] Support a bf16 recurrent state in the fused GDN decode kernel by @eky-amd in #6884
  • [CI] Update triton-nightly-watchers.txt with new users by @Boss2002n in #7250
  • [JIT] Build every listed CU count of each GPU_ARCHS arch by @RElbers in #6844
  • [HIP] [FlyDSL] [CI] Add aiter.paged_mqa_logits with a tuned Gluon/FlyDSL table by @amd-sriram in #6140
  • [Triton/Gluon] : Change RotatingBuffersBench API and update any existing benchmarks by @rahulbatra85 in #7121
  • [FlyDSL] [gfx950] Add explicit page strides and 64-bit rebase for FP4 MQA logits by @LiuYinfeng01 in #5518
  • [ASM] [HIP] [JIT] Clamp bf16 MLA decode kv_indices and move gfx950 kernels to opus/gfx950/ by @kaiyang-1 in #7395
  • [FlyDSL] pa_decode: MXFP4 K/V in the shared PA tile kernel (gfx950, query_length 1) by @ganyi1996ppo in #7056
  • [HIP] [Bugfix] Skip empty inputs in custom all-gather by @XinyuJiangCMU in #7302
  • [FlyDSL] gfx1250 PA decode: BF16 wave-pair pipeline at D64, drop redundant D128 checks by @zhiding512 in #7384
  • [HIP] [Perf] Route gfx950 Opus MoE sort auto dispatch to multi-phase for E=384 decode by @yelkhamra in #7359
  • [CI] [MLA] test_mla_v4_nm: make the script perf sweep opt-in by @liyjiang in #7321
  • [HIP] [MLA] Store per-request qo lengths before the v1_2 planner reads them by @nehmathe2 in #7418
  • [FlyDSL] [AOT] Build gfx1250-strict wheels from the gfx1250 kernel family by @zufayu in #7379
  • [HIP] [CK] [FlyDSL] [GEMM][gfx950] Tune narrow-N MXFP4 with CSV-selected preshuffle kernels by @yifehuan in #7317
  • [FlyDSL] [CI] [MoE] gemm2: bound atomic rows by the sort's valid token count by @gbyu-amd in #7294
  • [CI] Fix flash attention Windows smoke checkout by @gyohuangxin in #7438
  • [FlyDSL] Glm5 2 qkv proj opt for gfx950 by @IzacharyI in #6427
  • [ASM] [HIP] [CK] Add gfx950 Q256 fused MXFP4 prefill pipeline by @vstakhov-amd in #5988
  • [FlyDSL] Add QSA sparse prefill attention for MI308 by @luocheng25 in #6576
  • [FlyDSL] [CI] One stage and two-stage ring all-reduce kernels for PCIe interconnects by @vpietila-amd in #5461
  • [ASM] [HIP] [JIT] Fold gfx1250 MLA v4.1 sparse prefill split-K in-kernel with workgroup clusters by @kaiyang-1 in #7390
  • review-pr: publish the card as a PR review with findings anchored inline by @zufayu in #7303
  • [Triton/Gluon] fav3_sage: RDNA3 (gfx1100) support: FP16 V and a tuned gfx1100 table by @TwelveNights in #6770
  • [HIP] [FlyDSL] tune bmm and mxfp8 together ,update related tuned csv by @solinzby1 in #7439
  • [Triton/Gluon] Gluon PA decode: allow query_length > 4 on the single-KV-head PS path by @zejunchen-zejun in #6244
  • [FlyDSL] QuickAllReduce INT4: E5M2 group scale so groups above 480 no longer clip by @vorapolsiloai in #7258
  • [Triton/Gluon] bench_sage: route head dim 256 to the hd256 mxfp4/6 producers by @jcaraban in #7174
  • [CI] Build release wheels for ROCm 7.14, 10.0 and 10.1 (Drop 7.0 and 7.1) by @micah-wil in #7428
  • [Triton/Gluon] MHAv4 split-KV by @nsakkine in #6560
  • [Config] MiniMax-M3 MXFP8: tune N=1280 K=6144 (DSpark context KV projection) by @micah-wil in #7411
  • [CI] [Build] Pin FlyDSL to v0.4.2 by @coderfeli in #7340
  • Add gfx950 a8w8 blockscale GEMM tunings for Qwen3-14B-FP8 by @xiaofei-zheng in #7383
  • [Config] Tune Preshuffled A8W8 GEMM for DSR1 Shapes on gfx1250 by @farlukas in #7477
  • [CK] [FlyDSL] gfx942 a16wmix a16w16: honour swiglu_limit for BF16 SiLU and SwiGLU by @jatseng-ai in #6552
  • [HIP] [FlyDSL] Add gfx1250 implicit-GEMM conv3d by @huizzhan in #6662
  • [Triton/Gluon] [GFX950] Add custom sparse MLA kernel for tensor scale fp8 kv cache by @cagrikymk in #7429
  • [FlyDSL] Make PA planner batch and capacity runtime values by @fsx950223 in #7205
  • [ASM] [HIP] [FlyDSL] [gfx1250] asm mha fp8 fwd: add D192x128 per-tensor varlen kernel by @shay-li77 in #7375
  • [FlyDSL] optimize dsv4.1, k3, m3 moe perf for gfx1250 by @binding7012 in #7296
  • [Config] [gfx1250] Tune DeepSeek-V4.1-Flash MX32 FP8 GEMMs at large decode M by @karverma-amd in #7366
  • [Config] Add gfx950 a4w4 GEMM configs for Qwen3.8-2.4T GDN in_proj_qkvz at TP8 by @mjkvaak-amd in #7441
  • [FlyDSL] [MoE] Reuse stage2 scatter/reduce scratch in _flydsl_v2_stage2_wrapper by @afriedri in #7363
  • [Config] Add MI350P TP1 a4w4 GEMM tuning for Qwen3.8-27B-Quark-AWQ-MXFP4 by @a-sidorova in #7075
  • [Triton/Gluon] Add FP8 Flash Attention v2 kernel (gfx942/gfx950) by @WuLei-AMD in #5577
  • [FlyDSL] feat(2stageGR) Two-stage Hyper-Connection Gated Residual kernel by @damien-lejeune in #5913
  • [Config] Add Beam FP8 fused MoE configs for gfx950 by @jamesETsmith in #7224
  • [FlyDSL] Add flyc.compile to skills. and cache HERD host launches by @coderfeli in #7271
  • [HIP] [FlyDSL] [Bugfix] Size the custom allreduce pool from per-op AR/AG/RS max-size env vars by @TennyWang1223 in #7433
  • [Triton/Gluon] Add fused bf16 BMM + cat + FP8 quant + head pad for MLA decode by @lucas-santos-amd in #7171
  • [FlyDSL] [CI] MegaMoEV2: fuse Dispatch, GEMM1, GEMM2 and Combine into one kernel by @IzacharyI in #6881
  • [FlyDSL] optimize dsv4 flash perf by @binding7012 in #7272
  • [Triton/Gluon] [Bugfix] Fix uneven split-K bug in fused a8w8 blockscale GEMMs by @hunhokim in #6854
  • [OPUS] [FlyDSL] [AOT] Cover the full gfx1250 kernel family for gfx1250-strict wheels by @zufayu in #7434
  • [Config] Add Kimi-K3.1 Pro gfx950 BF16 GEMM tuned rows to kimik3 tables by @zhuyuhua-v in #7485
  • [HIP] fea: impl lamport allreduce on gfx9 by @TennyWang1223 in #7452
  • [HIP] [JIT] Guard Mega-MHC gfx1250 sources by build target by @junhaha666 in #7491
  • [FLYDSL] Fea/mxfp4 flydsl a8w4 opt & replace by @Bernard-Liu in #6464
  • [HIP] [Perf] Pipeline CSR segments in unsplit OPUS sparse prefill by @iraj465 in #6831
  • [HIP] [JIT] gfx1250 mha: fix the CK JIT builds and route the Python backward to asm by @slippedJim in #7487
  • [Triton/Gluon] KDA cleanup by @omuhamma in #7034
  • [Config] configs: add MiniMax-M3.1 gfx950 GEMM tuning by @XiaobingSuper in #7498
  • [Triton/Gluon] [Config] Add gfx1151 fused_sigmoid_mul config by @klin2024 in #6982
  • [Triton/Gluon] [Config] Add gfx1151 batched_gemm_a16w16 configs by @klin2024 in #6985
  • [Triton/Gluon] [Config] Add gfx1151 RMSNorm large-M/small-N config by @klin2024 in #6979
  • [Triton/Gluon] [Kernel] Add MXFP4 dequant H16 requantization by @DaiXindi-AMD in #6476
  • [Triton/Gluon] unified_attention: keep 3D split-KV padding rows from storing segment stats by @AndreasKaratzas in #7472
  • [FlyDSL] [AMD][DSV4] fix: drop expert -1 pad routes in the fused TP route/psumkernel by @karverma-amd in #7365
  • [ASM] Fix KV addressing, mask scratch and empty splits in gfx950 BF16 32-head MLA by @alexioslyrakis-amd in #6578
  • [Triton/Gluon] Add fused_reduce_add_rmsnorm by @eky-amd in #6678
  • [Triton/Gluon] MHA prefill: use the gfx1151 attn_fwd config on gfx1150 by @Ragua1 in #7257
  • [Triton/Gluon] [FlyDSL] [CI] [Bugfix] Never share scratch buffers whose zero fill was recorded into a graph by @Fangzhou-Ai in #7126
  • [HIP] [CI] [JIT] Add aiter.mqa_logits with a tuned Triton/FlyDSL table by @amd-sriram in #6185
  • [Triton/Gluon] Fix the hardcoded die and CU counts in the Triton kernels by @RElbers in #5505
  • [Config] [Tuning] Qwen3.8-Flash-Next TP4 prefill BF16 GEMMs on MI308X (gfx942, 80 CU) by @zovonoir in #7204
  • [FlyDSL] [gfx950] Add gdn fused prefill kernel by @amd-nprotaso in #6496
  • [CI] Make PR Self-Review verdicts evidence-driven instead of event-driven by @zufayu in #7482
  • [HIP] [MLA] Align gfx942 QH64 FP8 page-size planning by @vstakhov-amd in #7464
  • [CI] Scope test_moe_2stage.py to representative tokens in standard CI by @shantipriya-amd in #7330
  • [Feature][MoE] fused_moe: skip moe_sorting when the caller registers a BM=16 presort by @yuychang in #6484
  • [Config] [gfx1250] Tune MiniMax-M3 DP4/EP4 dense GEMMs by @XiaobingSuper in #7507
  • [HIP] Speed up gfx1250 two-stage AR RMSNorm FP8 amax reduction by @xytpai in #7080
  • [Bugfix] Clarify fused all-reduce limit units with MAX_BYTES by @yelkhamra in #7357
  • [HIP] Fuse a 32-point Hadamard rotation into the MXFP4 per-1x32 quant and AR + RMSNorm epilogue by @mjkvaak-amd in #6734
  • [CI] Skip Flash Attention CI on draft PRs by @gyohuangxin in #7541
  • [FlyDSL] moe_reduce: guard against -1 topk_ids padding sentinel by @rjrock in #7104
  • [Triton/Gluon] [FlyDSL] [MoE] Kimi-K3 HERD on gfx1250: K+1 sigmoid top-k and one route+psum kernel by @charlieguo1106 in #7494
  • [Config] [gfx1250] Tune DeepSeek-V4.1-Flash dense FP8 GEMM tiles for intermediate-M decode by @karverma-amd in #7234
  • [Config] Mistral Large 4 gfx950 tuned tables (TP8 + TP4) by @hongxiayang in #7235
  • [OPUS] [gfx1250] Clear HIP sticky error after A0 strict-CO probe failure (#7185 follow-up) by @zufayu in #7505
  • [HIP] [Bugfix] Skip grouped top-k launches with zero tokens by @XinyuJiangCMU in #7376
  • [FlyDSL] Move _e8m0_byte_to_f32 into mxfp4_gemm_common by @ganyi1996ppo in #7424
  • [ASM] [HIP] feat(fmha_bwd): gfx950 sliding window attention backward kernels by @slippedJim in #7215
  • [CK] [Bugfix] Align batch prefill full-attention dispatch by @lunarwhite in #6990
  • [FlyDSL] [AOT] Hide the GPUs from replays with empty device lists, not -1 by @zhiding512 in #7551
  • [Triton/Gluon] [ASM] [HIP] MHAv4 per-row block-sparse geometry, LSE and ragged KV on gfx950 by @nsakkine in #6575
  • [Triton/Gluon] [Feature] MHA: add out_dtype to the Triton forward entry point by @phambinhfin in #7232
  • [FlyDSL] Fix main CI: missing DROPPED_ROUTE_ROW import in moe_contiguous_psum by @RElbers in #7583
  • [ASM] Fix LDS race between V reads and R re-layout in gfx950 BF16 QH3… by @minmengdie in #7574
  • [FlyDSL] [AITER] FlyDSL a8w8 256x256 nb4/nb2 persistent kernel improvement by @plognjen in #7238
  • [HIP] [FlyDSL] [CI] Add tunable HGEMM K-stagger and B streaming loads by @xytpai in #7546
  • [FlyDSL] fused_compress_attn: zero padding kv in Phase 1 by @yhl-amd in #7550
  • [Config] [MoE] retune DeepSeek-V4.1-Flash TP2 decode buckets by @yhl-amd in #7602
  • [HIP] [OPUS] [JIT] MXFP4 MQA logits update: one API for V4 and V4.1 with Table/Free/Prefill schedules by @shay-li77 in #6952
  • [FlyDSL] Target the AOT dry run through FLYDSL_GPU_ARCH only by @coderfeli in #7598
  • [FlyDSL] [Bugfix] Zero consumed kv in the gfx1250 HCA Phase 2 loop by @hubertlu-tw in #7601
  • [ASM] [HIP] Qwe38 flash mxfp4 asm by @JohnNikolay84 in #5757
  • [Perf] DeepSeek-V4.1-Flash TP2 decode: indexer top-k split gate and router GEMM row by @yhl-amd in #7624
  • [HIP] [FlyDSL] Add OPUS FP4 MQA writer and query scale layouts by @junhaha666 in #7128
  • [ASM] [HIP] [gfx1250] asm mha bf16 hd256: batch and varlen forward by @shay-li77 in #7559
  • [ASM] [HIP] perf(pa): update gfx950 unified MTP code objects by @fangche123 in #7371
  • [Fix] Build DeepSeek RoPE caches on the default device, not hard-coded cuda by @XinyuJiangCMU in #7298
  • [Config] Tune Kimi-K3 gfx1250 gemm_a4w4 by @lirui927 in #7138
  • [HIP] Added an optional pre-quant k_out_unquantized under fp8 kv-cache to fused_qk_norm_rope_cache_pts_quant_shuffle by @jhu960213 in #6808
  • [CI] Run Windows smoke on Radeon R9700 runner by @gyohuangxin in #7634
  • [Triton/Gluon] [QSA] FP8 KV and FP8 / FP4 index caches, stacked-head decode scorer by @ZLkanyo009 in #7008
  • [CI] review-pr: watch for review requests starved of a hosted runner by @zufayu in #7638
  • review-pr: the queue notice was wrong in both directions by @zufayu in #7615
  • review-pr: route the design judgment instead of making it (Step 7.8) by @zufayu in #7563
  • [CI] Add area contacts and review guide; document merge rules and review timeline by @zufayu in #7560

New Contributors

Full Changelog: v0.1.28...v0.1.29

Contributors

ruanjm, rahulbatra85, and 97 other contributors
Assets 10
Loading