Skip to content

AITER v0.1.26

Choose a tag to compare

View all tags
@github-actions github-actions released this 19 Oct 08:00
v0.1.26
84077b3
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

AITER v0.1.26 - weekly release

Scheduled release from release/v0.1.26.

Diff base: v0.1.25

Wheels

Prebuilt manylinux_2_28 wheels, GPU_ARCHS=gfx942;gfx950:

ROCm Python Wheel
ROCm 7.0 cp310 amd_aiter-0.1.26+rocm7.0.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.0 cp312 amd_aiter-0.1.26+rocm7.0.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.1 cp310 amd_aiter-0.1.26+rocm7.1.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.1 cp312 amd_aiter-0.1.26+rocm7.1.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.2 cp310 amd_aiter-0.1.26+rocm7.2.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.2 cp312 amd_aiter-0.1.26+rocm7.2.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
What's Changed

Changes

  • [CI] ci: skip OPUS config on draft PRs by @gyohuangxin in #6418
  • [FlyDSL] fix(flydsl): support kmk3 ep moe & opt for ep moe gemm2 by @Bernard-Liu in #5804
  • [CI] CI: auto-update split test times by @aiter-gh-app[bot] in #6408
  • [Config] [Bugfix] Retune DSV3 fused-shared-expert fmoe rows on gfx942 by @simondanielsson in #6190
  • [ASM] [HIP] [JIT] mla version 6 by @feifei14119 in #6394
  • [CI] [Build] review-pr: answer the requester at the gate, and report a review that vanishes by @zufayu in #6352
  • [FlyDSL] [gfx1250][MoE] Row-major A1 e8m0 scale store for decode MoE (drop scattered preshuffle) by @karverma-amd in #6357
  • [Config] [gfx950] Add GLM-5.3-Flash TP4 fused KDA in-proj bf16 GEMM tunings by @Jacob0226 in #5982
  • docs: Copilot FlyDSL path instructions and review-pr C5 by @samremes in #5730
  • [HIP] [CK] Re-submit Relu2 Tier 2 MoE GEMM support (clean CK pin) by @shantipriya-amd in #6376
  • [Config] Add GLM-5.3-Flash TP2 tuned fused-MoE configs for gfx950 by @willhu-jpg in #6077
  • [Triton/Gluon] [GFX12] Gluon mhc_fused_post_pre by @k50112113 in #6226
  • [Triton/Gluon] Improving GFX950 Unified Attention for spec. decode and reduce opimizations by @cagrikymk in #6336
  • [CI] Add Windows smoke test for JIT ops by @gyohuangxin in #6242
  • [Triton/Gluon] Accept strided leading dimensions in GDN L2Norm by @vorapolsiloai in #5624
  • [Triton/Gluon] Enable fp fusion for kda by @nsusanto in #6289
  • [Triton/Gluon] [Config] Add gfx950 FP4 GEMM configs for Kimi-K3 shared-expert shapes by @yuychang in #6409
  • [Triton/Gluon] [Bench] bench_mla: pass the output buffer to mla_prefill_fwd by @zhanglx13 in #6251
  • [Triton/Gluon] : Remove non-Triton(batch prefill) benchmark from Trito… by @rahulbatra85 in #6349
  • [Triton/Gluon] [Config] Add Qwen3.8-27B bf16 GEMM tunings for gfx950 by @eky-amd in #6156
  • [Triton/Gluon] Conflict-free bf16 KV-tile stores in gfx950 sparse_mla rope-free walk by @Raiden-Makoto in #6276
  • [FlyDSL] Add gfx1250 candidates to the A16W16 tuning policy by @jhinpan in #6396
  • [FlyDSL] Dev/yadai moe v1 by @yadaish in #6269
  • [FlyDSL] [Perf] MegaMoE stage1: fold the ready-order tile wait into the scheduler block by @AMD-yanfeiwang in #6431
  • [FlyDSL] fix megamoe ubench by @yanboshao in #6425
  • [FlyDSL] [JIT] comm fused moe stage2 reducescatter for dsv4 dpa by @yifehuan in #5907
  • Report sparse prefill bandwidth in TB/s by @junxiaguo in #6420
  • [HIP] [CK] Revert " Re-submit Relu2 Tier 2 MoE GEMM support (clean CK pin)" by @gyohuangxin in #6460
  • [CI] Skip OPUS test for unrelated workflow changes by @gyohuangxin in #6465
  • [Config] small-M shapes for Kimi-K3 BF16 GEMMs for gfx950 by @samutamm in #5903
  • [OPUS] [Bugfix] Restore shadowed OPUS A16W16 cache-policy registrations by @adenzhou1350 in #5972
  • [Config] Add Qwen3.8-27B TP1 a8w8 blockscale B-preshuffle GEMM tunings for gfx950 by @prashant182 in #6033
  • [ASM] [HIP] [JIT] Add a gfx950 asm GEMM with BF16 activations and FP8 weights for decode by @maeehart in #6101
  • [CI] Schedule Windows smoke test daily by @gyohuangxin in #6457
  • [CI] Report Windows smoke test accuracy diffs by @gyohuangxin in #6466
  • [Config] Re-tune 136 unparseable Qwen3.5-397B a8w8 bpreshuffle rows on gfx950 by @jiaqiang-dot-liu in #6393
  • [OPUS] [JIT] Default RMSNorm BF16 stores to optimized RNE by @junhaha666 in #6300
  • [FlyDSL] [MoE] Add herd_fused_topk drop-in selector for fused_moe 2-stage by @charlieguo1106 in #5693
  • [HIP] feat(custom_ar): add torch.symm_mem transport(mori) for gfx1250 by @kawhil-amd in #5431
  • [Triton/Gluon] Add TDM store to bf16 BMM and tune DSR1 shapes by @azaidy in #6240
  • [Triton/Gluon] [Config] [Perf] [gfx950] Use 1x512 Gluon MXFP4 quant tiles for M <= 2 by @reger-men in #6443
  • [Triton/Gluon] [Config] [gfx1201] Add gfx1201 GEMM A8W8 tuning configs by @big-yellow-duck in #6245
  • [Triton/Gluon] chunk_kda gfx1250: use shared_memory_descriptor.reinterpret by @zhanglx13 in #6250
  • [Triton/Gluon] Muse fused kernel for rmsnorm + add + rmsnorm by @vgokhale in #6343
  • [Triton/Gluon] Add MOE Layout LRU cache and tunning by @k50112113 in #6338
  • [Triton/Gluon] [Bugfix] Stop the split-K tail at K in the blockscale GEMMs by @siliangchen-amd in #5873
  • [Fix] topk_per_row_argmax: return early on zero rows by @yitingw1 in #6471
  • [Triton/Gluon] [Bugfix] [Perf] Fix non-preshuffle MQA paging and large KV offsets, retune the decode grid, and cut plain-path instruction count by @zhiding512 in #5614
  • [HIP] [gfx950] Context-parallel block selection for the MiniMax-M3 indexer by @ukannika in #6359
  • [HIP] [CK] [JIT] torch-free a4w4 GEMM + C++ library build by @Micky774 in #4124
  • [JIT] Diagnose missing rocPRIM headers for the selected ROCm toolchain by @Qubitium in #5166
  • [Triton/Gluon] Support Muse QK norm scale in fused RoPE cache op by @vgokhale in #6290
  • [Triton/Gluon] [GFX12] Add optimized gluon backend for mhc_fused_post_pre by @k50112113 in #6442
  • [Triton/Gluon] [Config] Add gemm_a8w8 Triton config for gfx1101 by @Ragua1 in #5960
  • [Triton/Gluon] Avoid the peeled sparse MLA spill on 64-bit gathers below triton 3.8 by @1am9trash in #6042
  • [Triton/Gluon] [Perf] Cache gfx1250 Gluon MXFP4 preshuffle GEMM layouts by @qichu-yun in #6480
  • [Triton/Gluon] Fix HSTU test dtype failures by @azaidy in #6493
  • [Triton/Gluon] Fix shuffled kv cache usage for mla prefill bench by @azaidy in #6491
  • [Triton/Gluon] Bench UA add shuffled_kv_cache flag by @k50112113 in #6284
  • [Triton/Gluon] [CI] [Kernel] Gluon paged-decode attention with sigmoid output gate and group-128 FP8 epilogue (gfx950) by @rbrugaro-amd in #5715
  • [Triton/Gluon] sage_quant: use each tensor's own strides for Q/K/V inputs and outputs by @Shan2L in #6400
  • [CI] Run only the retuned kernels' tests for config-only PRs by @Boss2002n in #6495
  • [Triton/Gluon] Tuning based on Default config json by @Boss2002n in #6128
  • [ASM] Add gfx1250 QH128 page64 MLA with unified decode API by @junxiaguo in #6133
  • [ASM] fix(mla): prevent gfx950 FP8 softmax underflow by @fangche123 in #6422
  • [HIP] [CK] [FlyDSL] Accelerate MXFP4 FMoE AOT tuning ~9.5x on gfx950 by @Qubitium in #6405
  • [HIP] [OPUS] [JIT] Fp8 mxscale bmm bpreshuffle opt by @yzhou103 in #4747
  • [Config] [GEMM] [gfx950] Add MiniMax-M3 master BF16 GEMM table & tuned ViT linear projections by @amd-bishwoadhikari in #6235
  • [JIT] Check single-file tuned configs for duplicate shapes by @yzhou103 in #6365
  • [Triton/Gluon] [CI] Move common kernels into common/ by @vgokhale in #6505
  • [FlyDSL] [gfx1250] Use FP32 partials for A8W8 split-K GEMMs by @aoli26 in #6258
  • [ASM] [HIP] Add page16 PA_PS kernel and parameterized tests by @fangche123 in #6470
  • [FlyDSL] [MegaMoE] A4W4 mega_moe operator by @Yaowu-Xiong in #4980
  • [CI] review-pr: name the gate that withheld a review, and stop the bot triggering itself by @zufayu in #6516
  • [HIP] [BUG fix] Initialize reverse route indices before Opus sorting by @charlieguo1106 in #6467
  • [HIP] [Dist] Support gemma_norm in fused all-reduce + RMSNorm + per-group FP8 quant by @zhuyuhua-v in #6472
  • [Config] Add gfx950 a8w8 blockscale GEMM tunings for GLM-5.2-FP8 shapes by @jbelloncastro in #5994
  • [fix] Key GLM-5 MXFP4 EP4 fused-MoE rows on topk 9 and add prefill rows by @xiaofei-zheng in #6392
  • [HIP] [OPUS] feat(moe): fused route-sort + MXFP4 activation-quant kernel for… by @Emmanuel0612 in #6165
  • [FlyDSL] M3 index score flydsl by @ganyi1996ppo in #5626
  • [ASM] Chefang/pa ps page16 co variants by @fangche123 in #6473
  • [Config] Add Gemma-4-31B TP1 a4w4 blockscale GEMM tunings for gfx950 by @prashant182 in #5928
  • [HIP] [FlyDSL] add mxfp8 w_scale 1*32 (scalea/b nonpreshuffle) by @solinzby1 in #6407
  • refactor: [MORI] Make all-to-all manager creation demand-driven by @JiaoliangYu in #6517
  • [FlyDSL] Single-Launch Dynamic Decode TopK by @lirui927 in #6329
  • Move the a4w4 aux-sort fused/Opus crossover to 1.33 rows per expert by @nsusanto in #6225
  • [FlyDSL] [CI] [Build] Bump FlyDSL to 0.4.0 by @coderfeli in #6424
  • [Triton/Gluon] [Perf] Cache gfx1250 Gluon MoE A8W4 layouts by @zjin-lcf in #6500
  • [Triton/Gluon] simplify triton docs by @Boss2002n in #6499
  • [Triton/Gluon] benchmark and tune for fused_reduce_qk_norm_rope_swa_write on gfx950 by @nidal567 in #6109
  • [Triton/Gluon] tune prefill num_warps on gfx950 (sequence layout, gate prefill path) by @nidal567 in #6017
  • [Triton/Gluon] [PA] Support Q/K D192 with asymmetric K/V head dim by @sammysun0711 in #4968
  • [Triton/Gluon] Satya/1250 config cleanup by @Boss2002n in #6293
  • [ASM] Add gfx942 BF16 MLA persistent decode asm kernel with LSE output by @MiloLurati in #6220
  • [ASM] [HIP] [JIT] Chefang/pa ps page16 reduce co by @fangche123 in #6477
  • [Triton/Gluon] [MoE] Specialize gfx1250 biased sigmoid top-k by @XiaobingSuper in #6131
  • [FlyDSL] fix(comm-fused-moe): key winners by GPU model by @yifehuan in #6541
  • [ASM] fix(pa): correct gfx950 page16 short-tail entry guards by @fangche123 in #6528
  • [Triton/Gluon] [gfx950] Fix ff_a16w16_fused M<=8 launch failure, add tuned configs by @yuyzhang512 in #5399
  • [Triton/Gluon] [Bugfix] Advance KDA gate pointer by the token stride in fused_rearrange_sigmoid_gdr by @truong-v in #6321
  • [HIP] fix: remove stale MXFP4 MoE aux generated sources by @adenzhou1350 in #5936
  • [CI] [Build] Publish a py3 wheel without prebuilt kernels next to the prebuilt wheels by @aryaman-gupta in #6006
  • [Triton/Gluon] [CI] [gfx942] Tuned sparse MLA prefill for DSv4.1-Flash TP4; fix attn_sink in Triton fallback by @juuso-oskari in #6002
  • [FlyDSL] [Perf] Fuse gfx1250 gather_kv_b_proj by @XiaobingSuper in #6121
  • [Config] Tune a8w8 bpreshuffle GEMMs by @farlukas in #6547
  • [Config] [Tuning] Add gfx1250 A4W4 grouped-MoE tuning for the Kimi-K3 WideEP by @lirui927 in #6438
  • Skip comm-fused MoE test when MORI CCO is unavailable by @zufayu in #6571
  • Dispatch the LayerNorm ops to Triton on gfx1250 or when ENABLE_CK=0 by @jiaryang in #6169
  • [FlyDSL] [gfx950] Add dense BF16 x MXFP4 GEMM by @LiuYinfeng01 in #4772
  • [FlyDSL] [CI] Add HY4 MXFP8 heterogeneous MoE fusion by @qli88 in #6050
  • [ASM] [Bugfix] Fix gfx942 PA KV tail handling for lengths 241-255 by @fangche123 in #6569
  • [Config] [Perf][gfx950] Retune Kimi-K3 merged-front shapes for ROCm 10 by @jiacao-amd in #5956
  • [FlyDSL] split-K epilogue optimization in gfx950 a16w16 + Kimi-K3 router/in_proj tune by @nsusanto in #6494
  • [ASM] [CI] Optimize gfx950 MXFP GEMMs for Flux training shapes by @ksikiric in #6309
  • [Triton/Gluon] [QSA] Add paged sparse attention kernels by @mengfei-jiang in #6469
  • [Config] Revert " [Perf][gfx950] Retune Kimi-K3 merged-front shapes for ROCm 10" by @valarLip in #6587
  • [Triton/Gluon] Tune mla_gluon by @farlukas in #6283
  • [Triton/Gluon] [Config] Tune gfx1250 Gluon a16w16 GEMMs for GPT-OSS 120B by @Boss2002n in #6549
  • [HIP] [CK] [MoE] Fix missing SwiGLU non-split no-bias CK-Tile MXFP4 dispatch (follow-up to #6376 revert) by @shantipriya-amd in #6478
  • [ASM] Correct SGPR counts in gfx950 QH16 MLA metadata by @alexioslyrakis-amd in #6577
  • [CustomAllreduce] Re-take the expandable_segments decision at capture time by @okorzh-amd in #5799
  • [ASM] [Bugfix] Fix gfx950 MLA KV addressing and optimize gfx942 addressing by @alexioslyrakis-amd in #6481
  • [Config] Add Kimi-K3 gfx950 BF16 GEMM tuned configs by @yuychang in #6411
  • [HIP] [JIT] [Build] IQ2R glm53_moe: packed 2-bit MoE kernels for GLM-5.3 TP4/TP8 (BF16) by @ssharma4-amd in #5728
  • [HIP] Fix ar_ll128_unroll2 hang on consecutive calls of different sizes by @AndreasKaratzas in #6446
  • [Triton/Gluon] Fix GLM-4.7 MHA benchmark head dimension by @zufayu in #6608
  • [FlyDSL] [JIT] Fix Windows build arguments and optional imports by @Yasei-no-otoko in #6589
  • [ASM] [HIP] [JIT] MI355 MLA PS mode kernel add 4 wave fp8 kernel by @minmengdie in #6455
  • [Config] Add gfx942 Kimi-K3 a16wi4 tuned MoE configs for topk=16 by @xaguilar-amd in #6271
  • [HIP] [CK] [FlyDSL] add 3232/128128 mxfp8 and add tune with bmm(batch=1) by @solinzby1 in #6603
  • [ASM] [HIP] gfx1250 mha bwd kernel integration by @slippedJim in #6454
  • [ASM] [HIP] Add gfx1250 QH64 shuffled KV decode by @feifei14119 in #6561
  • [HIP] Opt qk norm rope mla cache gfx1250 by @yzhou103 in #5978
  • [MoE] Keep the CK-Tile split-k route for SwiGLU MXFP4 callers that leave gate_up_interleave unset (MiniMax-M3 TP8 regression from #6478) by @ZhiweiYan-96 in #6620
  • [CI] Enable triton test selection script by @Boss2002n in #6599
  • [Triton/Gluon] [Config] Tune Triton Conv3D launch parameters for gfx1100 by @saeid-rostami in #6588
  • [Triton/Gluon] PA decode: use more splits for small grids in get_recommended_splits by @NimitPtl in #6453
  • [FlyDSL] Optimize PA partition reduction (split from #5949) by @fsx950223 in #6619
  • [Triton/Gluon] [Config] Add gfx950 preshuffled AFP4WFP4 GEMM configs for Llama-3.1-405B MXFP4 TP8 by @afriedri in #6546
  • [Triton/Gluon] [Config] [gfx1100] Add gfx1100 (RDNA3) tuned Triton A16W16 GEMM configs by @WhatGhost in #4234
  • [Triton/Gluon] [Config] Tune gfx950 Muse projection GEMMs in Triton and FlyDSL by @vgokhale in #6507
  • [Triton/Gluon] [Config] [Perf] Retune gfx950 preshuffled AFP4WFP4 config for N=4096, K=14336 by @reger-men in #6521
  • [Triton/Gluon] batched_gemm_a16wfp4: ragged-K regression test and operand shape validation by @lijinpei-amd in #4685
  • [Docs] Update CLAUDE.md to clarify file precedence by @Boss2002n in #6542
  • [HIP] [OPUS] [FlyDSL] Simplify AITER compiler worker fan-out to smooth-out regressions by @Qubitium in #5164
  • [MHA] Fix -DONLY_FAV3=0 quoting that breaks the backward JIT build by @menglcai in #6616
  • [FlyDSL] [FMHA][gfx1250] m32x8 prefill: d64 support, wider shape coverage, and a rewritten KV pipeline by @ruanjm in #6355
  • [ASM] [HIP] Rebuild the gfx950 bf16 opus MLA decode kernels with named-register pins by @kaiyang-1 in #6536

New Contributors

Full Changelog: v0.1.25...v0.1.26

Contributors

Qubitium, Yasei-no-otoko, and 96 other contributors
Assets 8
Loading