Repository navigation
AITER v0.1.25
AITER v0.1.25 - weekly release
Scheduled release from release/v0.1.25.
Diff base: v0.1.24
Wheels
Prebuilt manylinux_2_28 wheels, GPU_ARCHS=gfx942;gfx950:
What's Changed
Changes
- [Config] Add gfx942 a8w8 blockscale GEMM tunings for Qwen3/Qwen3.5/GLM/DSV4 shapes by @siliangchen-amd in #5839
- [HIP] [OPUS] [FlyDSL] gfx950 MXFP8 e8m0 GEMM + DeepSeek-V4/V4.1 tuned configs by @valarLip in #5896
- [Triton/Gluon] [ASM] [HIP] Mha v4: adds bf16 sparse, LSE support, KV varlen, fixes, etc by @jcaraban in #5798
- [tuner] Fail a task as soon as its worker process exits by @siliangchen-amd in #5841
- [CI] Update Aiter artifact downloads to v8.0.1 by @leo-automation in #5908
- [MLA v4 nm] Test fix _run_one_point reading packed BF16 as FP32 partials by @liyjiang in #5905
- [FlyDSL] [JIT] [AOT] Inline the FlyDSL FP8 FMHA head shapes, drop the config CSVs by @gbyu-amd in #5901
- [FlyDSL] gfx942 fp8_mqa_logits: let _auto_variant choose rows_per_block by @jin-amd in #4963
- [Triton/Gluon] [Config] Add a tuned gfx1101 MHA config, split small_head/default by @Ragua1 in #4493
- [Triton/Gluon] Consolidate tuning harnesses by @Boss2002n in #5874
- [Triton/Gluon] [Bugfix][MLA] Follow-up: fix stale comments + test hardening (#5648) by @Rohan138 in #5860
- [Triton/Gluon] [Config] gemm_a16w16: gfx950 tuned per-shape configs by @NimitPtl in #5830
- [Triton/Gluon] [gfx950] [dsv4.1-flash] optimizing the mHC fused kernel by @ahmed-bsod in #5885
- [HIP] Tune DSV4.1 TP4 A4W4 fused MoE configs for gfx950 by @yifehuan in #5919
- [Config] Add Qwen3.8-27B TP1 a8w8 blockscale GEMM tunings for gfx942 by @phambinhfin in #5585
- [HIP] [OPUS] [JIT] Unify gfx950+gfx1250 MXFP4 MQA-logits by @shay-li77 in #5761
- [FlyDSL] feat(mega_moe/gfx1250): bind mori tokoff-ext allocator on the mori dispatch path by @jhchouuu in #5810
- [Bugfix] Do not write segment softmax state at NUM_SEGMENTS_PER_SEQ == 1 by @sogalin in #5935
- [Triton/Gluon] [Config] Enable unified-attention skip-mask for gfx950 hd256 FP8 prefill by @vorapolsiloai in #5598
- [Triton/Gluon] [Kimi-K3][ROCm] Add merged MoE front by @jiacao-amd in #5321
- [Triton/Gluon] Add SonicMoE pure-Triton grouped GEMM MoE by @WuLei-AMD in #5725
- [Triton/Gluon] Move the KDA_DECODE configs into the nested config layout by @Boss2002n in #5941
- [Triton/Gluon] Move Triton unit tests into their wrapper's folder by @Boss2002n in #5943
- [Triton/Gluon] Move _gluon_kernels/gfx1250/norm/ to _gluon_kernels/gfx1250/normalization/ by @Boss2002n in #5944
- [CI] Select impacted Triton and Gluon unit tests by @Boss2002n in #5878
- [Triton/Gluon] [Config] Drop no-op kpack from RDNA GEMM configs by @hyjuunn in #5917
- [Triton/Gluon] [GFX12] mxfp8 gemm cga update by @k50112113 in #5055
- [Triton/Gluon] stop fused_bmm_rope_kv_cache from using batched_gemm_a8w8_a_per_token… by @Boss2002n in #5877
- [Triton/Gluon] Add Triton-based Conv3D kernels by @saeid-rostami in #5952
- [HIP] [Perf] Fold shared-expert gate GEMV into topk_softmax kernel by @Emmanuel0612 in #5652
- [Config] Add Qwen3.8-27B gfx950 A4W4 GDN BA projection shapes by @vorapolsiloai in #5836
- [CI] Preserve install failure status after retries by @vgokhale in #5864
- [Config] Add Qwen3.8-Flash-Next PTPC-FP8 gfx942 a8w8 bpreshuffle and BF16 GEMM tunings by @zovonoir in #5974
- [ASM] [HIP] Add gfx950 MXFP4 SiTUv2 FLAT 16x192/16x128/16x32 kernels by @JohnNikolay84 in #5763
- [Config] Add tuned a8w4 fused-MoE configs for DeepSeek V4.1-Flash (TP4 + EP, gfx950) by @cpersson-amd in #5904
- [Config] Tune a8w8 bpreshuffle for Kimi-K3's KDA gate at tp4 by @XiaobingSuper in #5783
- [FlyDSL] Paged mla indexer by @fhuizing in #4221
- [Triton/Gluon] [GFX950] Fix LDS OOM on mxfp8 triton gemm by @k50112113 in #5963
- [CI] Triton board: drive PR status from the workflow by @Boss2002n in #5880
- [Triton/Gluon] Move Triton unit tests into their wrapper's folder by @Boss2002n in #5942
- [Triton/Gluon] [Kernel] Add fused SiLU-and-multiply backward by @DaiXindi-AMD in #5542
- [Triton/Gluon] Unified attention sglang compatability fix by @cagrikymk in #6009
- [OPUS] Speed up the gfx1250 32mx1 MLA sparse-prefill kernels by @kaiyang-1 in #5805
- [Config] add DSR1 gfx1250 bf16 GEMM tuning by @demonsan in #5939
- [HIP] [FlyDSL] [JIT] [gfx1250][GEMM] add MXFP8 1x32 bpreshuffle GEMM with tuned ASM/FlyDSL dispatch by @aoli26 in #5818
- [ASM] [HIP] [FlyDSL] Add pin gpr(nhead96/128) and round-robin DCP to persistent MLA PS1 FP8 decode for GFX1250 by @xiangM99 in #5844
- [HIP] [CK] Add relu2 (Squared ReLU) activation support to CK-Tile fused MoE GEMM by @shantipriya-amd in #5708
- [Triton/Gluon] [Config] Add gfx950 preshuffled AFP4WFP4 GEMM config for N=4608, K=8192 by @mjkvaak-amd in #5998
- [Triton/Gluon] Fix pa_decode compile error with newer Triton: don't index scalar alpha by @zhanglx13 in #5886
- [Triton/Gluon] [Config] Add gfx1250 A16W16 GEMM configs for Wan2.2 shapes by @sogalin in #6032
- [Triton/Gluon] Assert SPLIT_UNMASKED_LOOP is not used with sliding window by @amd-xavierwang in #5771
- [Triton/Gluon] Fix fp8 KV correctness reference in bench_unified_attention by @mpashkovskii in #5663
- [Triton/Gluon] [Config] Tune gfx1201 unified attention 2d for D=512 for full attention and D=256 for Sliding Window (Gemma4 shapes) and fix D=1024 launches by @GodRishUniverse in #5790
- [Triton/Gluon] Account for sliding_window in bench_unified_attention metrics by @mpashkovskii in #5665
- [Triton/Gluon] Skip MXFP4 logits tests on unsupported GPUs by @brunomazzottiamd in #5958
- [Triton/Gluon] Fix Conv3D CPU tests with a CUDA default device by @whx-sjtu in #5984
- [Triton/Gluon] Use a zero default for masked scales in gemm_afp8wfp8 by @cagrikymk in #6007
- [Triton/Gluon] gemm_a16w16_atomic accumulate into an existing output buffer by @nsusanto in #5895
- [Triton/Gluon] MHA fwd: gfx950 mid_head config for 64 < head_dim <= 128 by @NimitPtl in #5858
- [Triton/Gluon] gmm: gfx950 large-K/N config with JSON-driven shape dispatch by @NimitPtl in #5857
- [Triton/Gluon] Move Triton unit tests into their wrapper's folder by @Boss2002n in #5940
- [Triton/Gluon] Expert-parallel support for moe_gemm_a16w4 by @Rohan138 in #5863
- [Triton/Gluon] Add 3D neighborhood flash attention (na3d_flash) by @jjuvonen-amd in #5230
- [CK] [Bugfix] Keep two K tiles per split in blockscale MoE stage-1 split-K by @jin-amd in #5979
- [Triton/Gluon] tuning k3 shapes for a16w16 by @omuhamma in #5875
- [Triton/Gluon] [Config] Add gfx950 gemm_a16w16 config for N=288, K=4096 by @Jacob0226 in #5933
- [FlyDSL] Dev/a4w4 prefill v2 yadai by @yadaish in #5934
- [CK] [FlyDSL] add mxfp8 a8w8 blockscale for gfx950 by @solinzby1 in #4951
- fix topk_select restriction by @HaonanWang98 in #6080
- [Triton/Gluon] [HIP] gemm_a16w16: gfx950 config for MiniMax-M3 lm_head (N=50016, K=6144) at small M by @valarLip in #6079
- [Triton/Gluon] fix gluon import and benchmark + tune fused_clamp_act_mul for gfx950 by @nidal567 in #5800
- [Triton/Gluon] [Config] gfx950 unified attention prefill tile for head_dim 512 (Gemma 4 global layers) by @prashant182 in #5926
- [Triton/Gluon] spec decode support for gluon kda files by @omuhamma in #6034
- [Triton/Gluon] Gluon Chunked Kimi Delta Attention (950/1250) by @omuhamma in #5866
- [Triton/Gluon] Changes related to gla for Kimi K3 by @omuhamma in #6056
- [Triton/Gluon] enable TDM fusion on MXFP4 by @Boss2002n in #6097
- [Triton/Gluon] [Bugfix] batched_gemm_a8w8: keep scale loads in bounds by @akii96 in #6082
- [ASM] [CI] Remove committed scratch files and stop two from coming back by @Boss2002n in #5544
- [Triton/Gluon] Add a16w4 MoE tuned dispatch and gfx942 DSv4.1 tables by @akii96 in #5667
- [ASM][flydsl] opt mla kernel for dspr1(gfx1250) by @xiangM99 in #6087
- [Triton/Gluon] [Perf] MXFP8 MoE: keep the MFMA in FP8 when MX_SCALE_BLOCK_K == 1 by @akii96 in #5832
- [CI] [JIT] Fix ATOM GPU isolation and startup failures by @leo-automation in #5989
- [HIP] gfx950 A8W4 fused_moe configs for DSv4.1 TP-sharded experts by @valarLip in #6125
- fix(custom_all_reduce): allocate IPC handle tensors on the host by @valarLip in #6129
- [ASM] [HIP] [MLA v4 nm] Split plan helper and persistent decode kernel with in-kernel split and merge by @yhl-amd in #6126
- [FlyDSL] MegaMoEV2: decode/prefill optimizations and Stage1 LDS broadcast race fix by @yhl-amd in #6127
- [HIP] [CK] Revert " Add relu2 (Squared ReLU) activation support to CK-Tile fused MoE GEMM (#5708)" by @shantipriya-amd in #6142
- [Triton/Gluon] fix(gluon): harden paged MQA logits prefetch masks (context_length + block-table capacity bounds) by @fallow5 in #5600
- [Triton/Gluon] DSR1 GEMM tuning by @azaidy in #5867
- [Triton/Gluon] add multicast support to gfx1250 MXFP4 by @Boss2002n in #6118
- [Triton/Gluon] Fix Triton Conv2D/Conv3D caching for inference tensor weights by @saeid-rostami in #6092
- [Triton/Gluon] Add lds-pipelined MLA variant by @AlexAUT in #6098
- [Triton/Gluon] Add per token scale to a8w8 moe gemm by @nsusanto in #6106
- [Triton/Gluon] Add gluon support for MXFP4 and MXFP8 quant kernel in gfx950 by @NimitPtl in #4383
- [Triton/Gluon] [Bench] Fix bench_moe_gemm_int8_smoothquant and bench_moe_gemm_a8w8_blockscale by @zhanglx13 in #6071
- [Triton/Gluon] [Bench] Remove bench_gemm_afp4wfp4_pre_quant_atomic by @zhanglx13 in #6073
- [Triton/Gluon] [Config] : Add Inkling Model GEMM Configs by @rahulbatra85 in #5922
- [CK] Add dsv41 some MoE tuning configs by @kkHuang-amd in #5967
- [Triton/Gluon] Satya/dsr1 tuning oct4th by @Boss2002n in #6157
- [HIP] [FlyDSL] feat(flydsl): row-group FP4 paged MQA logits for 8- and 64-row pages by @valarLip in #6145
- perf(ci): init each multi-GPU test's process group once, not per case by @valarLip in #6163
- [Triton/Gluon] [gfx942] Enable sparse_mla_fwd on gfx942 by @jin-amd in #5721
- [Config] feat(configs): DeepSeek-V4.1-Flash TP4 bf16 GEMMs in the dsv41 bf16 table by @valarLip in #6174
- [Triton/Gluon] Remove bad moe gemm wrapper code by @lburzawa in #6035
- [Triton/Gluon] [CI] [Kimi-K3] Optimize speculative KDA decode on gfx950 by @xiaohuguo2023 in #5709
- [Triton/Gluon] [Config] k3 gfx950 tuning bf16 tp8 shapes by @nsusanto in #6019
- [Triton/Gluon] Gluon PA decode: fix VGPR blow-up on Triton 3.8 by @sshlyapn in #5909
- [CI] Run full AITER releases weekly at 1 a.m. Central by @vgokhale in #6182
- [Triton/Gluon] Fix MXFP4 and MXFP8 fallback for older Triton by @vgokhale in #6181
- [Triton/Gluon] Fused Qwen3-Next GDN decode op for gfx950 by @rbrugaro-amd in #5520
- [Triton/Gluon] unified_attention: mask V load and output store by value head size by @reger-men in #3698
- [Triton/Gluon] [Test] Add SDXL 1.0 conv2d shapes to conv_shapes.json by @Ragua1 in #4444
- [Triton/Gluon] Add gluon support for MXFP4 and MXFP8 quant kernel in gfx1250 by @NimitPtl in #4894
- [CI] Add stale pull request label by @Boss2002n in #5540
- [Triton/Gluon] [Config] triton: add Qwen3.8-27B MXFP4 GDN in_proj_ba AFP4WFP4 tiles (gfx950) by @abrahamzewoudie in #5669
- [HIP] Fix indexer_k_quant_and_cache launching a kernel the wave32 code object does not define by @jiaryang in #6166
- [FlyDSL] gfx942 a16w4: faster MXFP4 unpack via fp8 lookup + hardware convert by @msaffari-amd in #6144
- [HIP] [JIT] Split mla_reduce launcher instantiation across TUs (121.6s -> 7.7s) by @valarLip in #5481
- [Triton/Gluon] [Bugfix] Honor padding_value in HSTU reference helper by @adenzhou1350 in #6041
- [Triton/Gluon] Make scaleM_pad a runtime argument in the act_mul + MXFP4 quant kernels by @prashant182 in #5927
- [Triton/Gluon] GFX950 sparse mla optimizations by @cagrikymk in #6159
- [Triton/Gluon] [Bugfix] fused_reduce_rms_fp8_group_quant: mask inp2 columns with N2, not N1 by @truong-v in #6096
- [Triton/Gluon] [Config] Add gfx950 preshuffled AFP4WFP4 GEMM configs for the Qwen3.8-Flash-Next GDN projections at TP1/2/4/8 by @mjkvaak-amd in #6039
- [Triton/Gluon] [Config] [GFX12] MOE tune by @k50112113 in #6117
- [Triton/Gluon] Gluon PA decode: fix causal mask across splits by @sshlyapn in #5915
- [Triton/Gluon] PA decode: config-driven v1/v2 dispatch, gfx950 always v2 wh… by @NimitPtl in #5859
- [Config] gfx950 Kimi-K3 ptpc bpreshuffle GEMM for KDA in_proj (N=6288, K=7168) by @seungrokj in #6227
- Add GLM-5 MXFP4 EP4 decode fused-MoE rows by @xiaofei-zheng in #5986
- [Config] gfx950 Kimi K3 KDA in_proj bf16 GEMM tuning by @nsusanto in #6224
- [HIP] [JIT] Add Windows support by @0xDELUXA in #2699
- [HIP] [CK] [FlyDSL] Add scatter epilog to layout-v2 GEMM2 (persist-flat, non-atomic store + scatter_reduce) by @charlieguo1106 in #5812
- [Triton/Gluon] gfx1250 MXFP4 preshuffle GEMM: fuse TDM loads explicitly by @Boss2002n in #6231
- [Triton/Gluon] [Config] Fix docs and remove Kpack from 1250 configs by @Boss2002n in #5964
- [CI] Satya/stale branch archive by @Boss2002n in #5617
- [HIP] fix(sampling): drop unused out_idx workspace in topk_renorm_from_probs by @peizhang56 in #5191
- [CI] Use ci:multi-gpu label in welcome comment by @gyohuangxin in #6243
- [HIP] [OPUS] Use gfx1250 320KB LDS for large-D SiTUv2 quant by @XiaobingSuper in #6078
- [HIP] [OPUS] [Perf] Route gfx950 large-expert decode to multi-phase MoE sorting by @xiaofei-zheng in #5987
- [HIP] [Bugfix] Make get_ps_metadata_v1 transfers stream-ordered by @billishyahao in #6040
- [CI] CI: reduce split test timing update conflicts by @gyohuangxin in #6246
- [FlyDSL] Single-launch Mega-mHC on gfx950 by @anhminhnguyenhoang in #6179
- [Triton/Gluon] : Move Grouped MatMul into gemm/grouped by @rahulbatra85 in #5968
- [Triton/Gluon] [Config] [Perf] [gfx950] Tune afp4wfp4 decode configs for Llama-3.1-8B dense shapes by @reger-men in #6015
- [Triton/Gluon] [Config] [GFX950] Unified Attention configs tuning by @leonling-ll in #5432
- [Triton/Gluon] [tune][gfx1100] by @jasberc in #5333
- [Triton/Gluon] Opt attention kernels into FP fusion explicitly by @vgokhale in #6183
- [Docs] Add Triton PR Checklist by @brunomazzottiamd in #6102
- [Triton/Gluon] [gfx950] Add fused Q/K norm, RoPE, gate, and FP8 quantization by @nholmber in #5002
- [HIP] [MoE] [gfx950] Add MiniMax-M3 MXFP4 tuned FMOE configs across TP=2 and shape guards by @amd-bishwoadhikari in #6085
- [CI] [Tune] Add get_GEMM_A16W16_tuned_config, returning None when no tuned row exists by @amd-sriram in #6202
- [HIP] [Hardware] Enable gfx1250 sampling under cpp_itfs by @fsx950223 in #6253
- [CK] Repin CK submodule to fetchable develop commit (relu2/half_t fix) by @shantipriya-amd in #6178
- [ASM] [HIP] Optimize gfx950 MXFP GEMMs for Wan shapes by @ksikiric in #6256
- [Config] [Tuning] Add tuned grouped MoE config for Qwen3.5 FP4 on gfx1250 by @mqhc2020 in #6090
- [HIP] [JIT] Add M3 sequence-parallel collectives by @whx-sjtu in #6031
- [HIP] Optional sigmoid on gated_rmsnorm_fp8_per_token_quant by @rebklee in #5392
- [Triton/Gluon] [Config] Route GLM-5 qkv_a_proj decode tiers to the triton split-K kernel on gfx950 by @xiaofei-zheng in #5850
- [CI] Revert " ci: temporarily skip dependency check" by @gyohuangxin in #6331
- [Triton/Gluon] fused_qk_rope_cat_and_cache_mla: add a NoPE variant by @amd-mvarjoka in #5980
- [Config] remove duplicated csv row by @yadaish in #6332
- [Triton/Gluon] [Config] Tune gfx1250 BF16 GEMM for N=16, K=7168 by @AlexAUT in #6333
- [Triton/Gluon] Seed gfx1150 (RDNA3.5) configs from gfx1151 by @Ragua1 in #5997
- [HIP] [OPUS] [FlyDSL] Packed-row FP4 MQA logits / decode top-k, split-K gfx950 sparse prefill by @valarLip in #6334
- [CI] review-pr: let a review finish, report honestly, and bring the runner back by itself by @zufayu in #5803
- [HIP] [MoE] topk_gating: fuse sigmoid shared-expert suffix by @mjkvaak-amd in #6306
- [ASM] [Bugfix] Fix gfx942 BF16 MLA KV addressing beyond 4 GiB by @alexioslyrakis-amd in #6223
- [OPUS] [Test] MQA logits MXFP4 op test: configurable shapes by @shay-li77 in #6313
- [ASM] [HIP] [gfx950] asm mla: non-persistent fp8 gqa64 qlen1 kernel by @shay-li77 in #6132
- [JIT] Name dedup key and env var in tuned-config duplicate error by @yzhou103 in #6307
- [CI] ci: remove Kimi-K2.5 tests by @gyohuangxin in #6361
- [HIP] [Perf] Optionally emit fp8-quantized Q from fused_qk_norm_rope_cache_pts_quant_shuffle by @mpashkovskii in #5914
- [Config] config: GLM-5.3-Flash fused shared MoE rows for gfx942 by @jin-amd in #5953
- [FlyDSL] [CI] megamoe tune kimi-k3 by @Yaowu-Xiong in #6259
- [HIP] [JIT] [Build] Export ctypes entry points on Windows and add a JIT op smoke test by @0xDELUXA in #6273
- [CI] Pin review bot workflow actions by @gyohuangxin in #6367
- [FlyDSL] Modularize MI308 MoE kernels (1/3) by @luocheng25 in #5795
- [Triton/Gluon] [FlyDSL] [CI] Refactor PA decode, direct native autotune, and BF16 KV by @fsx950223 in #5809
- [FlyDSL] [CI] add mega_moe_tp support for gfx950 by @binding7012 in #5937
- [Triton/Gluon] [Perf] Publish gfx1250 FlashKDA K2 schedule by @XiaobingSuper in #6130
- [FlyDSL] Clean attention, compressor, and recurrent kernel APIs by @coderfeli in #6378
- [FlyDSL] [MoE] Extend fused G2L LUT to 1024 experts by @XiaobingSuper in #6115
- [FlyDSL] Update skills and clean up codes about typed memory and pointer APIs in communication kernels by @coderfeli in #6377
- [Config] [Perf] Add gfx950 bf16 GEMM configs for Qwen3.8-Flash-Next by @mjkvaak-amd in #6201
- [CI][Triton/Gluon] Fix pa_prefill default device leak by @gyohuangxin in #6406
- [Triton/Gluon] [MLA] gfx1250 head-tiled prestored q_pe rope + bf16-round fp8 GEMM epilogue by @karverma-amd in #6288
New Contributors
- @phambinhfin made their first contribution in #5585
- @hyjuunn made their first contribution in #5917
- @Emmanuel0612 made their first contribution in #5652
- @cpersson-amd made their first contribution in #5904
- @fhuizing made their first contribution in #4221
- @GodRishUniverse made their first contribution in #5790
- @jjuvonen-amd made their first contribution in #5230
- @prashant182 made their first contribution in #5926
- @fallow5 made their first contribution in #5600
- @AlexAUT made their first contribution in #6098
- @sshlyapn made their first contribution in #5909
- @reger-men made their first contribution in #3698
- @abrahamzewoudie made their first contribution in #5669
- @adenzhou1350 made their first contribution in #6041
- @truong-v made their first contribution in #6096
- @seungrokj made their first contribution in #6227
- @billishyahao made their first contribution in #6040
- @jasberc made their first contribution in #5333
- @amd-sriram made their first contribution in #6202
- @Yaowu-Xiong made their first contribution in #6259
Full Changelog: v0.1.24...v0.1.25