Skip to content

AITER v0.1.25

Choose a tag to compare

@github-actions github-actions released this 12 Oct 08:52
c53e9b3

AITER v0.1.25 - weekly release

Scheduled release from release/v0.1.25.

Diff base: v0.1.24

Wheels

Prebuilt manylinux_2_28 wheels, GPU_ARCHS=gfx942;gfx950:

ROCm Python Wheel
ROCm 7.0 cp310 amd_aiter-0.1.25+rocm7.0.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.0 cp312 amd_aiter-0.1.25+rocm7.0.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.1 cp310 amd_aiter-0.1.25+rocm7.1.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.1 cp312 amd_aiter-0.1.25+rocm7.1.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.2 cp310 amd_aiter-0.1.25+rocm7.2.manylinux.2.28-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
ROCm 7.2 cp312 amd_aiter-0.1.25+rocm7.2.manylinux.2.28-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
What's Changed

Changes

  • [Config] Add gfx942 a8w8 blockscale GEMM tunings for Qwen3/Qwen3.5/GLM/DSV4 shapes by @siliangchen-amd in #5839
  • [HIP] [OPUS] [FlyDSL] gfx950 MXFP8 e8m0 GEMM + DeepSeek-V4/V4.1 tuned configs by @valarLip in #5896
  • [Triton/Gluon] [ASM] [HIP] Mha v4: adds bf16 sparse, LSE support, KV varlen, fixes, etc by @jcaraban in #5798
  • [tuner] Fail a task as soon as its worker process exits by @siliangchen-amd in #5841
  • [CI] Update Aiter artifact downloads to v8.0.1 by @leo-automation in #5908
  • [MLA v4 nm] Test fix _run_one_point reading packed BF16 as FP32 partials by @liyjiang in #5905
  • [FlyDSL] [JIT] [AOT] Inline the FlyDSL FP8 FMHA head shapes, drop the config CSVs by @gbyu-amd in #5901
  • [FlyDSL] gfx942 fp8_mqa_logits: let _auto_variant choose rows_per_block by @jin-amd in #4963
  • [Triton/Gluon] [Config] Add a tuned gfx1101 MHA config, split small_head/default by @Ragua1 in #4493
  • [Triton/Gluon] Consolidate tuning harnesses by @Boss2002n in #5874
  • [Triton/Gluon] [Bugfix][MLA] Follow-up: fix stale comments + test hardening (#5648) by @Rohan138 in #5860
  • [Triton/Gluon] [Config] gemm_a16w16: gfx950 tuned per-shape configs by @NimitPtl in #5830
  • [Triton/Gluon] [gfx950] [dsv4.1-flash] optimizing the mHC fused kernel by @ahmed-bsod in #5885
  • [HIP] Tune DSV4.1 TP4 A4W4 fused MoE configs for gfx950 by @yifehuan in #5919
  • [Config] Add Qwen3.8-27B TP1 a8w8 blockscale GEMM tunings for gfx942 by @phambinhfin in #5585
  • [HIP] [OPUS] [JIT] Unify gfx950+gfx1250 MXFP4 MQA-logits by @shay-li77 in #5761
  • [FlyDSL] feat(mega_moe/gfx1250): bind mori tokoff-ext allocator on the mori dispatch path by @jhchouuu in #5810
  • [Bugfix] Do not write segment softmax state at NUM_SEGMENTS_PER_SEQ == 1 by @sogalin in #5935
  • [Triton/Gluon] [Config] Enable unified-attention skip-mask for gfx950 hd256 FP8 prefill by @vorapolsiloai in #5598
  • [Triton/Gluon] [Kimi-K3][ROCm] Add merged MoE front by @jiacao-amd in #5321
  • [Triton/Gluon] Add SonicMoE pure-Triton grouped GEMM MoE by @WuLei-AMD in #5725
  • [Triton/Gluon] Move the KDA_DECODE configs into the nested config layout by @Boss2002n in #5941
  • [Triton/Gluon] Move Triton unit tests into their wrapper's folder by @Boss2002n in #5943
  • [Triton/Gluon] Move _gluon_kernels/gfx1250/norm/ to _gluon_kernels/gfx1250/normalization/ by @Boss2002n in #5944
  • [CI] Select impacted Triton and Gluon unit tests by @Boss2002n in #5878
  • [Triton/Gluon] [Config] Drop no-op kpack from RDNA GEMM configs by @hyjuunn in #5917
  • [Triton/Gluon] [GFX12] mxfp8 gemm cga update by @k50112113 in #5055
  • [Triton/Gluon] stop fused_bmm_rope_kv_cache from using batched_gemm_a8w8_a_per_token… by @Boss2002n in #5877
  • [Triton/Gluon] Add Triton-based Conv3D kernels by @saeid-rostami in #5952
  • [HIP] [Perf] Fold shared-expert gate GEMV into topk_softmax kernel by @Emmanuel0612 in #5652
  • [Config] Add Qwen3.8-27B gfx950 A4W4 GDN BA projection shapes by @vorapolsiloai in #5836
  • [CI] Preserve install failure status after retries by @vgokhale in #5864
  • [Config] Add Qwen3.8-Flash-Next PTPC-FP8 gfx942 a8w8 bpreshuffle and BF16 GEMM tunings by @zovonoir in #5974
  • [ASM] [HIP] Add gfx950 MXFP4 SiTUv2 FLAT 16x192/16x128/16x32 kernels by @JohnNikolay84 in #5763
  • [Config] Add tuned a8w4 fused-MoE configs for DeepSeek V4.1-Flash (TP4 + EP, gfx950) by @cpersson-amd in #5904
  • [Config] Tune a8w8 bpreshuffle for Kimi-K3's KDA gate at tp4 by @XiaobingSuper in #5783
  • [FlyDSL] Paged mla indexer by @fhuizing in #4221
  • [Triton/Gluon] [GFX950] Fix LDS OOM on mxfp8 triton gemm by @k50112113 in #5963
  • [CI] Triton board: drive PR status from the workflow by @Boss2002n in #5880
  • [Triton/Gluon] Move Triton unit tests into their wrapper's folder by @Boss2002n in #5942
  • [Triton/Gluon] [Kernel] Add fused SiLU-and-multiply backward by @DaiXindi-AMD in #5542
  • [Triton/Gluon] Unified attention sglang compatability fix by @cagrikymk in #6009
  • [OPUS] Speed up the gfx1250 32mx1 MLA sparse-prefill kernels by @kaiyang-1 in #5805
  • [Config] add DSR1 gfx1250 bf16 GEMM tuning by @demonsan in #5939
  • [HIP] [FlyDSL] [JIT] [gfx1250][GEMM] add MXFP8 1x32 bpreshuffle GEMM with tuned ASM/FlyDSL dispatch by @aoli26 in #5818
  • [ASM] [HIP] [FlyDSL] Add pin gpr(nhead96/128) and round-robin DCP to persistent MLA PS1 FP8 decode for GFX1250 by @xiangM99 in #5844
  • [HIP] [CK] Add relu2 (Squared ReLU) activation support to CK-Tile fused MoE GEMM by @shantipriya-amd in #5708
  • [Triton/Gluon] [Config] Add gfx950 preshuffled AFP4WFP4 GEMM config for N=4608, K=8192 by @mjkvaak-amd in #5998
  • [Triton/Gluon] Fix pa_decode compile error with newer Triton: don't index scalar alpha by @zhanglx13 in #5886
  • [Triton/Gluon] [Config] Add gfx1250 A16W16 GEMM configs for Wan2.2 shapes by @sogalin in #6032
  • [Triton/Gluon] Assert SPLIT_UNMASKED_LOOP is not used with sliding window by @amd-xavierwang in #5771
  • [Triton/Gluon] Fix fp8 KV correctness reference in bench_unified_attention by @mpashkovskii in #5663
  • [Triton/Gluon] [Config] Tune gfx1201 unified attention 2d for D=512 for full attention and D=256 for Sliding Window (Gemma4 shapes) and fix D=1024 launches by @GodRishUniverse in #5790
  • [Triton/Gluon] Account for sliding_window in bench_unified_attention metrics by @mpashkovskii in #5665
  • [Triton/Gluon] Skip MXFP4 logits tests on unsupported GPUs by @brunomazzottiamd in #5958
  • [Triton/Gluon] Fix Conv3D CPU tests with a CUDA default device by @whx-sjtu in #5984
  • [Triton/Gluon] Use a zero default for masked scales in gemm_afp8wfp8 by @cagrikymk in #6007
  • [Triton/Gluon] gemm_a16w16_atomic accumulate into an existing output buffer by @nsusanto in #5895
  • [Triton/Gluon] MHA fwd: gfx950 mid_head config for 64 < head_dim <= 128 by @NimitPtl in #5858
  • [Triton/Gluon] gmm: gfx950 large-K/N config with JSON-driven shape dispatch by @NimitPtl in #5857
  • [Triton/Gluon] Move Triton unit tests into their wrapper's folder by @Boss2002n in #5940
  • [Triton/Gluon] Expert-parallel support for moe_gemm_a16w4 by @Rohan138 in #5863
  • [Triton/Gluon] Add 3D neighborhood flash attention (na3d_flash) by @jjuvonen-amd in #5230
  • [CK] [Bugfix] Keep two K tiles per split in blockscale MoE stage-1 split-K by @jin-amd in #5979
  • [Triton/Gluon] tuning k3 shapes for a16w16 by @omuhamma in #5875
  • [Triton/Gluon] [Config] Add gfx950 gemm_a16w16 config for N=288, K=4096 by @Jacob0226 in #5933
  • [FlyDSL] Dev/a4w4 prefill v2 yadai by @yadaish in #5934
  • [CK] [FlyDSL] add mxfp8 a8w8 blockscale for gfx950 by @solinzby1 in #4951
  • fix topk_select restriction by @HaonanWang98 in #6080
  • [Triton/Gluon] [HIP] gemm_a16w16: gfx950 config for MiniMax-M3 lm_head (N=50016, K=6144) at small M by @valarLip in #6079
  • [Triton/Gluon] fix gluon import and benchmark + tune fused_clamp_act_mul for gfx950 by @nidal567 in #5800
  • [Triton/Gluon] [Config] gfx950 unified attention prefill tile for head_dim 512 (Gemma 4 global layers) by @prashant182 in #5926
  • [Triton/Gluon] spec decode support for gluon kda files by @omuhamma in #6034
  • [Triton/Gluon] Gluon Chunked Kimi Delta Attention (950/1250) by @omuhamma in #5866
  • [Triton/Gluon] Changes related to gla for Kimi K3 by @omuhamma in #6056
  • [Triton/Gluon] enable TDM fusion on MXFP4 by @Boss2002n in #6097
  • [Triton/Gluon] [Bugfix] batched_gemm_a8w8: keep scale loads in bounds by @akii96 in #6082
  • [ASM] [CI] Remove committed scratch files and stop two from coming back by @Boss2002n in #5544
  • [Triton/Gluon] Add a16w4 MoE tuned dispatch and gfx942 DSv4.1 tables by @akii96 in #5667
  • [ASM][flydsl] opt mla kernel for dspr1(gfx1250) by @xiangM99 in #6087
  • [Triton/Gluon] [Perf] MXFP8 MoE: keep the MFMA in FP8 when MX_SCALE_BLOCK_K == 1 by @akii96 in #5832
  • [CI] [JIT] Fix ATOM GPU isolation and startup failures by @leo-automation in #5989
  • [HIP] gfx950 A8W4 fused_moe configs for DSv4.1 TP-sharded experts by @valarLip in #6125
  • fix(custom_all_reduce): allocate IPC handle tensors on the host by @valarLip in #6129
  • [ASM] [HIP] [MLA v4 nm] Split plan helper and persistent decode kernel with in-kernel split and merge by @yhl-amd in #6126
  • [FlyDSL] MegaMoEV2: decode/prefill optimizations and Stage1 LDS broadcast race fix by @yhl-amd in #6127
  • [HIP] [CK] Revert " Add relu2 (Squared ReLU) activation support to CK-Tile fused MoE GEMM (#5708)" by @shantipriya-amd in #6142
  • [Triton/Gluon] fix(gluon): harden paged MQA logits prefetch masks (context_length + block-table capacity bounds) by @fallow5 in #5600
  • [Triton/Gluon] DSR1 GEMM tuning by @azaidy in #5867
  • [Triton/Gluon] add multicast support to gfx1250 MXFP4 by @Boss2002n in #6118
  • [Triton/Gluon] Fix Triton Conv2D/Conv3D caching for inference tensor weights by @saeid-rostami in #6092
  • [Triton/Gluon] Add lds-pipelined MLA variant by @AlexAUT in #6098
  • [Triton/Gluon] Add per token scale to a8w8 moe gemm by @nsusanto in #6106
  • [Triton/Gluon] Add gluon support for MXFP4 and MXFP8 quant kernel in gfx950 by @NimitPtl in #4383
  • [Triton/Gluon] [Bench] Fix bench_moe_gemm_int8_smoothquant and bench_moe_gemm_a8w8_blockscale by @zhanglx13 in #6071
  • [Triton/Gluon] [Bench] Remove bench_gemm_afp4wfp4_pre_quant_atomic by @zhanglx13 in #6073
  • [Triton/Gluon] [Config] : Add Inkling Model GEMM Configs by @rahulbatra85 in #5922
  • [CK] Add dsv41 some MoE tuning configs by @kkHuang-amd in #5967
  • [Triton/Gluon] Satya/dsr1 tuning oct4th by @Boss2002n in #6157
  • [HIP] [FlyDSL] feat(flydsl): row-group FP4 paged MQA logits for 8- and 64-row pages by @valarLip in #6145
  • perf(ci): init each multi-GPU test's process group once, not per case by @valarLip in #6163
  • [Triton/Gluon] [gfx942] Enable sparse_mla_fwd on gfx942 by @jin-amd in #5721
  • [Config] feat(configs): DeepSeek-V4.1-Flash TP4 bf16 GEMMs in the dsv41 bf16 table by @valarLip in #6174
  • [Triton/Gluon] Remove bad moe gemm wrapper code by @lburzawa in #6035
  • [Triton/Gluon] [CI] [Kimi-K3] Optimize speculative KDA decode on gfx950 by @xiaohuguo2023 in #5709
  • [Triton/Gluon] [Config] k3 gfx950 tuning bf16 tp8 shapes by @nsusanto in #6019
  • [Triton/Gluon] Gluon PA decode: fix VGPR blow-up on Triton 3.8 by @sshlyapn in #5909
  • [CI] Run full AITER releases weekly at 1 a.m. Central by @vgokhale in #6182
  • [Triton/Gluon] Fix MXFP4 and MXFP8 fallback for older Triton by @vgokhale in #6181
  • [Triton/Gluon] Fused Qwen3-Next GDN decode op for gfx950 by @rbrugaro-amd in #5520
  • [Triton/Gluon] unified_attention: mask V load and output store by value head size by @reger-men in #3698
  • [Triton/Gluon] [Test] Add SDXL 1.0 conv2d shapes to conv_shapes.json by @Ragua1 in #4444
  • [Triton/Gluon] Add gluon support for MXFP4 and MXFP8 quant kernel in gfx1250 by @NimitPtl in #4894
  • [CI] Add stale pull request label by @Boss2002n in #5540
  • [Triton/Gluon] [Config] triton: add Qwen3.8-27B MXFP4 GDN in_proj_ba AFP4WFP4 tiles (gfx950) by @abrahamzewoudie in #5669
  • [HIP] Fix indexer_k_quant_and_cache launching a kernel the wave32 code object does not define by @jiaryang in #6166
  • [FlyDSL] gfx942 a16w4: faster MXFP4 unpack via fp8 lookup + hardware convert by @msaffari-amd in #6144
  • [HIP] [JIT] Split mla_reduce launcher instantiation across TUs (121.6s -> 7.7s) by @valarLip in #5481
  • [Triton/Gluon] [Bugfix] Honor padding_value in HSTU reference helper by @adenzhou1350 in #6041
  • [Triton/Gluon] Make scaleM_pad a runtime argument in the act_mul + MXFP4 quant kernels by @prashant182 in #5927
  • [Triton/Gluon] GFX950 sparse mla optimizations by @cagrikymk in #6159
  • [Triton/Gluon] [Bugfix] fused_reduce_rms_fp8_group_quant: mask inp2 columns with N2, not N1 by @truong-v in #6096
  • [Triton/Gluon] [Config] Add gfx950 preshuffled AFP4WFP4 GEMM configs for the Qwen3.8-Flash-Next GDN projections at TP1/2/4/8 by @mjkvaak-amd in #6039
  • [Triton/Gluon] [Config] [GFX12] MOE tune by @k50112113 in #6117
  • [Triton/Gluon] Gluon PA decode: fix causal mask across splits by @sshlyapn in #5915
  • [Triton/Gluon] PA decode: config-driven v1/v2 dispatch, gfx950 always v2 wh… by @NimitPtl in #5859
  • [Config] gfx950 Kimi-K3 ptpc bpreshuffle GEMM for KDA in_proj (N=6288, K=7168) by @seungrokj in #6227
  • Add GLM-5 MXFP4 EP4 decode fused-MoE rows by @xiaofei-zheng in #5986
  • [Config] gfx950 Kimi K3 KDA in_proj bf16 GEMM tuning by @nsusanto in #6224
  • [HIP] [JIT] Add Windows support by @0xDELUXA in #2699
  • [HIP] [CK] [FlyDSL] Add scatter epilog to layout-v2 GEMM2 (persist-flat, non-atomic store + scatter_reduce) by @charlieguo1106 in #5812
  • [Triton/Gluon] gfx1250 MXFP4 preshuffle GEMM: fuse TDM loads explicitly by @Boss2002n in #6231
  • [Triton/Gluon] [Config] Fix docs and remove Kpack from 1250 configs by @Boss2002n in #5964
  • [CI] Satya/stale branch archive by @Boss2002n in #5617
  • [HIP] fix(sampling): drop unused out_idx workspace in topk_renorm_from_probs by @peizhang56 in #5191
  • [CI] Use ci:multi-gpu label in welcome comment by @gyohuangxin in #6243
  • [HIP] [OPUS] Use gfx1250 320KB LDS for large-D SiTUv2 quant by @XiaobingSuper in #6078
  • [HIP] [OPUS] [Perf] Route gfx950 large-expert decode to multi-phase MoE sorting by @xiaofei-zheng in #5987
  • [HIP] [Bugfix] Make get_ps_metadata_v1 transfers stream-ordered by @billishyahao in #6040
  • [CI] CI: reduce split test timing update conflicts by @gyohuangxin in #6246
  • [FlyDSL] Single-launch Mega-mHC on gfx950 by @anhminhnguyenhoang in #6179
  • [Triton/Gluon] : Move Grouped MatMul into gemm/grouped by @rahulbatra85 in #5968
  • [Triton/Gluon] [Config] [Perf] [gfx950] Tune afp4wfp4 decode configs for Llama-3.1-8B dense shapes by @reger-men in #6015
  • [Triton/Gluon] [Config] [GFX950] Unified Attention configs tuning by @leonling-ll in #5432
  • [Triton/Gluon] [tune][gfx1100] by @jasberc in #5333
  • [Triton/Gluon] Opt attention kernels into FP fusion explicitly by @vgokhale in #6183
  • [Docs] Add Triton PR Checklist by @brunomazzottiamd in #6102
  • [Triton/Gluon] [gfx950] Add fused Q/K norm, RoPE, gate, and FP8 quantization by @nholmber in #5002
  • [HIP] [MoE] [gfx950] Add MiniMax-M3 MXFP4 tuned FMOE configs across TP=2 and shape guards by @amd-bishwoadhikari in #6085
  • [CI] [Tune] Add get_GEMM_A16W16_tuned_config, returning None when no tuned row exists by @amd-sriram in #6202
  • [HIP] [Hardware] Enable gfx1250 sampling under cpp_itfs by @fsx950223 in #6253
  • [CK] Repin CK submodule to fetchable develop commit (relu2/half_t fix) by @shantipriya-amd in #6178
  • [ASM] [HIP] Optimize gfx950 MXFP GEMMs for Wan shapes by @ksikiric in #6256
  • [Config] [Tuning] Add tuned grouped MoE config for Qwen3.5 FP4 on gfx1250 by @mqhc2020 in #6090
  • [HIP] [JIT] Add M3 sequence-parallel collectives by @whx-sjtu in #6031
  • [HIP] Optional sigmoid on gated_rmsnorm_fp8_per_token_quant by @rebklee in #5392
  • [Triton/Gluon] [Config] Route GLM-5 qkv_a_proj decode tiers to the triton split-K kernel on gfx950 by @xiaofei-zheng in #5850
  • [CI] Revert " ci: temporarily skip dependency check" by @gyohuangxin in #6331
  • [Triton/Gluon] fused_qk_rope_cat_and_cache_mla: add a NoPE variant by @amd-mvarjoka in #5980
  • [Config] remove duplicated csv row by @yadaish in #6332
  • [Triton/Gluon] [Config] Tune gfx1250 BF16 GEMM for N=16, K=7168 by @AlexAUT in #6333
  • [Triton/Gluon] Seed gfx1150 (RDNA3.5) configs from gfx1151 by @Ragua1 in #5997
  • [HIP] [OPUS] [FlyDSL] Packed-row FP4 MQA logits / decode top-k, split-K gfx950 sparse prefill by @valarLip in #6334
  • [CI] review-pr: let a review finish, report honestly, and bring the runner back by itself by @zufayu in #5803
  • [HIP] [MoE] topk_gating: fuse sigmoid shared-expert suffix by @mjkvaak-amd in #6306
  • [ASM] [Bugfix] Fix gfx942 BF16 MLA KV addressing beyond 4 GiB by @alexioslyrakis-amd in #6223
  • [OPUS] [Test] MQA logits MXFP4 op test: configurable shapes by @shay-li77 in #6313
  • [ASM] [HIP] [gfx950] asm mla: non-persistent fp8 gqa64 qlen1 kernel by @shay-li77 in #6132
  • [JIT] Name dedup key and env var in tuned-config duplicate error by @yzhou103 in #6307
  • [CI] ci: remove Kimi-K2.5 tests by @gyohuangxin in #6361
  • [HIP] [Perf] Optionally emit fp8-quantized Q from fused_qk_norm_rope_cache_pts_quant_shuffle by @mpashkovskii in #5914
  • [Config] config: GLM-5.3-Flash fused shared MoE rows for gfx942 by @jin-amd in #5953
  • [FlyDSL] [CI] megamoe tune kimi-k3 by @Yaowu-Xiong in #6259
  • [HIP] [JIT] [Build] Export ctypes entry points on Windows and add a JIT op smoke test by @0xDELUXA in #6273
  • [CI] Pin review bot workflow actions by @gyohuangxin in #6367
  • [FlyDSL] Modularize MI308 MoE kernels (1/3) by @luocheng25 in #5795
  • [Triton/Gluon] [FlyDSL] [CI] Refactor PA decode, direct native autotune, and BF16 KV by @fsx950223 in #5809
  • [FlyDSL] [CI] add mega_moe_tp support for gfx950 by @binding7012 in #5937
  • [Triton/Gluon] [Perf] Publish gfx1250 FlashKDA K2 schedule by @XiaobingSuper in #6130
  • [FlyDSL] Clean attention, compressor, and recurrent kernel APIs by @coderfeli in #6378
  • [FlyDSL] [MoE] Extend fused G2L LUT to 1024 experts by @XiaobingSuper in #6115
  • [FlyDSL] Update skills and clean up codes about typed memory and pointer APIs in communication kernels by @coderfeli in #6377
  • [Config] [Perf] Add gfx950 bf16 GEMM configs for Qwen3.8-Flash-Next by @mjkvaak-amd in #6201
  • [CI][Triton/Gluon] Fix pa_prefill default device leak by @gyohuangxin in #6406
  • [Triton/Gluon] [MLA] gfx1250 head-tiled prestored q_pe rope + bf16-round fp8 GEMM epilogue by @karverma-amd in #6288

New Contributors

Full Changelog: v0.1.24...v0.1.25