FlagGems-vllm v0.2.0

FlagGems-vllm v0.2.0

Part of FlagOS 2.2.

Changes since v0.1.0 (FlagOS 2.1).

188 pull requests and 4 additional commits.

Source: v0.2.0, commit b5c2c2249319.

New Features

• Add FlashMLA-style TLE split-kv path (#12) by @wuhulalala
• Add parallel_nsa_compression op with tests and benchmark (#20) by @YY4994
• Add chunk KDA operator (#21) by @kuoihao
• Add chunk gla operator (#22) by @czy1101
• Add TLE chunk KDA FLA operator (#23) by @kuoihao
• add fused_indexer_q_rope_quant op (#26) by @huangyiqun
• Add stage deepseek v4 mega moe inputs (#27) by @huangyiqun
• Add parallel_nsa op with tests and benchmark (#29) by @YY4994
• add some ops (#35) by @huangyiqun
• add gdn2 chunk_fwd operator (#38) by @Ghd-2077
• add router_gemm (#41) by @huangyiqun
• add conf/operators.yaml (#42) by @huangyiqun
• [KernelGen] Add triton_unified_attention Triton kernel (#44) by @Yangyang0906C
• feat: port run_tests.py and reporting tools from FlagGems (#49) by @wuwentao
• Add multi backends ci (#51) by @huangyiqun
• Add multi backends ci (#52) by @huangyiqun
• Add multi backends ci (#53) by @huangyiqun
• Add multi backends ci (#54) by @huangyiqun
• [KMCompiler] Add persistent topk operator (#56) by @LittleShun1214
• Add .claude in ignore (#61) by @0x45f
• Add scaled mm (#64) by @0x45f
• feat: add scaled_int8_quant operator with single-read dynamic quant (#66) by @chx7514
• feat(metax): add scaled_int8_quant operator for MetaX C550 (#67) by @chx7514
• feat(hygon): add scaled_int8_quant operator for Hygon BW (#68) by @chx7514
• feat(mthreads): add scaled_int8_quant operator for Moore Threads S5000 (#69) by @chx7514
• feat(iluvatar): add scaled_int8_quant operator for Iluvatar BI-V150 (Tianshu) (#70) by @chx7514
• feat(enflame): add scaled_int8_quant operator for Enflame ZW810 (PPU) (#71) by @chx7514
• feat(ascend): add scaled_int8_quant operator for Ascend 910B (#72) by @chx7514
• feat(kunlunxin): add scaled_int8_quant operator for Kunlunxin (#73) by @chx7514
• add CLA (#86) by @huangyiqun

48 more changes

• [KernelGen][Hygon] Add triton_scaled_mm Hygon specialized operator (#90) by @xzliu-opt
• [KernelGen][Ascend] Add fused_experts_impl specialized operator (#95) by @liuhycs
• [KernelGen][MUSA] Add fused_experts_impl specialized operator (#96) by @liuhycs
• [KernelGen][Hygon] Add fused_experts_impl specialized operator (#97) by @liuhycs
• [KernelGen][Tianshu] Add fused_experts_impl specialized operator (#98) by @liuhycs
• feat: optimize w8a8 block fp8 matmul for Hopper (#693) by @zikong20
• add @Vincent-Xiao to CODEOWNERS (#701) by @huangyiqun
• Add Qwen3.8-Flash-Next self-developed Triton kernels and vendor baselines (#713) by @CherryLemon
• [Operator][Ascend][XCS] Add OP swiglu using TLE (#728) by @neoskypig
• [Operator][Ascend][XCS] Add op causal_conv1d_update (#729) by @finneyyan
• [Operator][Ascend][XCS] Add ascend op: causal_conv1d_fn (#730) by @finneyyan
• [Operator][Ascend][XCS] Add op add_rms_norm (#731) by @finneyyan
• [AdvancedCompiler]add hygon fused_add_rms_norm (#736) by @zhoubo567
• [HYGON] Add moe_sum (#745) by @laylowch
• [KMCompiler] Feat/hygon fused moe (#746) by @LittleShun1214
• [QC] Add FP8 W8A8 variable-length FlashAttention-2 (#749) by @ggbondbest
• [QC] Add INT8 and FP8 W8A16 fused Marlin MoE (#750) by @ggbondbest
• [KMCompiler] Add and register thead fused persistent_topk (PPU-ZW810E) (#754) by @LittleShun1214
• [KMCompiler] Add Hygon persistent_topk (#756) by @LittleShun1214
• [KMCompiler] Add mthreads (MTT S5000) persistent_topk backend (#758) by @LittleShun1214
• Add gemma rmsnorm ascend (#762) by @songdejun
• [TLE][ASCEND]Add moe_sum (#763) by @laylowch
• Add gemma rmsnorm hygon (#765) by @songdejun
• Add gemma rmsnorm thead (#766) by @songdejun
• Add gemma rmsnorm (#767) by @songdejun
• [QC][Nvidia] Add fp8_einsum with tests and vLLM benchmarks (Migrated from FlagGems) (#769) by @monellz
• [THEAD] Add moe_sum (#772) by @laylowch
• [KMCompiler] Add Ascend persistent_topk implementation (#781) by @LittleShun1214
• add gemma_rmsnorm for mthreads platform (#782) by @songdejun
• [Operator][Ascend][XCS] add chunk_gated_delta_rule_fwd op (#788) by @FatJhon
• [TLE][Ascend][XCS] feat: add compressor operator (#789) by @Quamaly
• [TLE][Ascend][XCS] feat: add kv_rmsnorm_rope_cache (#790) by @Quamaly
• [TLE][Ascend][XCS] feat: add lightning indexer operator (#791) by @Quamaly
• [TLE][Ascend][XCS] feat: add sparse_attn_sharedkv operator (#792) by @Quamaly
• [QC][PPU] Add fused_inv_rope_fp8_quant operator (#796) by @July-h5kf3
• [QC][Hygon] Add fused Marlin MoE for INT4, INT8, FP8 and MXFP4 (#797) by @July-h5kf3
• [KMCompiler][Ascend] Add deepseek_v4 combine_topk_swa_indices operator and fix the vLLM op import in its tests (#800) by @LittleShun1214
• [QC][Hygon] Add fused_inv_rope_fp8_quant operator (#801) by @July-h5kf3
• [QC][Hygon] Add INT8 block-wise BMM kernel and int8_einsum (bhr,hdr->bhd) (#803) by @July-h5kf3
• [KMCompiler][Ascend] Add top_k_per_row_prefill and top_k_per_row_decode implementation on Ascend platform. (#806) by @Caeruleann
• Add Ascend TLE-optimized topk_softplus_sqrt kernel (#821) by @ph0375
• [QC][MetaX] Add and optimize fused_inv_rope_fp8_quant (#826) by @FENGZIYI0711
• [KMCompiler] feat(persistent_topk): add MetaX (MC550) backend (#834) by @LittleShun1214
• [QC][MThreads] Add shared INT4 and FP8 fused Marlin MoE (#835) by @ggbondbest
• Add mthreads TLE-optimized topk_softplus_sqrt kernel (#838) by @ph0375
• Add Thead TLE-optimized topk_softplus_sqrt kernel (#845) by @ph0375
• Add hygon TLE-optimized topk_softplus_sqrt kernel ([#846](https://gi...

v0.1.1-rc0

Fix enflame scaled_int8_quant (#101)

FlagOS 2.1 — FlagGems-vllm v0.1.0

Release v0.1.0

Initial Release

CI/Infrastructure

• [CI/CD] Add basic CI (#8) by @douxetpur
• Fix select corresponding tests (#25) by @douxetpur

Other

• Optimize flashmla_sparse (#1) by @sunnycase
• Inline flashmla_sparse launch wrappers (#2) by @sunnycase
• Fix code style (#5) by @huangyiqun
• Add some OPs (#7) by @huangyiqun
• Fix_tests_benchmark (#10) by @huangyiqun
• Simplify code style CI checks (#17) by @douxetpur
• Update tests & benchmark (#18) by @huangyiqun
• Optimize chunk_gated_delta_rule (#15) by @sunnycase
• Add FlashMLA-style TLE split-kv path (#12) by @wuhulalala
• Add chunk KDA operator (#21) by @kuoihao
• Add TLE chunk KDA FLA operator (#23) by @kuoihao
• Add fused_indexer_q_rope_quant op (#26) by @huangyiqun
• Add parallel_nsa_compression op with tests and benchmark (#20) by @YY4994
• Add chunk gla operator (#22) by @czy1101

Contributors

Thanks to all contributors who made this release possible:

• @huangyiqun (5 PRs)
• @douxetpur (3 PRs)
• @sunnycase (3 PRs)
• @kuoihao (2 PRs)
• @YY4994
• @czy1101
• @wuhulalala

---

Generated by Release Notes Generator