Skip to content

Release v0.2.4

Latest
Latest

Choose a tag to compare

View all tags
@zhe-pang zhe-pang released this 13 Jul 09:42
v0.2.4
fa657ba

Highlights

This release expands mixed-precision MoE support with new DeepGEMM APIs, introduces FP8 block-scaling MoE kernels, extends FA3 to support FP8 with QV for FP8 MLA inference, and improves wrapper version management. It also includes multiple stability and correctness fixes across FA3, DSA, DeepGEMM, and KDA.

What's Changed

DeepGEMM Updates

Added new APIs for mixed-precision MoE workloads.

  • Added mixed-input ragged/masked MoE GEMM APIs for mixed-precision DeepGEMM.
  • Currently supports:
    • Weight: S4 block-scaled (1 × 128 granularity)
    • Activation: FP8 (1 × -1 granularity)

Added a new FP8 MoE interface.

  • Added fp8_fp8_mega_moe for FP8 block-scaling MoE.

FMHA Updates

Expanded FA3 support for FP8 inference.

  • Added support for FP8 with QV.
  • Enables FP8 MLA inference scenarios.

Wrapper Updates

Improved wrapper version identification.

  • All wrappers now include a MUSA-specific local version suffix to distinguish them from upstream community releases.

Bug Fixes

Fixed the following issues:

  • Fixed hangs in FA3 backward under certain scenarios.
  • Relaxed overly strict stride validation in DSA for specific use cases.
  • Fixed compilation failures for FP8 FA3 in certain scenarios.
  • Fixed numerical accuracy issues for FP8 FA3 when using Softcap.
  • Fixed out-of-bounds (OOB) issues in DeepGEMM MQA Logits and the MUTLASS backend.
  • Added a workaround for KDA compilation failures with compiler version 4.3.6.
Assets 2
Loading