Skip to content

Release v0.2.5

Choose a tag to compare

View all tags
@zhe-pang zhe-pang released this 02 Aug 06:45
· 8 commits to main since this release
v0.2.5
e3e10c3

Highlights

This release significantly expands attention operator coverage, improves MoE mixed-precision support, enhances model compatibility for FlashMLA and KDA, and introduces the optional mate-mubin package for managing assembly implementations. It also improves FMHA Forward/Backward compatibility with FA3 and adds support for additional model inference scenarios.

What's Changed

FMHA Updates

Expanded FMHA Forward and Backward functionality with improved FA3 compatibility.

FMHA Forward

Added new features and optimizations:

  • Added QV-only mode support.
    • Compatible with the SGLang FA interface.
  • Added FP8 E5M2 specialization support.
  • Optimized split count selection for selected workloads.

FMHA Backward

Expanded FMHA Backward coverage to fully support FA3 backward features.

Added support for:

  • Softcap
  • Sliding Window Attention (SWA)
  • Deterministic mode

Mixed Precision MoE GEMM

Extended mixed-precision MoE GEMM support.

  • Added FP4FP8 mode for MixedDType MoE GEMM.
  • Enables MXFP4 input support under specific dynamic range constraints.

DeepGEMM Updates

Improved DeepGEMM functionality and stability.

  • Added BF16 output support for k_contiguous.
  • Fixed hangs under certain scenarios.

FlashMLA Updates

Enhanced FlashMLA dense and sparse compatibility.

  • Improved compatibility with additional official FlashMLA test scenarios.

KDA Updates

Expanded KDA support and improved stability.

  • Added KDA Decode support.
  • Fixed several KDA Prefill issues.
  • Added support for KIMI K3 model inference.

MSA Support

Added MSA support and a corresponding wrapper.

  • Added MSA implementation.
  • Added MSA Wrapper.
  • Enabled MiniMax M3 model inference support.

MATE Mubin Package

Introduced the optional mate-mubin package for managing assembly implementations.

  • If mate-mubin is not installed, assembly kernels will be downloaded automatically at runtime.
  • For offline environments or scenarios requiring reduced startup overhead, mate-mubin can be installed in advance.

Installation options:

# Install via pip
pip install mate-mubin --index-url https://dl.mthreads.com/repo/api/pypi/pypi/simple

# Install via mate-cli
mate install-mubin-wheel
Assets 2
Loading