ZenDNN 6.0.0 — Major Release
ZenDNN 6.0.0 is a major release building on the 5.2.1 runtime architecture. It deepens the Low Overhead API (LowOHA) as the primary inference path, expands MoE group GEMM and FP16 operator coverage, and adds production-grade post-op and weight caching, with corresponding extensions to BenchDNN, gtests, and operator documentation.
Highlights at a Glance
| Area | What's New |
|---|---|
| MoE / Group MatMul | Regime-aware AUTO routing, F16 group matmul, custom DQ kernels, gated activations |
| FP16 Expansion | SDPA, normalization, quant reorder, quant embedding, softmax |
| Caching | Post-op metadata cache, in-place weight caching; reorder & ZP caches on by default |
| Quantization | Vectorized per-group dynamic quant kernel; BenchDNN DQ workloads |
| AutoTuner | Customizable algorithm evaluation sequence |
| BenchDNN | SDPA, normalization, MoE fused inputs, HW perf-counter infra |
| Activations | MISH support; FP32 ↔ FP16/BF16 conversion in reorder |
| Build | CMake minimum raised to 3.26 |
MoE / Group MatMul
- Regime-aware AUTO routing (decode vs. prompt).
- F16 group matmul support.
- Group Dynamic Quant API.
- N-tile DQ-INT8 and M-tile vertical-fusion custom kernels.
- Gated-activation enhancements.
- New runtime knob:
ZENDNNL_GRP_MATMUL_ALGO.
FP16 Expansion
FP16 coverage now includes SDPA, normalization, quant reorder, quant embedding, and softmax, extending the F16 matmul paths already present in 5.2.1 (AOCL DLP and OneDNN).
Caching
- Post-op metadata cache (
ZENDNNL_ENABLE_POSTOP_CACHE). - In-place weight caching (
ZENDNNL_MATMUL_WEIGHT_CACHE=2). - Weight-reorder and ZP-compensation caches are now enabled by default (they were disabled in 5.2.1).
Quantization
- Vectorized per-group dynamic quant kernel.
- BenchDNN dynamic-quantization workloads.
AutoTuner
- The
ZENDNNL_MATMUL_AUTO_ALGO_CANDIDATESenvironment variable customizes the algorithm evaluation sequence.
BenchDNN
- SDPA and normalization workloads.
- MoE fused inputs.
- Group-matmul hardware perf-counter infrastructure.
Activations
- MISH activation support.
- Float conversion paths in reorder (FP32 ↔ FP16/BF16).
Build & Compatibility
- CMake minimum raised to 3.26.