v0.8.2

Overview

Liger-Kernel v0.8.2 delivers performance, correctness, and numerical-precision improvements across cross entropy, MoE, RMSNorm, JSD, and other operators. The release also improves Megatron integration, testing, and overall kernel robustness.

This release further expands experimental backends and features, including CuTe DSL, cuTile, and LigerCute, with new kernels and optimizations targeting Hopper and Blackwell GPUs.

Summary

• Added fused linear scaled cross entropy, providing an efficient frontend compatible with verl-style fused log-probability computation for RL training. See [verl's fused linear cross-entropy implementation](https://github.com/volcengine/verl/blob/main/verl/utils/kernel/linear_cross_entropy.py?utm_source=chatgpt.com).
• Improved performance and numerical accuracy of cross-entropy kernels.
• Fixed correctness and robustness issues across cross entropy, RMSNorm, JSD, and MoE.
• Added Megatron SwiGLU integration and improved NPU support.
• Expanded experimental CuTe DSL, cuTile, and LigerCute support, including additional Hopper and Blackwell kernels.

What's Changed

• Pin ruff version to <0.16.0 to avoid checkstyle breakage by @vaibhavjindal in #1318
• Guard the FLCE addmm fast path on compute capability 8.0 or higher by @albertvillanova in #1316
• feat: Add accum dtype support to fused linear JSD by @KyleMylonakisProtopia in #1322
• fix(test): prevent in-place RMSNorm backward from clobbering shared grad buffers by @yueyiming2009 in #1305
• perf(ce): single-shot dx_y gradient correction by @justinhh4 in #1268
• Fix wrong grad_weight/grad_bias for fused linear cross entropy with reduction='none' by @kashif in #1298
• Guard Triton launches by tensor device by @kashif in #1304
• fix(ce): restore true-class gradient precision in low-precision buffers by @vaibhavjindal in #1329
• [CuteDSL] Add benchmarking, CE memory optimizations, and match Triton num_warps by @justinhh4 in #1290
• [Tests] Move cutedsl tests to test/cutedsl/ and add make test-cutedsl target by @vaibhavjindal in #1337
• Add Hopper CuTe DSL fused scaled cross entropy by @kolehma8 in #1334
• Add SM100 CuTe DSL fused linear cross entropy by @justinhh4 in #1339
• perf(cutedsl): optimize RMSNorm CuTe DSL kernel (fused backward + host launch + width-aware forward warps) by @Charlesrizzly in #1349
• fix(triton): pin fp64 scalar kernel params to fp32 in swiglu and rms_norm by @vaibhavjindal in #1350
• Add LigerCute: fused expert-parallel MoE kernels for Hopper and Blackwell (CUTLASS + NVSHMEM) by @vaibhavjindal in #1355
• Add cutedsl implementation of Liger Rope Kernel by @Celaena24 in #1282
• Add fused linear scaled cross entropy frontend by @kolehma8 in #1354
• Add CuTe DSL implementation of the swiglu liger kernel by @Celaena24 in #1277
• [cuTile] Add fused scaled cross entropy by @yueyiming2009 in #1359
• [Megatron] Add SwiGLU integration by @buffett0323 in #1326
• [Liger Cute] Fix fused MoE combine for partial token tiles by @supercharleszhu in #1361
• perf(moe): add Blackwell 2SM MLP3 and MLP4 by @autinn in #1365
• perf(cutedsl): optimize Hopper FLSCE forward by @kolehma8 in #1370
• [Hopper] Add CuTe DSL and cuTile fused linear cross entropy by @Celaena24 in #1356
• Fix single-token fused MoE auto dispatch by @supercharleszhu in #1368
• [NPU] Fix: recompute true-class CE gradient in fp32 on no-weight backward path by @zheliuyu in #1341
• Pin fp64 scalar kernel params to fp32 in cross entropy by @buffett0323 in #1362
• Pin fp64 scalar kernel params via constexpr in grpo_loss by @buffett0323 in #1366
• Pin fp64 scalar kernel params to fp32 in remaining norm ops by @buffett0323 in #1358
• Pin fp64 scalar kernel params to fp32 in modulated_rms_norm and attn_res by @buffett0323 in #1364
• Support strided Liger MoE input weights by @supercharleszhu in #1377
• Tune SM100 GEMM scheduling for dW shapes by @justinhh4 in #1380
• Optimize and harden fused MoE kernels by @kolehma8 in #1381
• Update CuTe DSL backend docs: op list and DSL version by @PKUWZP in #1386
• perf(cutedsl): add Blackwell fast path to RMSNorm by @PKUWZP in #1388
• [NPU] align fused linear JSD forward with accum_dtype API by @zheliuyu in #1346
• Update pyproject.toml for v0.8.2 release by @kolehma8 in #1394

New Contributors

• @KyleMylonakisProtopia made their first contribution in #1322
• @buffett0323 made their first contribution in #1326
• @supercharleszhu made their first contribution in #1361
• @autinn made their first contribution in #1365

Full Changelog: v0.8.1...v0.8.2