v0.8.1

Liger-Kernel v0.8.1

Highlights

This is an incremental release on top of v0.8.0, bundling a new Megatron-Core integration, additional model coverage, early scaffolding for alternative kernel backends, and a large batch of Ascend NPU work — all opt-in, with the default CUDA path unchanged.

🧩 Megatron-Core integration (new)

Initial, opt-in support for plugging Liger kernels directly into Megatron-Core through native integration points — no monkey-patching of Megatron internals required. Exposed under the new liger_kernel.megatron namespace:

• RMSNorm via a spec provider (#1254)
• Vocab-parallel cross-entropy, including TP > 1 (#1207, #1260)
• RMSNorm benchmark with reference H100 measurements (#1257)

This is an initial surface covering two kernels; it's designed to grow.

🤖 New model support

• Gemma-4 multimodal — apply_liger_kernel_to_gemma4 (#1203)
• DeepSeek-v4 — apply_liger_kernel_to_deepseek_v4 (#1245)
• Fix for Gemma-3 multimodal forward (#1242)

🧪 Alternative kernel backends (experimental, opt-in)

Early groundwork for selecting alternative kernel implementations via LIGER_KERNEL_IMPL=<backend>. These are off by default and do not change existing behavior:

• cuTile kernels for geglu, layer_norm, fused_linear_jsd, cross_entropy, RoPE, and more (#1250, #1269, #1228)
• CuTe DSL cross-entropy scaffolding for Blackwell / B200 (#1279)
• CUTLASS CuTe DSL RMSNorm (#1299)
• infer_device_arch() for coarse GPU-architecture detection (#1273)

➕ New operators & kernel tuning

• Fused modulated RMSNorm (#1225)
• Hardware-gated cross-entropy and SwiGLU tuning for newer architectures (#1266, #1267, #1271, #1239)

🧠 Post-training / alignment

• DPO: added hinge, bco_pair, robust, exo_pair, and discopop loss types (#1204)
• Expose alpha through the LigerFusedLinearDPO public API (#1194)
• GRPO/TRL-Liger fix and a SAPO torch.compile graph-break fix (#1202, #1274)

🖥️ Ascend NPU

A substantial batch of NPU work landed this cycle — optimized cross_entropy, fused_linear_cross_entropy, SwiGLU, layer_norm, embedding, fused_moe, and attention kernels, plus A3-machine support and dependency bumps (torch/torch_npu 2.7.1, triton-ascend 3.2.1).

🔧 Compatibility & fixes

• transformers v5.8.0 support (#1221, #1236)
• Triton compatibility: LOG2_E as tl.constexpr (#1275)
• int32 pointer-overflow fix in fused MoE, with an autotune escape hatch (#1248)
• ROCm source-install extras (#1243)
• Convergence and test-suite stabilization (#1306, #1309, #1310, #1313)

👀 On the horizon

The alternative-backend scaffolding (cuTile / CuTe DSL / CUTLASS) and the new Megatron-Core surface are early foundations — both are areas we expect to keep building on in upcoming releases.

---

What's Changed

• [Benckmark] Benchmark refactor example by @lowdy1 in #1199
• [NPU] optimize cross_entropy for ASCEND NPU by @sunyi0505 in #1206
• [DPO] Add hinge, bco_pair, robust, exo_pair, discopop loss types by @kashif in #1204
• [npu, feat] SwiGLU — multiplier API, mixed precision, UB/grid tune by @zheliuyu in #1219
• [NPU] Add NPU optimized Attention Residual Kernel by @lowdy1 in #1210
• [Benchmark] Add compute_seq_len_sweep_config_with_probe with linear/quadratic scaling support by @shivam2199 in #1218
• [GRPO] fix TRL-liger missing by @kashif in #1202
• [NPU, A3] Add NPU kernel support for A3 machines [1/2] by @zheliuyu in #1220
• [Misc]: update transformers to v5.8.0 by @zheliuyu in #1221
• [NPU] optimize fused_linear_cross_entropy by @sunyi0505 in #1222
• [Benchmark] Refactor benchmark for linear kernels by @lowdy1 in #1223
• Make skills vendor-agnostic via .agents/skills + symlink by @vaibhavjindal in #1226
• [Benchmark] add benchmark for grpo_loss kernel by @UserChen666 in #1216
• [NPU, A3] Add NPU kernel support for A3 machines [2/2] by @zheliuyu in #1227
• Add fused modulated RMSNorm by @yukiu00 in #1225
• [Gemma 4] Add multimodal support (apply_liger_kernel_to_gemma4 for Gemma4ForConditionalGeneration) by @dvdimitrov13 in #1203
• [NPU A3] Fix benchmark issues for fused_linear_jsd and dyt. by @sunyi0505 in #1231
• Update maintainers list in contributing guide by @PKUWZP in #1233
• [NPU, Feat] Pin triton-ascend to v3.2.0 for npu by @zheliuyu in #1236
• Remove dead lce_forward_deprecated path from qwen2 by @vaibhavjindal in #1241
• Add cutile jsd by @xjmxyt in #1228
• fix: expose alpha param through LigerFusedLinearDPO public API by @micdoh in #1194
• Improve ROCm source install extras by @kashif in #1243
• Fix broken gemma3 multimodal forward by @sadreamer in #1242
• [Feature] Support apply_liger_kernel_to_deepseek_v4 & unit tests by @zheliuyu in #1245
• [NPU] optimize layer_norm for A3(X86) by @sunyi0505 in #1247
• fix: int32 pointer overflow in fused MoE kernels (+ autotune env escape hatch) by @Mecoli1219 in #1248
• [NPU] remove ineffective params for ASCEND NPU by @sunyi0505 in #1249
• [Megatron] Add RMSNorm integration by @vaibhavjindal in #1254
• [Megatron] Add cross-entropy integration by @PrathyushaPolepalli in #1207
• [Megatron] Add RMSNorm benchmark + measured H100 results by @vaibhavjindal in #1257
• [NPU] optimize embedding for A3(X86) by @sunyi0505 in #1258
• [NPU] optimize fused_moe for A3(X86) by @sunyi0505 in #1256
• [NPU] fix ut issues for A3(X86) by @sunyi0505 in #1259
• [Megatron] Add vocab-parallel cross-entropy (TP>1 support) by @vaibhavjindal in #1260
• [NPU] Bump Ascend NPU deps to torch/torch_npu 2.7.1 and triton-ascend 3.2.1 by @zheliuyu in #1262
• remove incorrect import for qwen3_5 monkey patch with cross_entropy=True by @aeflores in #1263
• [cuTile] Add cutile backend for geglu, layer_norm, fused_linear_jsd, cross_entropy by @xjmxyt in #1250
• perf(ce): hardware exp2 (ex2.approx) for softmax by @justinhh4 in #1266
• Make LOG2_E a tl.constexpr so the CE kernel compiles on newer Triton by @kashif in #1275
• Avoid torch.compile graph break in SAPO loss by @kashif in #1274
• Add -m flag for modal runs by @vaibhavjindal in #1278
• perf: optimize grad_weight accumulation with addmm by @maskyuanzh in #1239
• Add infer_device_arch() for coarse GPU architecture detection by @yueyiming2009 in #1273
• perf(ce): dtype-aware num_warps (Blackwell-gated) by @justinhh4 in #1267
• perf(swiglu): architecture-aware column tiling for Blackwell (B200) by @Celaena24 in #1271
• [CuteDSL] Add CuTe DSL cross-entropy kernel and CuteDSL integration scaffolding (B200) by @justinhh4 in #1279
• [NPU] refact fused_neighborhood_attention by @sunyi0505 in #1265
• fix: align _input_chunk dtype in FLCE grad_weight addmm under AMP by @justinhh4 in #1283
• [cuTile] Add rope/qwen2vl_mrope/kl_div/group_norm/multi_token_attention by @xjmxyt in #1269
• Add CUTLASS CuTe DSL RMSNorm kernel by @Charlesrizzly in #1299
• Make all the convergence tests run with make test-convergence by @vaibhavjindal in #1306
• Fix failing tests in fused_add_rms_norm and polynorm by @vaibhavjindal in #1309
• Stop printing coverage report by default in make test / test-convergence and fix tolerance in polynorm by @vaibhavjindal in #1310
• Fix convergence test failures by @vaibhavjindal in #1313
• Update pyproject.toml for v0.8.1 release by @vaibhavjindal in #1314

New Contributors

• @dvdimitrov13 made their first contribution in #1203
• @xjmxyt made their first contribution in #1228
• @micdoh made their first contribution in #1194
• @sadreamer made their first contribution in #1242
• @PrathyushaPolepalli made their first contribution in #1207
• @aeflores made their first contribution in #1263
• @justinhh4 made their first contribution in #1266
• @maskyuanzh made their first contribution in #1239
• @yueyiming2009 made their first contribution in #1273
• @Celaena24 made their first contribution in #1271
• @Charlesrizzly made their first contribution in #1299

Full Changelog: v0.8.0...v0.8.1