v0.10.0

[nvluxiaoz]

TensorRT Edge-LLM 0.10.0 Release 2026-08-12

We are excited to announce the release 0.10.0 of TensorRT Edge-LLM!

TensorRT Edge-LLM 0.10.0 provides Day-0 support for NVIDIA Nemotron-3.5 Lightning with MTP and DFlash. It also adds support for Cosmos3-Edge, DiffusionGemma, Nemotron-3.5-ASR, and DSpark speculative decoding. This release also introduces an experimental direct TensorRT engine builder without ONNX export, multi-turn KV-cache reuse, and video input for the experimental OpenAI-compatible server.

Key Features

• Added NVIDIA Nemotron-3.5 Lightning support with MTP and DFlash speculative decoding.
• Added an experimental ONNX-less workflow to build TensorRT engines directly from supported checkpoints.
• Added Cosmos3-Edge multimodal reasoning and experimental policy/action-generation support.
• Added DiffusionGemma NVFP4 block-diffusion inference.
• Added experimental Nemotron-3.5-ASR inference support.
• Added DSpark speculative decoding with DDTree drafting.
• Added paged KV-cache reuse for multi-turn LLM requests and media-aware KV reuse for VLM requests.
• Added video input support to the experimental server.
• Expanded Qwen3-TTS with language and instruction conditioning, VoiceDesign, and on-device voice cloning. Thanks to @suharvest for the CustomVoice language-conditioning contribution in GitHub PR #98.

Other Important Features

Runtime and Performance

• Replaced the legacy embedded-cubin FMHA-v2 backend with CuTe DSL FMHA-v2 and removed the checked-in FMHA-v2 cubin artifacts.
• Replaced FFPA with dense FMHA-v2 kernels for head dimension 512.
• Added Blackwell paged FMHA-v2 kernels for head dimension 512.
• Added NVFP4-A16 Marlin MoE and migrated the INT4 AWQ plugin to the CuTe DSL kernel path.
• Added Qwen3-Omni full-loop CUDA Graph execution.
• Added GPU bicubic resizing for ViT image preprocessing.
• Fixed DFlash verification, MTP/GDN accepted-state replay, and Qwen3-Omni video temporal MRoPE correctness.

Export and Quantization

• Added paired-checkpoint MTP support for Gemma 4 12B.
• Added Gemma 4 26B-A4B INT4 AWQ MoE and Qwen3-Omni 30B-A3B INT4 AWQ quantization and export support.

Server and API

• Added Qwen3-Omni streaming audio output, /v1/audio/speech for Qwen3-TTS, and /v1/audio/transcriptions for Qwen3-ASR.
• Added Claude Code and OpenClaw integration to the experimental server.

Examples and Documentation

• Added NeMo Evaluator integration.
• Added an official platform and software support matrix.

NVIDIA Contributors

@nvluxiaoz @nvamberl @xiangg-nv @willg-nv @mahu888 @nv-samcheng @duofant @Caohanwen0 @JCalafato @duanyaqi @zhazhang-nv @zhaoyuanh-nvidia @zhijial-nvidia @ever-wong @Jasper-NV @jhalabi-nv @levichen-nvidia @sunghyunp-nvidia @nvyocox @ruocheng-nv @yuanyao-nv @qikail-ctrl

---

This discussion was created from the release v0.10.0.