Skip to content

Dynamo v1.5.0-nemotron-3.5-lightning-dev.1

Pre-release
Pre-release

Choose a tag to compare

View all tags
@ynpandey-nv ynpandey-nv released this 11 Aug 17:54
· 1 commit to release/1.5.0-nemotron-3.5-lightning-dev.1 since this release
v1.5.0-nemotron-3.5-lightning-dev.1
bfcf713
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Release Notes

Dynamo v1.5.0-nemotron-3.5-lightning-dev.1 is an experimental snapshot build with early NVIDIA Nemotron-3.5-Lightning support on the Dynamo vLLM and TensorRT-LLM backends. It is not recommended for production and is not a QA-gated release. APIs, behavior, and defaults may change before stable support. Use it for evaluation, testing, and early feedback only.

Summary

Dynamo v1.5.0-nemotron-3.5-lightning-dev.1 adds serving for nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 on Dynamo vLLM and TensorRT-LLM. The published containers are built from the Day 0 release stack (runtime/container bumps plus compliance rebaselining for the vLLM and TensorRT-LLM images).

Kubernetes recipes cover aggregate and selected 1P/1D disaggregated vLLM profiles (MTP, DFlash, DSpark, plus KV-router DSpark variants) and aggregate TensorRT-LLM profiles (no-spec and MTP) across H100, H200, B200, and GB200, with shared model-cache PVC/download manifests and a perf/trace reference under recipes/nemotron-3.5-lightning.

Release Branch: release/1.5.0-nemotron-3.5-lightning-dev.1, tip bfcf71343acf52711303eb3b667f74008b098bcb (2026-08-11)

vLLM: nightly cb8104839c141609d99f1254459ef3a4f1bd4263 base (CUDA 13), NIXL v1.3.1

TensorRT-LLM: 1.3.0rc24 (nvcr.io/nvidia/tensorrt-llm/release), NIXL v1.3.1

Container Images

Backend Arch Image
vLLM (CUDA 13) multi-arch (amd64 + arm64) nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0-nemotron-3.5-lightning-dev.1
TensorRT-LLM (CUDA 13.1) multi-arch (amd64 + arm64) nvcr.io/nvidia/ai-dynamo/tensorrtllm-runtime:1.5.0-nemotron-3.5-lightning-dev.1

Backend Versions

Backend Source CUDA Python Notes
vLLM Dynamo-built runtime on vllm/vllm-openai:nightly-cb8104839c141609d99f1254459ef3a4f1bd4263, rebaselined on NVIDIA CUDA for compliance 13.0 3.12 NIXL v1.3.1
TensorRT-LLM Dynamo-built runtime on nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc24 13.1 3.12 NIXL v1.3.1; stale PyNvVideoCodec purged from the runtime image

Models

Model Precision GPU Notes
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 NVFP4 (modelopt_fp4) H100, H200, B200, GB200 served as the target model; nemotron_nano tool-call / reasoning parsers
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash NVFP4 with DFlash recipes draft checkpoint for DFlash speculative decoding
nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark NVFP4 with DSpark recipes draft checkpoint for DSpark speculative decoding

About Nemotron-3.5-Lightning

Nemotron-3.5-Lightning is an NVIDIA preview model published on Hugging Face as nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 (NVFP4). This build serves it through Dynamo's vLLM and TensorRT-LLM backends with MTP / DFlash / DSpark speculative-decoding options on vLLM and MTP / no-spec aggregate options on TensorRT-LLM.

Full Changelog

vLLM Backend

  • Lightning runtime stack: Day 0 container/runtime changes for Nemotron-3.5-Lightning serving on the Dynamo vLLM path.
  • CUDA compliance rebaseline: Rebaselined the vLLM runtime on the NVIDIA CUDA floor (replacing the prior vLLM self-baseline).

TensorRT-LLM Backend

  • Lightning runtime stack: Day 0 container/runtime changes for Nemotron-3.5-Lightning serving on TensorRT-LLM.
  • TensorRT-LLM 1.3.0rc24: Bumped the TensorRT-LLM runtime base to 1.3.0rc24 and purged stale PyNvVideoCodec from the image (37c984c7, 6c5eec97).

Recipes

  • Nemotron-3.5-Lightning recipes: Added recipes/nemotron-3.5-lightning — shared model-cache PVC/download job, vLLM aggregate and selected 1P/1D disaggregated profiles (MTP / DFlash / DSpark, plus KV-router DSpark variants), TensorRT-LLM aggregate profiles (no-spec and MTP), and a perf/trace reference (recipe folder, README).

Getting Started

Deployment is Kubernetes-based (Dynamo Platform + DynamoGraphDeployment). Full steps — namespace, model-cache PVC, Hugging Face download job (target + DFlash/DSpark drafts), DGD apply, and a /v1/chat/completions smoke test — are in the recipe README. Recipes expect an nvcr.io image-pull secret and a Hugging Face token secret named hf-token-secret.

Known Issues / Limitations

  • Experimental snapshot build — not QA-gated; not for production.
  • TensorRT-LLM recipes in this build are aggregate-only (no disaggregated TensorRT-LLM profiles).
  • GB200 recipes require ARM-node scheduling.
  • Disaggregated vLLM recipes require matching speculative-decoding settings on prefill and decode for compatible NIXL cache metadata.
Assets 2
Loading