Skip to content

Dynamo v1.4.0-qwen-3.8-2.4t-dev.1

Pre-release
Pre-release

Choose a tag to compare

View all tags
@ynpandey-nv ynpandey-nv released this 27 Aug 22:43
v1.4.0-qwen-3.8-2.4t-dev.1
65c4c57
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Release Notes

Dynamo v1.4.0-qwen-3.8-2.4t-dev.1 is an experimental snapshot build with early Qwen3.8-2.4T-A95B-FP8 support on the Dynamo vLLM and SGLang backends. It is not recommended for production and is not a QA-gated release. APIs, behavior, and defaults may change before stable support. Use it for evaluation, testing, and early feedback only.

Summary

Dynamo v1.4.0-qwen-3.8-2.4t-dev.1 adds serving for Qwen/Qwen3.8-2.4T-A95B-FP8 on Dynamo vLLM and SGLang. The published vLLM runtime is built on vLLM 0.27.1.

Kubernetes recipes cover vLLM aggregate chat and agentic profiles on GB300 and GB200, plus SGLang aggregate and disaggregated chat profiles on the same SKUs, with shared model-cache PVC/download manifests under recipes/qwen3.8-2.4t-a95b-fp8.

Release Branch: release/1.4.0-qwen-3.8-2.4t-dev.1, tip 65c4c57b0122d7502fa3aa886d331246be71cd63 (2026-08-27)

vLLM: v0.27.1-ubuntu2404 (vllm/vllm-openai), NIXL v1.3.2

SGLang: v0.5.16-cu130-runtime (lmsysorg/sglang), NIXL v1.3.0

Container Images

Backend Arch Image
vLLM (CUDA 13) multi-arch (amd64 + arm64) nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.4.0-qwen-3.8-2.4t-dev.1
SGLang (CUDA 13) multi-arch (amd64 + arm64) nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.4.0-qwen-3.8-2.4t-dev.1

Backend Versions

Backend Source CUDA Python Notes
vLLM Dynamo-built runtime on vllm/vllm-openai:v0.27.1-ubuntu2404 13.0 3.12 NIXL v1.3.2
SGLang Dynamo-built runtime on lmsysorg/sglang:v0.5.16-cu130-runtime 13.0 3.12 NIXL v1.3.0

Models

Model Precision GPU Notes
Qwen/Qwen3.8-2.4T-A95B-FP8 FP8 weights, FP8 KV GB300, GB200 hybrid gated-delta-net + MoE; 262,144-token context

About Qwen3.8-2.4T

Qwen3.8-2.4T-A95B is a Qwen preview model published on Hugging Face as Qwen/Qwen3.8-2.4T-A95B-FP8. This build serves it through Dynamo's vLLM and SGLang backends with TP16 MNNVL aggregate profiles on GB300 and GB200, plus SGLang 1P1D / 2P1D disaggregated chat profiles.

Full Changelog

vLLM Backend

  • Qwen3.8-2.4T runtime support: Added Dynamo vLLM serving for Qwen3.8-2.4T-A95B, including frontend and recipe wiring for the FP8 checkpoint (#13105, #13265).
  • vLLM 0.27.1: Bumped the vLLM runtime image to vllm/vllm-openai:v0.27.1-ubuntu2404 (#13059).

SGLang Backend

  • Recipe-specific SGLang images: Declared the Qwen3.8-2.4T SGLang runtime image used by these recipes (#13232).
  • SGLang GB200 frontend args: Corrected SGLang GB200 frontend launch flags (dropped --model-path, fixed parser flags) (#13324).

Recipes

  • Qwen3.8-2.4T recipes: Added recipes/qwen3.8-2.4t-a95b-fp8 — shared model-cache PVC/download job, vLLM aggregate chat and agentic profiles on GB300 and GB200, and SGLang aggregate plus 1P1D/2P1D disaggregated chat profiles (recipe folder, README) (#13105, #13265, #13232, #13324).

Getting Started

Deployment is Kubernetes-based (Dynamo Platform + DynamoGraphDeployment). Full steps — namespace, model-cache PVC, Hugging Face download job, DGD apply, and a /v1/chat/completions smoke test — are in the recipe README. Recipes expect an nvcr.io image-pull secret and a Hugging Face token secret named hf-token-secret.

Known Issues / Limitations

  • Experimental snapshot build — not QA-gated; not for production.
  • Stock vLLM 0.27.1 has no MTP mapping for qwen3_5_moe_text, so these vLLM profiles do not enable --speculative-config.
  • Stock lmsysorg/sglang:v0.5.16 cannot load the text-only checkpoint; SGLang recipes need a patched or v0.5.17+ runtime.
  • GB200 and GB300 recipes require ARM-node scheduling and the NVIDIA DRA driver with ComputeDomain support for multi-node NVLink.
Assets 2
Loading