Skip to content

Dynamo v1.6.0-deepseek-v4.1-flash-dev.1

Pre-release
Pre-release

Choose a tag to compare

View all tags
@ynpandey-nv ynpandey-nv released this 12 Sep 02:33
v1.6.0-deepseek-v4.1-flash-dev.1
7931147
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Release Notes

Dynamo v1.6.0-deepseek-v4.1-flash-dev.1 is an experimental snapshot build with early DeepSeek-V4.1-Flash support on the Dynamo SGLang backend. It is not recommended for production and is not a QA-gated release. APIs, behavior, and defaults may change before stable support. Use it for evaluation, testing, and early feedback only.

Summary

Dynamo v1.6.0-deepseek-v4.1-flash-dev.1 adds serving for deepseek-ai/DeepSeek-V4.1-Flash on Dynamo SGLang. The published container is built on the CUDA 13 SGLang dev-dsv41 preview base and includes native DeepSeek V4.1 frontend support: Rust prompt rendering, automatic deepseek_v41 reasoning and tool-call parsing, streamed DSML handling, and rejection of completed malformed tool-call arguments (#14690).

Kubernetes recipes cover 8x GB200 aggregated (two TP4/EP4 workers, KV-aware routing, DSpark speculative decoding) and 1P1D disaggregated (TP4/EP4 per role, Mooncake KV transfer over TCP or GKE RDMA) profiles, plus a shared model-cache PVC and Hugging Face download job pinned to revision dba1be0a40aa45a94ad051997016db3960a90277. Both targets serve up to 1,048,576 tokens of context. Neither target is benchmarked (#14682, #14763).

Release Branch: release/1.6.0-deepseek-v4.1-flash-dev.1, tip 7931147d403c554575438c2451da4248e53e10b8 (2026-09-12)

SGLang: lmsysorg/sglang:dev-dsv41@sha256:e56358a68b06427362283c8c8a9d7d706448082ae53098ea1131b5d51aa1fd62 (CUDA 13), NIXL v1.4.0

Container Images

Backend Arch Image
SGLang (CUDA 13) multi-arch (amd64 + arm64) nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.6.0-deepseek-v4.1-flash-dev.1

Backend Versions

Backend Source CUDA Python Notes
SGLang Dynamo-built runtime on lmsysorg/sglang:dev-dsv41 (digest-pinned), with non-runtime packages pruned from the upstream development image 13.0 3.12 NIXL v1.4.0; DeepSeek V4.1 has not shipped in a numbered SGLang release yet

Models

Model Precision GPU Notes
deepseek-ai/DeepSeek-V4.1-Flash FP8 dense, FP4 MoE experts, FP8 KV (fp8_e4m3) 8x GB200 SGLang dev-dsv41; aggregated TP4/EP4 (DSpark k=5, KV-aware routing) or 1P1D disagg TP4/EP4 per role (no speculation); 1M context; deepseek_v41 reasoning and tool-call parsers

About DeepSeek-V4.1-Flash

DeepSeek-V4.1-Flash is a Mixture-of-Experts checkpoint from DeepSeek (deepseek-ai/DeepSeek-V4.1-Flash; about 510 GB). Routed experts ship FP4, dense weights are FP8, and the engine uses an FP8 KV cache. This preview serves the full 1,048,576-token window on GB200 through Dynamo's SGLang backend, with reasoning, tool calling, streaming chat, and schema-constrained output. The checkpoint also accepts images; the recipes in this build serve text only.

Full Changelog

SGLang Backend

  • DeepSeek V4.1 frontend: Native Rust prompt rendering and unified reasoning/tool-call parsing for DeepSeek V4.1, including automatic parser selection from model metadata, preserved partial DSML markers across streamed chunks, normalized reasoning-effort values, and rejection of completed malformed DSML arguments during batch aggregation (#14690).
  • SGLang dev-dsv41 runtime: Pins the CUDA 13 SGLang runtime to the digest-pinned lmsysorg/sglang:dev-dsv41 preview base required to serve this model, and trims non-runtime packages from that development image (#14690).

Recipes

  • DeepSeek-V4.1-Flash SGLang GB200 recipes: Added recipes/deepseek-v4.1-flash — shared model-cache PVC/download job (pinned HF revision), aggregated GB200 (sglang/agg-gb200: two TP4 workers, KV-aware routing, DSpark block size 5, decode CUDA-graph batch capped at 64), and disaggregated GB200 1P1D (sglang/disagg-gb200: generic Mooncake TCP target plus GKE RDMA variant). Both profiles use sglang-runtime:1.6.0-deepseek-v4.1-flash-dev.1, deepseek_v41 parsers, and 1M context. Day-0 functional scope only; no published performance claim (recipe folder, README, docs) (#14682, #14763).

Getting Started

Deployment is Kubernetes-based (Dynamo Platform + DynamoGraphDeployment). Full steps — namespace, Hugging Face token, model-cache PVC, download job, DGD apply, and a /v1/chat/completions smoke test — are in the recipe docs and the recipe README. Recipes expect an nvcr.io image-pull secret and a Hugging Face token secret named hf-token-secret with access to deepseek-ai/DeepSeek-V4.1-Flash. GB200 recipes require ARM-node scheduling (two nodes, 4 GPUs each).

Read the smoke-test response body, not only the HTTP status. On the disaggregated target a KV transfer failure can still return HTTP 200 with content: null and zero completion tokens.

Known Issues / Limitations

  • This is a branch-specific snapshot build, not a QA-gated stable release.
  • Day-0 recipes only: both targets passed a functional probe; neither is benchmarked.
  • Disaggregated SGLang does not support DSpark for this model. Use the aggregated target for speculative decoding.
  • Generic disaggregated KV is pinned to Mooncake TCP (MC_FORCE_TCP=1). The GKE variant uses RDMA and DMA-BUF (WITH_NVIDIA_PEERMEM=0). The NVLink fabric is faster but needs both workers in one NVLink clique; a split placement reports Ready and returns empty completions.
  • KV-aware routing is a no-op on the disaggregated target with a single decode worker.
  • Recipes serve text only, even though the checkpoint accepts images.
  • Both targets set SGLANG_DEFAULT_THINKING=true. Thinking is off by default for this model, and then reasoning_content is empty.
  • GB200 recipes require ARM-node scheduling.
Assets 2
Loading