Skip to content

Dynamo v1.5.0-deepseek-v4-pro-0813-dev.1

Pre-release
Pre-release

Choose a tag to compare

View all tags
@ynpandey-nv ynpandey-nv released this 04 Sep 01:27
v1.5.0-deepseek-v4-pro-0813-dev.1
28f9c30
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Release Notes

Dynamo v1.5.0-deepseek-v4-pro-0813-dev.1 is an experimental snapshot build with early DeepSeek-V4-Pro-0813 support on the Dynamo vLLM backend. It is not recommended for production and is not a QA-gated release. APIs, behavior, and defaults may change before stable support. Use it for evaluation, testing, and early feedback only.

Summary

Dynamo v1.5.0-deepseek-v4-pro-0813-dev.1 publishes a vLLM runtime on v0.28.0 (CUDA 13.0) for DeepSeek-V4-Pro-0813. The CUDA image was already on that vLLM pin at branch cut; this snapshot also aligns the XPU vLLM pin to 0.28.0 (#14269).

On top of that runtime, the release ships aggregated and disaggregated vLLM recipes for the public deepseek-ai/DeepSeek-V4-Pro-0813 checkpoint (MXFP4 experts + FP8 KV, 1M context, no CPU KV offload): 8x GB200 and 8x H200 aggregated, plus 1P1D 16-GPU disaggregated profiles on both SKUs, with KV-aware routing, shared model-cache/download jobs, and an AIPerf Mooncake trace-replay benchmark (#14307). This is a different checkpoint from deepseek-ai/DeepSeek-V4-Pro and must not share a model cache with those recipes.

Release Branch: release/1.5.0-deepseek-v4-pro-0813-dev.1, cut from main commit dc08c938c5 (2026-08-31)

Release Tip: 28f9c307dcff4130412ac04e4ea07c96b0cde6f5

vLLM: v0.28.0 (vllm/vllm-openai:v0.28.0-ubuntu2404 base), NIXL v1.3.2 (UCX v1.21.0)

Container Images

Backend Arch Image
vLLM (CUDA 13) multi-arch (amd64 + arm64) nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.5.0-deepseek-v4-pro-0813-dev.1

Backend Versions

Backend Source CUDA Python Notes
vLLM Dynamo-built runtime on vllm/vllm-openai:v0.28.0-ubuntu2404 13.0 3.12 NIXL v1.3.2; vLLM-Omni v0.28.0rc1

Models

Model Precision GPU Notes
deepseek-ai/DeepSeek-V4-Pro-0813 MXFP4 experts + FP8 KV 8x GB200 vLLM 0.28.0; aggregated TP8/EP8 (2 nodes, MNNVL) or disagg 1P1D (16 GPU); DSpark k=5; KV-aware routing; 1M context
deepseek-ai/DeepSeek-V4-Pro-0813 MXFP4 experts + FP8 KV 8x H200 vLLM 0.26.0 via vllm-runtime:1.4.0; aggregated TP8/EP8 or disagg 1P1D (16 GPU); no speculative decoding; KV-aware routing; 1M context

About DeepSeek-V4-Pro-0813

DeepSeek-V4-Pro-0813 is a 1.6T-parameter text MoE from DeepSeek (deepseek-ai/DeepSeek-V4-Pro-0813; ~832 GiB). Routed experts ship MXFP4 and the engine requires FP8 KV. This preview serves the full 1,048,576-token window on GB200 and H200, aggregated or disaggregated, with reasoning and tool calling. It is not a revision of DeepSeek-V4-Pro.

Full Changelog

vLLM Backend

  • XPU vLLM 0.28.0 pin: Aligned the XPU vLLM base/runtime and matching SBOM stem to v0.28.0 on this branch. The CUDA 13.0 runtime was already on v0.28.0-ubuntu2404 at cut; these commits do not change the CUDA image contents (#14269).

Recipes

  • DeepSeek-V4-Pro-0813 vLLM recipes: Added recipes/deepseek-v4/deepseek-v4-pro-0813 — agg-gb200-agentic, disagg-gb200-agentic, agg-h200-agentic, disagg-h200-agentic, model-cache PVC and download job, AIPerf Mooncake trace-replay under perf/, and Fern catalog/docs. GB200 workers use vllm-runtime:1.5.0-deepseek-v4-pro-0813-dev.1; H200 workers stay on digest-pinned vllm-runtime:1.4.0 (recipe folder, README) (#14307).

Getting Started

Deployment is Kubernetes-based (Dynamo Platform + DynamoGraphDeployment). Full steps — namespace, Hugging Face token, model-cache PVC, download job, DGD apply, smoke test, and the AIPerf workflow — are in the recipe README. The recipes expect an nvcr.io image-pull secret and a Hugging Face token with access to deepseek-ai/DeepSeek-V4-Pro-0813.

Known Issues / Limitations

  • This is a branch-specific snapshot build, not a QA-gated stable release.
  • GB200 disaggregated RDMA is provider-specific. vllm/disagg-gb200-agentic/deploy.yaml ships GKE networking.gke.io annotations and rdma-0–rdma-3 resource requests that were used for the validated numbers. Applying it unchanged on another provider leaves those pods Pending. Replace that RDMA block with the cluster’s NIC/RDMA mechanism before deploy. Aggregated GB200 and both H200 recipes are unaffected.
  • H200 recipes run vllm-runtime:1.4.0 (vLLM 0.26.0). GB200 recipes run this tag’s vllm-runtime:1.5.0-deepseek-v4-pro-0813-dev.1 image (vLLM 0.28.0). Do not mix those worker images across SKUs.
  • vLLM 0.27.x has an upstream accuracy regression on this checkpoint; do not use it.
  • --kv-cache-dtype fp8 is required (DeepseekV4 fp8_ds_mla layout).
  • message.content can be null when --dyn-reasoning-parser deepseek_v4 is set and the generation never emits </think>; the answer is in reasoning_content (upstream vllm-project/vllm#48645).
  • An invalid response_format: json_schema is returned as HTTP 500 rather than 4xx.
  • Speculative decoding is enabled on GB200 (DSpark k=5) and is not used on H200.
Assets 2
Loading