Dynamo v1.3.0-cosmos3-dev.1

Release Notes

Dynamo v1.3.0-cosmos3-dev.1 is an experimental dev build giving an early look at NVIDIA Cosmos3 support on the Dynamo vLLM-Omni backend. It is not recommended for production — features may be incomplete and APIs, behaviors, and defaults may change before the stable release. Use it for evaluation, testing, and early feedback only.

Summary

Dynamo v1.3.0-cosmos3-dev.1 adds NVIDIA Cosmos3 multimodal generation — text-to-image, text-to-video, and image-to-video — to the Dynamo vLLM-Omni backend (#10132), backed by the native Cosmos3 pipeline in vllm-project/vllm-omni#3454. Only the vLLM container is published.

Branch: release/1.3.0-cosmos3-dev.1, cut from main commit 5b4bc1d (2026-05-29)
vLLM-Omni Pin: vllm-project/vllm-omni#3454 @ 65b83d8

Container Images

Backend	Arch	Image
vLLM (CUDA 13)	multi-arch (amd64 + arm64)	nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.3.0-cosmos3-dev.1

Backend Versions

Backend	Source	CUDA	Python	Notes
vLLM	Dynamo-built runtime + vLLM-Omni installed from vllm-omni#3454 (git)	13.0	3.12	vLLM-Omni Cosmos3OmniDiffusersPipeline (Diffusers-based diffusion)

Models

Model	HuggingFace ID	Modalities	Notes
Cosmos3-Nano	nvidia/Cosmos3-Nano	t2i / t2v / i2v	One modality per worker
Cosmos3-Super	nvidia/Cosmos3-Super	t2i / t2v / i2v	One modality per worker

About Cosmos3

Cosmos3 is part of NVIDIA's Cosmos Physical-AI world-foundation-model family, shipped in two sizes — Nano and Super. Beyond conditional generation it is a world model that reasons about actions and dynamics (policy / forward_dynamics / inverse_dynamics) and can layer sound onto video. The family spans text-to-image, text-to-video, image-to-video, and video-to-video; this v1.3.0-cosmos3-dev.1 build exposes t2i / t2v / i2v through Dynamo's vLLM-Omni backend, with the remaining modes to follow.

It is served by vLLM-Omni's Cosmos3OmniDiffusersPipeline — a single Diffusers-based diffusion pipeline that handles all three exposed modalities; the task is selected per request via the modalities field on the prompt.

Full Changelog

vLLM-Omni Backend (Cosmos3)

• Cosmos3 generation support — serve Cosmos3 t2i/t2v/i2v through Dynamo's OpenAI-compatible surface; image responses on /v1/images/generations (#10132).
• Image output handling — normalize_image_frames() converts the pipeline's raw numpy frame arrays ([batch, frames, H, W, C]) into PNG-encoded responses (#10132).
• Guardrails toggle — --cosmos3-guardrails / --no-cosmos3-guardrails, routed into AsyncOmni(model_config={"guardrails": ...}) to skip the Cosmos3 safety guardrails at startup. Guardrails (upstream CosmosSafetyChecker: a text blocklist + Qwen3Guard text moderation, and RetinaFace video face-blur) are enabled by default (#10132).

Upstream pipeline (vLLM-Omni #3454)

Pinned at commit 65b83d8. What the pinned PR provides:

• Cosmos3OmniDiffusersPipeline — a single Diffusers-based pipeline for T2I / T2V / I2V (nvidia/Cosmos3-Nano, -Super); a new modalities field on OmniTextPrompt disambiguates the task for the shared pipeline.
• Guardrails adapter around the cosmos_guardrail package (CosmosSafetyChecker); guardrail-less inference still works when the package is absent and guardrails are disabled.
• Diffusion runtime — a cache-dit diffusion-caching backend and cuDNN / SDPA attention backends; OpenAI-compatible image and video serving.
• 91 new model/pipeline test cases upstream.
• t2v / i2v / t2i only in this PR — sound-on-video and the action/dynamics modes land in follow-up upstream PRs.

Examples & Docs

• Per-modality launch scripts agg_omni_cosmos3_{image,video,i2v}.sh and sample request payloads under examples/backends/vllm/launch/cosmos3/ (#10132).
• New guide docs/backends/vllm/cosmos3.md — install, serving, request formats, gotchas (#10132).

Getting Started

Each worker serves a single modality. Pull the container, then launch with the matching script:

# text-to-image
bash examples/backends/vllm/launch/agg_omni_cosmos3_image.sh
# text-to-video
bash examples/backends/vllm/launch/agg_omni_cosmos3_video.sh
# image-to-video
bash examples/backends/vllm/launch/agg_omni_cosmos3_i2v.sh

Send requests using the sample payloads in examples/backends/vllm/launch/cosmos3/ (image generation is served on /v1/images/generations). Full request formats and options are in docs/backends/vllm/cosmos3.md.