Dynamo v1.4.0-inkling-dev.1

Release Notes

Dynamo v1.4.0-inkling-dev.1 is an experimental dev build giving an early look at Thinking Machines Inkling support on the Dynamo SGLang backend. It is not recommended for production — features may be incomplete and APIs, behaviors, and defaults may change before the stable release. Use it for evaluation, testing, and early feedback only.

Summary

Dynamo v1.4.0-inkling-dev.1 adds Thinking Machines Inkling (thinkingmachines/Inkling-NVFP4, served as inkling-model) serving on the Dynamo SGLang backend, with an aggregated 8x B200 deployment recipe at NVFP4 precision. Inkling is a multimodal (text + image) hybrid Mamba/attention mixture-of-experts model with dedicated inkling reasoning and tool-call parsers. The build runs on a custom SGLang Inkling CUDA 13 image. This is the first Dynamo dev preview published on the SGLang container rather than vLLM.

Release Branch: release/1.4.0-inkling-dev.1, cut from main commit b01e0619bb (2026-07-15)
SGLang: custom Inkling build (lmsysorg/sglang:inkling-cu13 base), NIXL v1.3.0

Container Images

Backend	Arch	Image
SGLang (CUDA 13)	multi-arch (amd64 + arm64)	nvcr.io/nvidia/ai-dynamo/sglang-runtime:1.4.0-inkling-dev.1

Backend Versions

Backend	Source	CUDA	Python	Notes
SGLang	Dynamo-built runtime on a custom SGLang Inkling base (lmsysorg/sglang:inkling-cu13)	13.0	3.12	NIXL v1.3.0

Models

Model	Precision	GPU	Notes
thinkingmachines/Inkling-NVFP4	NVFP4 (modelopt_fp4)	8x B200	served as inkling-model; aggregated, TP8; text + image; reasoning + tool calling; FA4 attention, FlashInfer-TRT-LLM FP4 GEMM and routed MoE, Torch symmetric-memory AllReduce, Mamba radix cache (extra_buffer)

About Inkling

Inkling is a model from Thinking Machines, published on Hugging Face as thinkingmachines/Inkling-NVFP4 (NVFP4-quantized; the checkpoint is roughly 592 GB). This build serves it through Dynamo's SGLang backend as a multimodal (text + image) model with a hybrid Mamba / sliding-window-attention architecture and a mixture-of-experts runner, using dedicated inkling reasoning and tool-call parsers. The v1.4.0-inkling-dev.1 recipe runs aggregated on 8x B200 at NVFP4 with tensor parallelism 8.

Full Changelog

SGLang Backend (Inkling)

• Inkling CUDA 13 Image: Pointed the SGLang runtime at the custom Inkling CUDA 13 base image so the Inkling build is used for the published container (2099e46).
• Remote Quant Config Resolution: Fixed resolution of Inkling's remote quantization config so the modelopt_fp4 weights load correctly at startup (c96ec29); vendored as container/deps/sglang/inkling-remote-quant-config.patch.
• Model EOS In Min Tokens: Fixed minimum-new-tokens handling so the model's EOS token is honored rather than suppressed by the min-tokens floor (366a868); vendored as container/deps/sglang/inkling-min-new-tokens-eos.patch.

Recipes

• TML Inkling SGLang agg-b200 Recipe: Added the recipes/tml-inkling SGLang aggregated B200 recipe — README, model-cache PVC plus HuggingFace model-download job, and the DynamoGraphDeployment manifest that serves inkling-model at NVFP4/TP8 with the FA4 / FlashInfer-TRT-LLM / Mamba-radix-cache runtime settings (recipe folder, README), and made the worker and download job tolerate the nvidia.com/gpu node taint so they schedule on tainted GPU nodes.

Getting Started

Deployment is Kubernetes-based (Dynamo Platform + DynamoGraphDeployment). Full steps — namespace, model-cache PVC, the ~592 GB HuggingFace download job, DGD apply, and a /v1/chat/completions smoke test — are in the recipe README. The recipe expects an nvcr.io image-pull secret and an optional HuggingFace token.

Known Issues / Limitations

• Experimental dev preview — not QA-gated; not for production.
• SGLang aggregated only; single agg-b200 (8x B200, TP8, NVFP4) profile — no disaggregated or non-B200 recipe in this build.

---