Skip to content

Dynamo v1.6.2

Latest
Latest

Choose a tag to compare

View all tags
@dagil-nvidia dagil-nvidia released this 05 Nov 22:11
v1.6.2
28f9765
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Dynamo v1.6.2 - Release Notes

Summary

Dynamo v1.6.2 is a patch release on top of v1.6.1. It refreshes the OS and Python packages in the runtime images, moves the SGLang image to CPython 3.12.16, returns HTTP 400 instead of HTTP 500 when a TensorRT-LLM prompt exceeds the engine token budget, and adds an opt-in garbage-collection freeze for SGLang workers.

One change can affect existing workflows. See Dependency Changes.

Base Branch: release/1.6.2

Dependency Changes

  • Runtime OS Package Refresh: Upgraded libgnutls30t64 in the Frontend, vLLM, and sidecar images (#18064) and libbrotli1 in the Frontend, SGLang, TensorRT-LLM, and vLLM images (#17913) to the current Ubuntu patch releases.
  • SGLang Image CPython 3.12.16: Moved the SGLang runtime image from Ubuntu's Python 3.12.3 to a uv-installed CPython 3.12.16, which bundles OpenSSL 3.5.9 (#18081). The Intel XPU image is unchanged.

Note: cuda-gdb in the SGLang image no longer supports Python scripting, because its Python front end links the Ubuntu libpython3.12 that this change removes.

  • Planner Python 3.13.18: Moved the Planner image to python:3.13.18-slim for the build and the distroless Python 3.13-v4.1.10 base for the runtime (#17915). The build now fails if the runtime base bundles an older pip.
  • Mooncake 0.3.14: Raised the Mooncake transfer engine floor from 0.3.13.post1 to 0.3.14 in the SGLang and vLLM images (#18091). The TensorRT-LLM image does not ship Mooncake.
  • PyNvVideoCodec 2.2.4: Upgraded PyNvVideoCodec from 2.2.3 to 2.2.4 in the SGLang, TensorRT-LLM, and vLLM images (#17960), a patch release with decoder fixes.
  • pyasn1 0.6.5 Floor: Floored pyasn1 at >=0.6.5,<0.7 in the SGLang and vLLM images (#17925), which otherwise kept the 0.6.4 copy from their upstream base.
  • SGLang diffusers 0.40.0: Pinned diffusers at 0.40.0 in the SGLang image (#17926), matching the TensorRT-LLM and vLLM images. The Intel XPU build keeps SGLang's own pin.

Bug Fixes

  • Oversized Prompt Rejection on TensorRT-LLM: A prompt longer than the engine's max_num_tokens now returns HTTP 400 invalid_request_error instead of HTTP 500 api_error, and the worker keeps running (#16193). This fixes the v1.6.0 known issue for non-streaming /v1/messages.
  • SGLang Garbage-Collection Stalls: Added an opt-in --freeze-gc-after-init flag (DYN_SGL_FREEZE_GC_AFTER_INIT) that freezes garbage collection after startup on SGLang decode and aggregated workers, which reduces streaming stalls in long-running workers (#15203, #16761). It is off by default. Check memory use against a control worker before enabling it widely.

Build and Test

  • Release Version Update: Set the package versions, Helm charts, and image references across the repository to 1.6.2 (#18157).
  • Kubernetes 1.36 Warning Filter: Scoped the pytest warning filter so the kr8s supported-version warning on Kubernetes 1.36 no longer fails the deploy tests (#17285).

Key Dependencies

Backend runtime versions are unchanged from v1.6.1:

Dynamo SGLang TensorRT-LLM vLLM NIXL UCX
v1.6.2 v0.5.21 v1.3.0rc29 v0.31.0 v1.5.0 (Frontend, SGLang) / v1.3.1 (TensorRT-LLM) / v1.4.1 (vLLM) v1.21.0

CUDA Variants

SGLang TensorRT-LLM vLLM
13.0 13.1 13.0

Dynamo Ecosystem

AIPerf AISimulate Grove ModelExpress
v0.13.0 v0.13.0 v0.1.0-alpha.14 v0.7.0

Known Issues

  • Carried Forward from v1.6.0: The known issues documented in the v1.6.0 release notes still apply, except the TensorRT-LLM HTTP 500 on oversized non-streaming /v1/messages requests, which this release fixes.

Full Changelog: v1.6.1...v1.6.2

Assets 2
Loading