Dynamo v1.3.1

Dynamo v1.3.1 - Release Notes

Summary

Dynamo v1.3.1 is a patch release on top of v1.3.0, addressing disaggregated SGLang serving over AWS EFA on GB200. The SGLang EFA runtime moves to the released NIXL 1.3.2 wheel, and all three EFA runtime images move to EFA Installer 1.49.0.

Base Branch: release/1.3.1

Bug Fixes

• EFA / SGLang Disaggregated Serving on GB200: Fixed a GB200-specific KV-transfer stall in which disaggregated SGLang inference hung and returned an empty response over AWS EFA (#12106). The SGLang EFA runtime now installs the released nixl==1.3.2 wheel in place of the NIXL built into the upstream framework image, and all three EFA images move to EFA Installer 1.49.0.

Key Dependencies

The following dependencies changed in this release:

Dependency	Version
NIXL (SGLang EFA runtime)	1.3.2 (released wheel)
EFA Installer	1.49.0
libfabric (EFA runtime images)	2.4.0amzn5.0 (EFA installer stock)

The EFA Installer and libfabric changes apply to all three -efa runtime images. The NIXL wheel change applies to sglang-runtime only.

Backend runtime versions are unchanged from v1.3.0:

Backend	Version	CUDA
vLLM	0.23.0	13.0
SGLang	0.5.14	13.0
TensorRT-LLM	1.3.0rc19	13.1

Known Issues

• Intermittent stall in disaggregated serving over EFA with SGLang: Requests return an empty HTTP 200 response with zero completion tokens after approximately 300 seconds. The condition is intermittent and more likely on newly started decode workers, including after a restart. It affects SGLang disaggregated serving on the NIXL LIBFABRIC backend over EFA; other transports are unaffected. The defect is in the EFA provider in libfabric. The data path and the network are healthy and EFA counters show no drops or errors, so the failure surfaces as a silent stall with no error reported. Targeted fix: an AWS EFA release in the coming weeks. It is not included in this release.

• Partial EFA device allocation can mismatch GPU PCIe topology: Workers that request fewer than all EFA devices on a node may be allocated devices that are not on the same PCIe switch as their GPUs, and disaggregated KV transfer fails. Affected requests return an empty HTTP 200 response with zero completion tokens after approximately 10 to 20 seconds, and the decode worker logs Lost connection with prefill instance. This is a limitation of EFA device allocation on Kubernetes and is not specific to Dynamo: GPUs and EFA devices are allocated by two independent device plugins, and upstream Kubernetes provides no mechanism for one plugin to observe another's allocations. The impact is greatest where each GPU is paired one-to-one with an EFA NIC, as on GB200, since a mismatched allocation leaves a GPU with no local network device. Mitigations, in order of preference: request all EFA devices on the node, which constrains scheduling to one worker per node; or use EFA DRA together with NVIDIA DRA, which allows requesting a GPU and EFA devices on the same PCIe switch. See the EKS EFA device plugin documentation.

• dynamo-mocker aic-forward-pass feature on crates.io: The published dynamo-mocker 1.3.0 and 1.3.1 crates do not compile with aic-forward-pass enabled, or with --all-features. Their aiconfigurator-core requirement resolves to 0.10.0, which added required cp_size and perf_db_sources fields that the mocker source does not supply. Consumers building with default features are unaffected, as are the published wheels and containers: the wheel builder compiles the feature from local workspace source at the pinned commit and does not read the crates.io metadata. No published aiconfigurator-core version satisfies the feature, since 0.9.0 predates the pinned commit and fails on missing APIs. Workaround: build dynamo-mocker from release-branch source. Targeted fix: a corrective dynamo-mocker release once a matching aiconfigurator-core is published.

• Carried forward from v1.3.0: The known issues documented in the v1.3.0 release notes still apply.

---