0.21.4

Presets

PD disaggregation

Presets can now be created for prefill-decode disaggregated setups. Previously the agent was restricted to single-node trials, and PD disaggregation was listed as a limitation.

The agent now runs multi-node trials using task node groups, benchmarks the router replica rather than individual prefill or decode workers (each of which only handles part of a request), and records the resources of every node of every group in the trial results.

Grouped services are also exported and stored in the groups syntax:

type: service
name: qwen3-pd

groups:
  - replicas: 1   # router
    commands: [...]
  - replicas: 2   # prefill
    commands: [...]
  - replicas: 2   # decode
    commands: [...]

dstack preset -v reports the hardware of all nodes a session's trials used, e.g. H200:141GB:8 x3 for a CPU router with one prefill and two decode nodes, or H200:141GB:8 + H100:80GB:8 x2 when the groups ran different GPUs.

Note

PD disaggregation requires the trial task and the resulting service to run in a fleet with placement: cluster, since the replicas need an interconnect between instances.

Services

Replica groups

Heterogeneous services are now stored and returned in the groups layout introduced in 0.21.3, instead of being converted back to the legacy replicas: [{count}] list. The legacy layout is still accepted in configuration files, and servers still send it to clients older than 0.21.3.

Warning

Ensure you update the CLI to 0.21.5, which includes a hotfix for using the CLI with servers older than 0.21.3.

Gateways

Gateways have been tuned to handle more concurrent connections.

Instances

Shim upgrades on busy instances

dstack-shim can now be upgraded on instances with running jobs, as long as the instance runs shim 0.21.3 or later. Previously the server only restarted the shim when no task was running, so an instance hosting a long-lived service or a multi-day training job kept its old shim indefinitely.

SSH access to co-located jobs

On an instance that runs several of your jobs at once — a fleet with blocks — the first job to finish no longer revokes SSH access for the jobs still running.

Backends

Container-based backends (Kubernetes, Runpod, Slurm, and Vast.ai) now honor DSTACK_SERVER_SSHPROXY_ENFORCED. Previously they added the user's key to the container regardless of the setting, unlike VM-based backends. See Tenant isolation.

What's changed

• Allow restarting shim >= 0.21.3 with running tasks by @un-def in #4224
• [shim] Keep shim task files out of the container by @un-def in #4223
• [shim] Don't revoke host SSH access of co-located tasks by @un-def in #4226
• [chore]: Drop gateway-based routers implementation by @jvstme in #4236
• Fix adding authorized keys in Kubernetes and Slurm by @un-def in #4235
• Tune Nginx config on the gateway by @jvstme in #4237
• Replace the legacy replica groups serializer with a client patch by @Bihan in #4248
• [Presets] Add PD Disaggregation Support by @Bihan in #4227
• Honor sshproxy enforcement in container-based backends by @un-def in #4244
• Drop gateway update.sh by @jvstme in #4245
• Fix target type filter on Events page by @hhhhzzzj in #4253
• Drop legacy dstack-gateway package by @jvstme in #4252

New contributors

• @hhhhzzzj made their first contribution in #4253

Full changelog: 0.21.3...0.21.4