0.21.1

Presets

Patching framework

Presets can now patch source code (e.g. serving frameworks, kernels, etc). Patches are stored with the preset as unified diffs and applied when the preset starts, so dstack preset apply reproduces the optimized environment.

type: service

files:
  - local_path: patches/rocm_aiter_mla.py.patch
    path: /patches/rocm_aiter_mla.py.patch

commands:
  - for p in /patches/*.patch; do patch -p1 -d / < "$p"; done
  - vllm serve ...

Previous sessions

Agents can now use results from previous preset sessions to continue optimization instead of starting from scratch. Pass one or more session IDs with --previous:

$ dstack preset create -f config.yml --previous e8b7e09c

The previous configuration property can also be used to provide multiple sessions.

baseline now defaults to true, ensuring every session has an anchor for comparison. The first trial uses the framework's recommended configuration, or reproduces the previous best when building on a previous session.

Benchmark datasets

Presets can benchmark against real datasets instead of synthetic prompts. Set dataset to a supported benchmark dataset or a Hugging Face dataset ID:

dataset: sharegpt

When a dataset is specified, it provides the requests, so input_tokens, output_tokens, and shared_prefix_tokens are no longer set explicitly. The preset records the measured means instead. Gated datasets can be accessed by providing HF_TOKEN in env.

dstack preset now displays the dataset in place of the synthetic request shape.

Failed preset creations are also now shown by dstack preset, making unsuccessful optimization attempts visible instead of silently disappearing.

Metrics

dstack metrics can now display all jobs and replicas of a run at once.

$ dstack metrics vllm-qwen

                         UTILIZATION                  MEMORY
 replica=0 job=0  cpu    ▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂ 20%  ▁▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂ 85GB/440GB
                  gpu=0  ▄▄▃▃▃▃▃▃▃▃▄▄▅▅▅▅▅▅▅▄▄▄▄ 51%  ▃▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅ 60GB/80GB

 replica=1 job=0  cpu    ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ 6%   ▁▁▁▁▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▁▁▁ 73GB/440GB
                  gpu=0  ▂▂▁▁▁▁▁▁▁▁▂▂▂▂▃▃▃▃▃▂▂▁▁ 14%  ▃▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅ 60GB/80GB

 replica=2 job=0  cpu    ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ 1%   ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ 61GB/440GB
                  gpu=0  ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ 2%   ▃▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅ 60GB/80GB

                         6 Aug 17:11 ┄┄┄┄┄┄┄ now      6 Aug 17:11 ┄┄┄┄┄┄┄ now

Troubleshooting

Provisioning failures now provide more actionable status and error information when all provisioning attempts fail.

$ dstack apply -y

my-task provisioning completed (failed)
No capacity
Failed to provision in fleet 'cloud-fleet': tried 1 of 1 offers, all failed.
Errors: p5.48xlarge in aws/us-west-1: NoCapacityError.
See https://dstack.ai/docs/guides/troubleshooting/#provisioning-fails

Gateways

Scaling

Gateways can now be scaled out and scaled in by updating the replicas property in-place:

$ dstack apply -f my-gateway.dstack.yml
Found gateway my-gateway. Detected changes that can be updated in-place:
- replicas

Update the gateway? [y/n]:

When scaling in, dstack stops the oldest gateway replicas first.

Fault tolerance

An unhealthy gateway replica no longer prevents the service from provisioning, as long as the service can be registered on at least one healthy gateway replica. Similarly, an unhealthy gateway replica does not prevent service replica provisioning, as long as the service replica can be registered on another gateway replica.

Registration progress for each gateway replica can be tracked in events.

$ dstack event --within-run my-service | grep Service
[2026-08-14 17:51:10] [run my-service, gateway test-gateway] Service registered on gateway replica 0
[2026-08-14 17:51:10] [run my-service, gateway test-gateway] Service registered on gateway replica 1
[2026-08-14 17:52:03] [job my-service-0-1] Service replica ready to receive requests
[2026-08-14 17:52:03] [job my-service-0-0] Service replica ready to receive requests
[2026-08-14 17:52:23] [job my-service-0-1, gateway test-gateway] Service replica registered on gateway replica 1
[2026-08-14 17:52:23] [job my-service-0-0, gateway test-gateway] Service replica registered on gateway replica 1
[2026-08-14 17:52:23] [job my-service-0-1, gateway test-gateway] Service replica registered on gateway replica 0
[2026-08-14 17:52:23] [job my-service-0-0, gateway test-gateway] Service replica registered on gateway replica 0

During rolling deployments, however, the old service replica is not terminated until the new service replica has been registered on all gateway replicas.

Backends

AWS

AWS p5en.48xlarge instances are now supported, with EFA configured automatically for improved connectivity in clusters.

$ dstack offer -b aws --instance-type p5en.48xlarge --max-offers 5
 #   BACKEND           RESOURCES                                              INSTANCE TYPE  PRICE
 1   aws (us-east-2)   cpu=192 mem=2048GB disk=100GB gpu=H200:141GB:8 (spot)  p5en.48xlarge  $26.5714
 2   aws (us-west-2)   cpu=192 mem=2048GB disk=100GB gpu=H200:141GB:8 (spot)  p5en.48xlarge  $26.9369
 3   aws (us-east-1)   cpu=192 mem=2048GB disk=100GB gpu=H200:141GB:8 (spot)  p5en.48xlarge  $27.1988
 4   aws (eu-north-1)  cpu=192 mem=2048GB disk=100GB gpu=H200:141GB:8 (spot)  p5en.48xlarge  $39.8158
 5   aws (us-east-2)   cpu=192 mem=2048GB disk=100GB gpu=H200:141GB:8         p5en.48xlarge  $63.296

Runpod

dstack no longer provides Runpod spot offers, since Runpod announced that Spot and Interruptible Pods will no longer be available after Friday, September 11th.

Breaking changes

The deprecated dstack_server_request* Prometheus metrics are no longer available. Use the OpenTelemetry http_server_* metrics instead.

What's changed

• Forbid agents to create git tags outside the release process by @r4victor in #4107
• Add AWS p5en.48xlarge (8:H200 + Sapphire Rapids) by @jvstme in #4110
• Improve provisioning failure message by @r4victor in #4113
• Add Tags to ServiceConfiguration.replicas by @un-def in #4112
• Minor updates to docs and README by @peterschmidt85 in #4115
• [runner,shim] Rework GPU autodetection by @un-def in #4116
• Show every job in dstack metrics by @peterschmidt85 in #4108
• Fix busy_blocks lost decrement when terminating jobs concurrently by @r4victor in #4120
• Rework GPU vendor/CPU arch and image validation by @un-def in #4114
• Use tmp dir for SERVER_DIR_PATH in tests by @r4victor in #4121
• Drop deprecated dstack_server_request* Prometheus metrics by @r4victor in #4130
• Add missing foreign key indexes by @r4victor in #4131
• Distinguish router and non-router replicas by @jvstme in #4101
• Support in-place update for gateway default by @jvstme in #4128
• Show failed creations in dstack preset by @peterschmidt85 in #4109
• Keep now on the metrics axis while a job is reporting by @peterschmidt85 in #4132
• [Presets] Patching framework and --previous support by @peterschmidt85 in #4118
• Simplify certbot error messages by @jvstme in #4133
• Bump testcontainers>=4.15.0 by @un-def in #4137
• Ignore google package warning about near-EOL Python version in tests by @un-def in #4138
• [Presets] Benchmark dataset support by @peterschmidt85 in #4134
• Fix IndexError in dstack apply by @jvstme in #4141
• Gateway state sync mechanism by @jvstme in #4135
• Allow manual gateway scaling (drop feature flag) by @jvstme in #4142
• Update gpuhunt contributing guide by @r4victor in #4143
• Drop pydantic_compat tests by @r4victor in #4144
• Fix server compatibility with 0.20.[27-29] clients by @jvstme in #4148

Full Changelog: 0.21.0...0.21.1