Skip to content

0.21.0

Latest
Latest

Choose a tag to compare

View all tags
@r4victor r4victor released this 06 Aug 13:31
0.21.0
7aaae67
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

dstack 0.21.0 is a major release that migrates the codebase to Pydantic v2, introduces new features, and removes a number of previously deprecated features. New 0.21.x CLIs do not work with older 0.20.x servers. New servers continue to support older CLIs, so ensure to update the server and the CLI at the same time or update the server first.

Pydantic v2

dstack has finally migrated from Pydantic v1 to Pydantic v2. Besides faster validation and serialization and faster CLI startup, the migration unblocks Python 3.14 support and integration with latest Python libraries.

If you use the Python API or have dstack plugins installed, ensure the code works with Pydantic v2 models before upgrading dstack.

Python 3.14

dstack now supports Python 3.14 both for installing dstack and inside runs:

type: dev-environment
ide: vscode
python: "3.14"

Gateways

Replicated gateways with HTTPS

Replicated gateways previously required an external load balancer for TLS termination. AWS gateways with an acm certificate can now have more than one replica with HTTPS handled by dstack:

type: gateway
name: example-gateway

backend: aws
region: eu-west-1

domain: example.com

certificate:
  type: acm
  arn: arn:aws:acm:eu-west-1:164099421079:certificate/3670388f-f43b-4872-aaf8-907b107a170d

replicas: 2

Load balancing across replicas is performed by a single ALB associated with the gateway, which becomes the gateway's hostname:

$ dstack gateway list
 NAME             BACKEND          HOSTNAME                                                  DOMAIN       DEFAULT  STATUS
 example-gateway                   dstack-qe1na76o-lb-187858581.eu-west-1.elb.amazonaws.com  example.com  ✓        running
    replica=0     aws (eu-west-1)  18.202.25.65                                                                    running
    replica=1     aws (eu-west-1)  3.255.100.238                                                                   running

Replicated gateways remain experimental. For other backends and certificate types, an external load balancer is still required for TLS termination.

CLI

Improved dstack metrics

dstack metrics now shows CPU, memory, and GPU utilization over the last hour of the job, allowing to track changes in metrics:

$ dstack metrics gentle-mayfly-1

        UTILIZATION                             MEMORY
 cpu    ▅▄▄▄▃▃▃▃▃▃▃▃▃▃▃▃▅▅▄▃▃▃▃▃▃▃▃ 41% of 128  ▃▃▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄ 581GB/960GB

 gpu=0  ▁▂▃▆▆▆▆▆▆▆▆▆▆▆▆▆▆▁▆▆▆▆▆▆▆▆▆ 89%         ▄▅▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆ 71GB/80GB
 gpu=1  ▁▂▆▆▆▅▆▆▆▆▆▆▆▆▆▆▁▁▅▆▆▅▆▆▆▆▆ 84%         ▄▅▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆ 71GB/80GB
 gpu=2  ▁▂▆▆▆▆▆▆▆▆▆▆▆▆▆▆▁▆▆▆▆▆▆▆▆▆▆ 87%         ▄▅▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆ 71GB/80GB
 gpu=3  ▂▃▆▅▅▅▅▅▅▆▅▅▆▆▆▆▁▅▅▅▅▅▆▅▅▅▅ 82%         ▄▅▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆ 71GB/80GB

        4 Aug 14:10 ┄┄┄┄┄┄┄┄┄┄┄ now             4 Aug 14:10 ┄┄┄┄┄┄┄┄┄┄┄ now

Like dstack logs, the command now shows a single job: --replica and --job select one, both defaulting to 0.

GPU driver

dstack fleet -v now shows a DRIVER column with the accelerator driver installed on the host (NVIDIA, AMD, or Tenstorrent), also available as Instance.gpu_driver in the API. The driver is detected by dstack-shim and refreshed on instance checks, so it is filled in for existing instances after an upgrade and updated after a driver upgrade on the host. Container-based backends report no driver for now.

Offers

dstack offer and dstack apply gain two new flags: --full-offers and --unallocated (and corresponding full_offers and unallocated_resources API parameters):

  • --full-offers requests not to adjust offers by requirements on backends that pack multiple jobs onto a node and return full node resources. Currently supported by Kubernetes and Slurm backends.
  • --unallocated subtracts already-allocated resources on backends that pack multiple jobs onto a node, so offers reflect only the available capacity. Currently supported by Kubernetes backend.

The flags are discovery-only.

Presets

dstack preset create gains a pinned benchmark workload, a baseline trial to measure against, and a per-trial record of what each trial taught. The properties that decide what "verified" means are now required rather than optional, so two creations from the same configuration produce comparable presets.

type: preset
name: dsv4-flash

# The agent picks a compatible variant of the base model
base: deepseek-ai/DeepSeek-V4-Flash

# Consider only the specified fleets
fleets: [b200-2x]

# The requirements the preset must meet (time to first token is in milliseconds)
min_context_length: 1048576
max_ttft: 675

# The number of simultaneous requests every benchmark uses
concurrency: 1

# The request shape every benchmark uses (defaults to 1024 and 1024)
input_tokens: 10000
output_tokens: 1500

# Make the first trial a reference point rather than an optimization attempt
baseline: true

# The number of benchmarked trials
trials: 10

Each trial now records what it learned, whether it broke a constraint, and the largest context it actually handled, so context_length on a saved preset is measured rather than assumed. A trial that breaks a constraint keeps its benchmark for the next trial to learn from and is excluded from best-trial selection.

dstack preset gains ps-style filtering with -a, -n, --base, and --repo, CONSTRAINTS and BENCHMARK columns, and one glyph per trial:

$ dstack preset -a --base deepseek-ai/DeepSeek-V4-Flash
 ID        BASE                           GPU           CONSTRAINTS                   BENCHMARK                          STATUS              SUBMITTED
 c83375b4  deepseek-ai/DeepSeek-V4-Flash  B200:180GB:2  io=10000/1500 conc=1          tok/s/user=319 ttft=220ms ctx=1M    interrupted (5/10)  8 hours ago
 de7a07c1  deepseek-ai/DeepSeek-V4-Flash  B200:2        io=10000/1500 conc=1          tok/s/user=140 ttft=519ms ctx=1M    verified (5)        19 hours ago
 014c3216  deepseek-ai/DeepSeek-V4-Flash  H100:80GB:4   io=8K/1K prefix=90% conc=648  tok/s/user=11.2 ttft=4.89s ctx=1M   verified (8)        yesterday

Presets remain an experimental feature. The configuration properties changed in a backward-incompatible way, see the breaking changes below.

Backends

AWS

The AWS backend gains an experimental_instance_types setting that allows provisioning instance types outside the standard supported families, as long as they are present in dstack's pricing catalog:

projects:
  - name: main
    backends:
      - type: aws
        creds:
          type: default
        experimental_instance_types: [p5en.48xlarge]
$ dstack offer -b aws --instance-type p5en.48xlarge
 #   BACKEND           RESOURCES                                              INSTANCE TYPE  PRICE
 1   aws (us-east-2)   cpu=192 mem=2048GB disk=100GB gpu=H200:141GB:8 (spot)  p5en.48xlarge  $26.6319
 2   aws (us-west-2)   cpu=192 mem=2048GB disk=100GB gpu=H200:141GB:8 (spot)  p5en.48xlarge  $26.9512
 3   aws (us-east-2)   cpu=192 mem=2048GB disk=100GB gpu=H200:141GB:8         p5en.48xlarge  $63.296

Slurm

Slurm offers no longer report a fake disk size derived from the requested disk.size range. Disk size is now reported as unknown, and offers are no longer filtered by the disk requirement instead of promising storage that may not be there.

Breaking changes

  • The dstack Python API and plugins now work with Pydantic v2 models. Update the code before upgrading dstack.
  • Dropped the /api/project/{project_name}/runs/submit endpoint, deprecated in favor of /api/project/{project_name}/runs/apply.
  • Dropped the /api/project/{project_name}/fleets/create endpoint, deprecated in favor of /api/project/{project_name}/fleets/apply.
  • Dropped the top-level router property from gateway and run configurations, deprecated since 0.20.17 in favor of replica-based routers.
  • The server no longer accepts resources.cpu as an integer range in the request format used by clients older than 0.19.8. Upgrade the CLI and API clients before upgrading the server.
  • UTC datetimes are serialized as Z instead of +00:00.
  • Preset configurations (experimental): max_trials is now trials, context_length is now min_context_length, and max_ttft, min_context_length, and concurrency no longer have defaults. Existing configurations fail with extra fields not permitted; no aliases were added.

Deprecations

  • Resources.description, Gateway.backend, and Gateway.region are no longer populated by the server and are excluded by the client. They will be removed in 0.22.

What's Changed

New Contributors

Full Changelog: 0.20.29...0.21.0

Contributors

complynx, un-def, and 4 other contributors
Assets 2
Loading