[Roadmap] Q2 2024

[peterschmidt85]

This issue outlines the major items planned for Q2 2024. Note that it doesn't include bug fixes, except for major issues.

Warning

This roadmap is replaced by #1350.

Core features

• Implement dstack apply Implement dstack apply for gateways #1223 Support running run configurations with dstack apply #1326 Introduce Fleets for cluster provisioning and management #1327 dstack apply should be in the attached mode by default for any configuration type #1571 [UX] Show a plan for applying a fleet #1472
• Allow attaching volumes [Feature]: Support dstack volumes #1158 Support multi-attach volumes #1467
• Multi-node support Support multinode training #1103 Support multi-host TPUs #1337
• Support for on-prem servers Added the ability of adding own instances #1115 Fix DSTACK_NODE_RANK #1189
• Support private subnets Support provisioning instances without public IPs on AWS #1201 Support internal-facing gateways on AWS #1171 Support gateways behind ALB with ACM certificate #1264 Support specifying non-default VPC for GCP backend #1304
• Allow instance reuse by other users The instance cannot be reused by another user #896
• Make sure retry policy works for all types of instances Improve retry policy #1200
• More granular permissions Narrow the scope of required GCP permissions and make them more granular #1101 Narrow down required Azure permissions #1138
• Parity YAML and CLI [UX] Allow the configuration YAML file to configure any option supported by .dstack/profiles.yml #943 [Feature]: Global ".dstack/profile" #1011

Architecture

• AMD GPU Support AMD with SSH fleets #1413
• Intel Gaudi
• TPU [Feature]: Add TPU support #956 Support multi-host TPUs #1337
• Intel GPU
• AWS Inferentia
• AWS Trainium

New providers

• RunPod Add runpod backend #1063 [Bug]: The runpod backend always uses the Docker image's default entrypoint #1137 Issues with RunPod as an offline provider #1118 [Bug]: Private registry_auth does not seem to be respected for Runpod deployments #1297
• Oracle [Feature]: Oracle Cloud Infrastructure backend #1194

Provider improvements

• Reserved instances in AWS [Feature]: Support the Capacity Blocks for ML feature for AWS #1155
• H100 support in Azure [Feature]: Support H100 with Azure #1239
• Reserved instances in GCP
• Reserved instances in Azure
• AMD support in Azure
• AMD support in GCP
• Spot instances in RunPod
• Bare metal support in Oracle [Feature]: Oracle Cloud Infrastructure backend #1194
• Multi-node support in Oracle [Feature]: Oracle Cloud Infrastructure backend #1194
• Spot instances in Oracle
• Spot instances in TensorDock
• Spot instances in Vast.ai
• A10 support in Azure [Bug] Cannot provision instances with the A10 GPU on the azure backend #1014
• H100 support in AWS [Feature]: Support H100 with AWS #1238
• H100 support in GCP [Feature]: Support H100 with GCP #1240
• L4 support in AWS Support NVIDIA L4 with AWS (g6.* instance type) #1235

Examples

Important

Community help is needed!

• TensorRT-LLM
• Triton
• Function calling
• LoRAX
• H4 Alignment Handbook Example/h4alignment handbook #1180
• Axolotl Examples/axolotl #1187

Documentation

• Troubleshooting
• Protips

Important

If you notice something important missing here, please write about it in the comments.

[peterschmidt85]

Replaced by #1350