[Epic] Support clusters (aka pools of reserved instances) #790
Description
When using on-demand instances, one of the biggest challenges is the uncertainty of their availability at the time of need. For example, if you plan to use an instance for a specific task, such as a development environment or a service, stopping the run could result in the instance no longer being available. This can be problematic, especially if you need an instance on short notice.
To address this issue, reserving an instance in advance can provide several benefits. Firstly, reserved instances can be obtained at a good price. Secondly, by maintaining reserved instances in a pool, you can ensure the availability of the instance when needed.
How it may work:
$ dstack cluster create [--name|-n NAME]
The cluster `orb-galaxy-1` is created.
Then, you can add or remove instances to it:
$ dstack cluster add --gpu 24GB --nodes 3
If you want to create a cluster already with some instances, e.g. you can use dstack cluster create --gpu 24GB --nodes 3.
Finally, and most importantly, the dstack run command will allow passing a cluster (e.g. --cluster orb-galaxy-1). In this case, it will re-use the instances from the cluster.
Benefits
- You can provision an instance in advance and work with it, without fear of losing access to the instance, between restarts of your workloads.
- You can now reserve multiple instances with good GPUs at a low price in advance, perfect for team collaboration.
- This particular feature can benefit any user, irrespective of their provider. For instance, whether you're using Lambda or Vast, you can utilize dstack solely for this feature—reserving on-demand instances.
- By using this feature, we can offer dstack Cloud users reserved instances
- In the future, this feature might serve as the foundation for additional new features, such as auto-scaling of services, multi-node training, and serverless functionalities.