Steps to reproduce
- Provision many Verda instances so that some of them are failed due to no capacity, e.g.
type: fleet
name: test-verda-fleet
nodes:
min: 0
target: 30
backends: [verda]
resources:
cpu: 1..
memory: 1GB..
gpu: 0
placement: any
spot_policy: on-demand
- Even if some instance are failed in Verda they are still Provisioning in dstack which means the fleet and instances stay in Provisioning until the provisioning timeout is due (minutes=10)
Actual behaviour
Not confirmed but possible consequence of #3723
Expected behaviour
update_provisionig_data() fails the instance by raising ProvisioningError when the instance in failed or terminated in the provider.
dstack version
master
Server logs
Additional information
No response