-
Notifications
You must be signed in to change notification settings - Fork 204
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
dstack version
master
Python version
3.11
Host OS
macOS 13.4
Host Arch
ARM
What happened?
- Submit a run via Python API like so:
from dstack.api import Client, Task
from dstack.api._public.resources import GPU, Resources
task = Task(
image="ghcr.io/huggingface/text-generation-inference:latest",
env={"MODEL_ID": "TheBloke/Llama-2-13B-chat-GPTQ"},
commands=[
"text-generation-launcher --trust-remote-code --quantize gptq",
],
ports=["80"],
resources=Resources(gpu=GPU(memory="24GB")),
)
client = Client.from_config()
print(client.runs.list(all=True))
run = client.runs.submit(
run_name="my-awesome-run",
configuration=task,
repo=None,
)
- Stop the run.
- The instance will continue to run forever.
This is caused by the fact that the server allows termination_policy to be None. When None is passed, the instance is never destroyed.
Fix: always use destroy_after_idle as server default.
CLI logs
No response
Server logs
No response
Runner logs
No response
Additional Information
No response
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working