Skip to content

[Bug]: Runs submitted via Python API have no termination policy #955

@r4victor

Description

@r4victor

dstack version

master

Python version

3.11

Host OS

macOS 13.4

Host Arch

ARM

What happened?

  1. Submit a run via Python API like so:
from dstack.api import Client, Task
from dstack.api._public.resources import GPU, Resources

task = Task(
    image="ghcr.io/huggingface/text-generation-inference:latest",
    env={"MODEL_ID": "TheBloke/Llama-2-13B-chat-GPTQ"},
    commands=[
        "text-generation-launcher --trust-remote-code --quantize gptq",
    ],
    ports=["80"],
    resources=Resources(gpu=GPU(memory="24GB")),
)

client = Client.from_config()
print(client.runs.list(all=True))
run = client.runs.submit(
    run_name="my-awesome-run",
    configuration=task,
    repo=None,
)

  1. Stop the run.
  2. The instance will continue to run forever.

This is caused by the fact that the server allows termination_policy to be None. When None is passed, the instance is never destroyed.

Fix: always use destroy_after_idle as server default.

CLI logs

No response

Server logs

No response

Runner logs

No response

Additional Information

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions