Skip to content

[Bug]: Python API Run.logs() returns nothing for a finished run #4175

Description

@r4victor

Steps to reproduce

from dstack.api import Client
c = Client.from_config()
r = c.runs.get("<a run that has finished>")
print(len(b"".join(r.logs())))                 # 0
print(len(b"".join(r.logs(replica_num=0))))    # the actual logs

Actual behaviour

logs() yields nothing for any terminal run - no exception, no warning. dstack logs <same run> prints the logs in full.

_find_job matches on replica_num alone when it is given, but when it is None it matches only a submission whose status is RUNNING:

def _find_job(self, replica_num: Optional[int], job_num: int) -> Optional[Job]:
for j in self._run.jobs:
if (
replica_num is not None
and j.job_spec.replica_num == replica_num
or replica_num is None
and j.job_submissions[-1].status == JobStatus.RUNNING
) and j.job_spec.job_num == job_num:
return j
return None

Once a run is terminal nothing is RUNNING, so _find_job returns None and logs() returns an empty generator. The CLI is unaffected because its --replica argparse default is 0, so it never passes None.

Expected behaviour

logs() with default arguments should return the logs of a finished run, as the CLI does - fall back to replica 0 when no submission is running, or raise instead of yielding nothing.

dstack version

master at 2517069, server run from source.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions