Skip to content

[Bug]: 0.19.27 client (unreleased) is not compatible with pre-0.19.27 servers #3062

@un-def

Description

@un-def
  • client - master (be36bde)
  • server, shim, runner - 0.19.26

Here is an example of a configuration that works with client 0.19.26, but stops working after updating the client to master:

> cat test.dstack.yml
type: task
commands:
- echo hi
- sleep 5

> git checkout 0.19.26

> dstack apply -f test.dstack.yml --reuse -y

weak-panther-1 provisioning completed (running)
hi
Exited (0)

> git checkout master

> git log --oneline | head -1
be36bde4 Bind shim and exposed container ports to localhost (#3057)

> dstack apply -f test.dstack.yml --reuse -y

working_dir is not set. Using legacy working directory /workflow

In future versions the default value will be taken from the image
- To keep using legacy working directory, set working_dir to /workflow
- To prepare for the future default, set working_dir to the image default working directory (only until the new default takes effect)

 NAME            BACKEND          RESOURCES                        PRICE    STATUS   SUBMITTED 
 nasty-wombat-1  aws (eu-west-3)  cpu=2 mem=8GB disk=100GB (spot)  $0.0287  running  15:30     

nasty-wombat-1 provisioning completed (running)
Exited (none)
Check dstack logs -d nasty-wombat-1 for more details.

> dstack logs -d nasty-wombat-1
time=2025-09-03T09:31:28.065881-04:00 level=info msg=Run job log_level=trace
time=2025-09-03T09:31:28.065987-04:00 level=trace msg=checking if repo checkout is needed
time=2025-09-03T09:31:28.068047-04:00 level=trace msg=Extracting tar archive
time=2025-09-03T09:31:28.068097-04:00 level=trace msg=Extracting code archive dst=/workflow src=/tmp/runner/code
time=2025-09-03T09:31:28.069852-04:00 level=info msg=Job state changed new=running
time=2025-09-03T09:31:28.080899-04:00 level=info msg=Continuing without updating LD_LIBRARY_PATH
time=2025-09-03T09:31:28.080954-04:00 level=error msg=Exec failed err=[executor.go:371 executor.(*RunExecutor).execJob] [exec.go:12 executor.joinRelPath] path must be relative
time=2025-09-03T09:31:28.081019-04:00 level=info msg=Job state changed new=failed

Originally posted by @jvstme in #3041 (comment)

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions