-
Notifications
You must be signed in to change notification settings - Fork 204
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Steps to reproduce
- Create a fleet with an on-prem server
- Reboot this server
Actual behaviour
dstack-shim.service fails to start
Expected behaviour
dstack-shim.service starts
dstack version
0.18.7
Server logs
No response
Additional information
$ journalctl -u dstack-shim --boot=0
Jul 31 04:45:49 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 1.
Jul 31 04:45:49 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 2.
Jul 31 04:45:49 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 3.
Jul 31 04:45:50 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 4.
Jul 31 04:45:50 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 5.
Jul 31 04:45:50 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Start request repeated too quickly.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:50 defpc systemd[1]: Failed to start dstack-shim.
$ journalctl --boot=0 | grep -E 'Reached target Network|dstack-shim'
Jul 31 04:45:49 defpc systemd[1]: Reached target Network.
Jul 31 04:45:49 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 1.
Jul 31 04:45:49 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 2.
Jul 31 04:45:49 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:49 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 3.
Jul 31 04:45:50 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 4.
Jul 31 04:45:50 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: Started dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Main process exited, code=exited, status=1/FAILURE
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Scheduled restart job, restart counter is at 5.
Jul 31 04:45:50 defpc systemd[1]: Stopped dstack-shim.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Start request repeated too quickly.
Jul 31 04:45:50 defpc systemd[1]: dstack-shim.service: Failed with result 'exit-code'.
Jul 31 04:45:50 defpc systemd[1]: Failed to start dstack-shim.
Jul 31 04:45:53 defpc systemd[1]: Reached target Network is Online.
# tail -5 /root/.dstack/shim.log
[runner.go:37 shim.(*CLIArgs).DownloadRunner] [runner.go:92 shim.downloadRunner] Get "https://dstack-runner-downloads-stgn.s3.eu-west-1.amazonaws.com/latest/binaries/dstack-runner-linux-amd64": dial tcp: lookup dstack-runner-downloads-stgn.s3.eu-west-1.amazonaws.com on 127.0.0.53:53: server misbehaving
2024/07/31 04:45:50 Downloading runner from https://dstack-runner-downloads-stgn.s3.eu-west-1.amazonaws.com/latest/binaries/dstack-runner-linux-amd64
[runner.go:37 shim.(*CLIArgs).DownloadRunner] [runner.go:92 shim.downloadRunner] Get "https://dstack-runner-downloads-stgn.s3.eu-west-1.amazonaws.com/latest/binaries/dstack-runner-linux-amd64": dial tcp: lookup dstack-runner-downloads-stgn.s3.eu-west-1.amazonaws.com on 127.0.0.53:53: server misbehaving
2024/07/31 04:45:50 Downloading runner from https://dstack-runner-downloads-stgn.s3.eu-west-1.amazonaws.com/latest/binaries/dstack-runner-linux-amd64
[runner.go:37 shim.(*CLIArgs).DownloadRunner] [runner.go:92 shim.downloadRunner] Get "https://dstack-runner-downloads-stgn.s3.eu-west-1.amazonaws.com/latest/binaries/dstack-runner-linux-amd64": dial tcp: lookup dstack-runner-downloads-stgn.s3.eu-west-1.amazonaws.com on 127.0.0.53:53: server misbehaving
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working