[Bug]: [dind] dockerd fails to start sometimes

[un-def]

Steps to reproduce

0. (Presumably) use the AWS backend.
1. Start a run with the dstackai/dind:24.04-20240827003913-1 image.

Actual behaviour

start-dockerd fails after timeout:

failed to start dockerd:
1.otlp\"..." type=io.containerd.tracing.processor.v1
time="2024-10-24T14:57:10.473568977Z" level=info msg="skip loading plugin \"io.containerd.tracing.processor.v1.otlp\"..." error="skip plugin: tracing endpoint not configured" type=io.containerd.tracing.processor.v1
time="2024-10-24T14:57:10.473583117Z" level=info msg="loading plugin \"io.containerd.internal.v1.tracing\"..." type=io.containerd.internal.v1
time="2024-10-24T14:57:10.473597738Z" level=info msg="skip loading plugin \"io.containerd.internal.v1.tracing\"..." error="skip plugin: tracing endpoint not configured" type=io.containerd.internal.v1
time="2024-10-24T14:57:10.473629288Z" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1
time="2024-10-24T14:57:10.473644619Z" level=info msg="loading plugin \"io.containerd.nri.v1.nri\"..." type=io.containerd.nri.v1
time="2024-10-24T14:57:10.473656019Z" level=info msg="NRI interface is disabled by configuration."
time="2024-10-24T14:57:10.473845093Z" level=info msg=serving... address=/var/run/docker/containerd/containerd-debug.sock
time="2024-10-24T14:57:10.473953775Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock.ttrpc
time="2024-10-24T14:57:10.474049326Z" level=info msg=serving... address=/var/run/docker/containerd/containerd.sock
time="2024-10-24T14:57:10.474076367Z" level=info msg="containerd successfully booted in 0.028706s"
time="2024-10-24T14:57:11.433134367Z" level=info msg="detected 127.0.0.53 nameserver, assuming systemd-resolved, so using resolv.conf: /run/systemd/resolve/resolv.conf"

Run failed with error code CONTAINER_EXITED_WITH_ERROR.
Check CLI, server, and run logs for more details.

Expected behaviour

No response

dstack version

0.18.20

Server logs

No response

Additional information

No response

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[un-def]

Fixed in #1908 with increased default timeout