Docker Desktop fails to start after unclean shutdown.

[radoslawkoziol]

• I have tried with the latest version of Docker Desktop
• I have tried disabling enabled experimental features
• I have uploaded Diagnostics
• Diagnostics ID: A5263C4A-D100-4B3A-82C1-5EA8F4B756D1/20230507111128

Actual behavior

Docker desktop doesn't start with PC startup. It happens randomly, not always. Nothing seems to be fixing it, besides restarting PC and trying again.

System.Exception:
1 error occurred:
	* waiting for docker daemon: context deadline exceeded


   w Docker.Engines.WSL2.LinuxWSL2Engine.<DoStartAsync>d__10.MoveNext() w C:\workspaces\PR-21718\src\github.com\docker\pinata\win\src\Docker.Engines\WSL2\LinuxWSL2Engine.cs:wiersz 93
--- Koniec śladu stosu z poprzedniej lokalizacji, w której wystąpił wyjątek ---
   w System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   w System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   w Docker.ApiServices.StateMachines.TaskExtensions.<WrapAsyncInCancellationException>d__0.MoveNext() w C:\workspaces\PR-21718\src\github.com\docker\pinata\win\src\Docker.ApiServices\StateMachines\TaskExtensions.cs:wiersz 29
--- Koniec śladu stosu z poprzedniej lokalizacji, w której wystąpił wyjątek ---
   w System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   w System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   w Docker.ApiServices.StateMachines.StartTransition.<DoRunAsync>d__7.MoveNext() w C:\workspaces\PR-21718\src\github.com\docker\pinata\win\src\Docker.ApiServices\StateMachines\StartTransition.cs:wiersz 79
--- Koniec śladu stosu z poprzedniej lokalizacji, w której wystąpił wyjątek ---
   w System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   w Docker.ApiServices.StateMachines.StartTransition.<DoRunAsync>d__7.MoveNext() w C:\workspaces\PR-21718\src\github.com\docker\pinata\win\src\Docker.ApiServices\StateMachines\StartTransition.cs:wiersz 108
--- Koniec śladu stosu z poprzedniej lokalizacji, w której wystąpił wyjątek ---
   w System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   w Docker.Engines.Engines.<WaitCurrentTransitionsAsync>d__31.MoveNext() w C:\workspaces\PR-21718\src\github.com\docker\pinata\win\src\Docker.Engines\Engines.cs:wiersz 292
--- Koniec śladu stosu z poprzedniej lokalizacji, w której wystąpił wyjątek ---
   w System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   w System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   w Docker.Engines.Engines.<StartAsync>d__25.MoveNext() w C:\workspaces\PR-21718\src\github.com\docker\pinata\win\src\Docker.Engines\Engines.cs:wiersz 116

Information

It happens randomly. I'm not sure what was the docker version that started this problem, but it happens for some time already.

• WSL2 (1.2.5.0)
• Windows 10 (10.0.19045.2846)
• Docker Desktop version 4.19.0 (106363)

• Windows Version: Windows 10 (10.0.19045.2846)
• Docker Desktop Version: 4.19.0 (106363)
• WSL2 or Hyper-V backend? WSL2
• Are you running inside a virtualized Windows e.g. on a cloud server or a VM: no

Steps to reproduce the behavior

Happens randomly.

[pmev0]

for me it never works. PLEASE FIX ffs

[ctalledo]

Hi @radoslawkoziol, thanks for filing the issue.

Per the diagnostics bundle you uploaded, first error I see is:

log/host/com.docker.backend.exe.log.6: 2023-05-05T 19:29:17.323036800Z [com.docker.backend.exe][E] getting WSL version: exit status 1

Could you run wsl --version on a Windows command prompt or PowerShell and post the output please?

NOTE: likely duplicate of #13445.

[radoslawkoziol]

@ctalledo sure:

C:\Users\Radek>wsl --version
Wersja podsystemu WSL: 1.2.5.0
Wersja jądra: 5.15.90.1
Wersja usługi WSLg: 1.0.51
Wersja MSRDC: 1.2.3770
Wersja Direct3D: 1.608.2-61064218
Wersja DXCore: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Wersja systemu Windows: 10.0.19045.2846

Sorry, it's in Polish, but I'm sure you'll get it.

So it's like Docker tried to start before WSL is running? But then, restarting docker should help - but it doesn't. Even if I kill all of docker processes. I need to restart my PC

[ctalledo]

Hi @radoslawkoziol, thanks that helps.

So it's like Docker tried to start before WSL is running?

No; Docker Desktop 4.19 checks the version of WSL (via the wsl --version command), but for some reason in your host sometimes it works, sometimes it doesn't.

For example, at time 19:29:14 it works:

[2023-05-05T19:29:14.393744400Z][com.docker.backend.exe][I] WSL version (1.2.5.0) meets ECI requirement (>= 1.1.3)
[2023-05-05T19:29:14.534777000Z][com.docker.backend.exe][I] WSL kernel version (5.15.90) meets ECI requirement (>= 5.15)

But just a few seconds later it fails:

[2023-05-05T19:29:17.323036800Z][com.docker.backend.exe][E] getting WSL version: exit status 1

Not quite sure why.

But looking closer at the diagnostic bundle, I think that error is not the root cause of the problem because Docker Desktop continues to boot.

Later in the log I see a bigger problem:

[2023-05-05T19:29:33.248536800Z][com.docker.backend.exe][E] containerd panic: [2023-05-05T19:29:33.090240936Z][containerd][I] fatal error: unexpected signal during runtime execution
[2023-05-05T19:29:33.092381937Z][containerd][I] [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x556ebbafdc72]

Will continue debugging. Thanks again.

[radoslawkoziol]

@ctalledo any news on that bug? It still happens sometimes.

[ctalledo]

Hi @radoslawkoziol, we are looking at it right now, thanks.

I see two panics in the logs of the diagnostigs bundle you attached:

First containerd on 2023-05-05:

[2023-05-05T19:29:33.248536800Z][com.docker.backend.exe][E] containerd panic: [2023-05-05T19:29:33.090240936Z][containerd][I] fatal error: unexpected signal during runtime execution
[2023-05-05T19:29:33.092381937Z][containerd][I] [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x556ebbafdc72]

Later dockerd on 2023-05-07:

[2023-05-07T10:38:12.581890671Z][dockerd][I] panic: runtime error: invalid memory address or nil pointer dereference
[2023-05-07T10:38:12.581912271Z][dockerd][I]    panic: runtime error: invalid memory address or nil pointer dereference
[2023-05-07T10:38:12.581960471Z][dockerd][I] [signal SIGSEGV: segmentation violation code=0x1 addr=0x30 pc=0x5632d0479476]

We are looking at these to understand what caused them. Thanks again.

[ctalledo]

After analyzing the logs, we believe the panics are caused by an unclean shutdown of the Docker Desktop VM (i.e., specifically the Docker Engine and containerd processes inside that VM). We will be making improvements in Docker Desktop to avoid this scenario going forward.

[ctalledo]

@radoslawkoziol: you indicated the issue happens "randomly"; how often do you see it?

[radoslawkoziol]

After analyzing the logs, we believe the panics are caused by an unclean shutdown of the Docker Desktop VM (i.e., specifically the Docker Engine and containerd processes inside that VM). We will be making improvements in Docker Desktop to avoid this scenario going forward.

@ctalledo I can confirm I shut down my PC without exiting Docker Desktop first.

@radoslawkoziol: you indicated the issue happens "randomly"; how often do you see it?

I'd say once a week at least.
3 days ago it happened twice in a row for the first time. I turned on my PC, I got the error, I restarted my PC then I got the error again, restarted my PC, and then it worked fine.

[ctalledo]

Thanks for the quick response. In general even if you shutdown your PC without stopping DD first, DD should do the right thing and stop the DD VM gracefully (which it's not doing). So that's something we have to improve.

Interesting that you see the issue so frequently, I was looking at other reports of similar panics reported by DD but could not find any 🤔 .

[radoslawkoziol]

I guess most people use laptops and they are just hibernating them, without shutting them down completely.
Or it's just a Windows 10 thing and doesn't happen on Windows 11?

[ctalledo]

I'd say once a week at least.

Has this problem always happened with this frequency? Or did something change lately (e.g., DD version, or enabling WSL, etc)?

[radoslawkoziol]

I'd say once a week at least.

Has this problem always happened with this frequency? Or did something change lately (e.g., DD version, or enabling WSL, etc)?

It's a difficult question, I didn't pay attention to when it happens. I think it's more or less the same. I'll try writing it down from now on.

[ctalledo]

FYI, retitled issue based on analysis.

[radoslawkoziol]

@ctalledo ok.
BTW, it hasn't happened since the last time. I didn't update Docker Desktop, not sure about Windows Update.

[ctalledo]

@ctalledo ok. BTW, it hasn't happened since the last time. I didn't update Docker Desktop, not sure about Windows Update.

Let me close this issue then, and please re-open if it re-occurs. We have an internal ticket to improve the way DD shutsdown to preven this type of problem in the future.

Thanks again @radoslawkoziol!

[radoslawkoziol]

@ctalledo Hello again.

It was okay for a long time, but unfortunately, it happened again. The story is a bit different this time.

1. I turned on my PC, and Docker was loading for about 5 minutes. Nothing was happening, so I thought, "Oh well, again."
2. I went into troubleshoot -> diagnose
3. It took about 10 minutes, but the diagnosis was successful. At this point, I noticed that Docker had finally booted up. It took at least 15 minutes from the PC start until this point. Normally, Docker boots up in about 15 seconds.
4. I attempted to upload the diagnostic data, but it failed to upload. An error message appeared for about 3 seconds, but I missed it.
5. I clicked on "diagnose" again, but it got stuck on "Diagnosing..." for nearly an hour, so I'm giving up.

I just wanted to share it with you, maybe it will be helpful.

Docker version: Docker Desktop 4.20.1 (110738)
WSL:

Wersja podsystemu WSL: 1.2.5.0
Wersja jądra: 5.15.90.1
Wersja usługi WSLg: 1.0.51
Wersja MSRDC: 1.2.3770
Wersja Direct3D: 1.608.2-61064218
Wersja DXCore: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Wersja systemu Windows: 10.0.19045.2965

[Daasin]

Same issue here, I think I had ubuntu pre-installed in wsl before using docker

[ctalledo]

Same issue here, I think I had ubuntu pre-installed in wsl before using docker

Hi @Daasin, if possible could you upload a Docker Desktop diagnostics bundle?

Thanks!