Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WSL2: docker: Error response from daemon: cgroups: cannot find cgroup mount destination: unknown. #4189

Open
PatrickLang opened this issue Jun 18, 2019 · 22 comments
Labels

Comments

@PatrickLang
Copy link

@PatrickLang PatrickLang commented Jun 18, 2019

Please fill out the below information:

  • Your Windows build number: 18912

  • What you're doing and what's happening:

Intermittently, Docker CE quits working with a cgroup error. If I run wsl --shutdown then restart Ubuntu, it will work again.

docker run -it --rm -v /home/user/aks-engine:/go/src/github.com/Azure/aks-engine -w /go/src/github.com/Azure/aks-engine  quay.io/deis/go-dev:v1.22.1 bash
Unable to find image 'quay.io/deis/go-dev:v1.22.1' locally
v1.22.1: Pulling from deis/go-dev
9ff7e2e5f967: Pull complete                                                                                                                                                                      59856638ac9f: Pull complete                                                                                                                                                                      6f317d6d954b: Pull complete                                                                                                                                                                      a9dde5e2a643: Pull complete                                                                                                                                                                      692b28bf9c4d: Pull complete                                                                                                                                                                      5075eab28b88: Pull complete                                                                                                                                                                      Digest: sha256:e872262e17cbc9b562a0d98d7abd75c6ee35eee782f13ba6e0ad0ec5e0b92753
Status: Downloaded newer image for quay.io/deis/go-dev:v1.22.1
docker: Error response from daemon: cgroups: cannot find cgroup mount destination: unknown.
Makefile:55: recipe for target 'dev' failed
make: *** [dev] Error 125
  • What's wrong / what should be happening instead:

This should work consistently.

  • Docker Version
$ docker version
Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77
 Built:             Sat May  4 02:35:57 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 01:59:36 2019
  OS/Arch:          linux/amd64
  Experimental:     false
@simmessa
Copy link

@simmessa simmessa commented Jul 11, 2019

+1

@simmessa
Copy link

@simmessa simmessa commented Jul 19, 2019

Windows build number: 1903 (OS Build 18932.1000)

Docker version:

Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77
 Built:             Sat May  4 02:35:57 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 01:59:36 2019
  OS/Arch:          linux/amd64
  Experimental:     false

Here's some steps to reproduce the bug on my WSL2 / Ubuntu:

Create an image with this Dockerfile:
https://gist.github.com/simmessa/5304c435b62739f4b86946a4f653c1be

docker build . -t wsl2bugimage:latest

docker run -td --name wsl2bug --privileged -p 222:22 wsl2bugimage:latest

At this point running any new container in Docker will result in this error:

docker run -it hello-world
docker: Error response from daemon: cgroups: cannot find cgroup mount destination: unknown.

And you will have to restart wsl just like @PatrickLang explained.

Hope this helps a bit.

Simone.

@mylesagray
Copy link

@mylesagray mylesagray commented Aug 5, 2019

I encountered this too - the "fix" was running this in WSL:

sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

Not really a viable fix, but it made my command work.

My environment:

Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89
 Built:             Thu Jul 25 21:21:05 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:19:41 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

The command I was running that caused it to error (interestingly, on my system it seemed to be a system load based error, only under high demand did I see this.

 kind create cluster --config cluster.yaml 

Where cluster.yaml was this (essentially creating 6 k8s nodes):

kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
nodes:
- role: control-plane
- role: control-plane
- role: control-plane
- role: worker
- role: worker
- role: worke

@chris-han
Copy link

@chris-han chris-han commented Jan 23, 2020

Windows 2004 (19041.12)
Docker version 19.03.5, build 633a0ea838
Reproduced by running

$ sudo docker run hello-world

@mylesagray 's fix works for me.

@albero94
Copy link

@albero94 albero94 commented Mar 24, 2020

Windows 2004 (19041.153(
Docker Version 19.03.8 build afacb8b7f0

Same as @chris-han and also solved with @mylesagray fix.

@liamness
Copy link

@liamness liamness commented Apr 17, 2020

For anyone else who found their way to this issue, I just thought it would be worth mentioning that Docker Desktop (or at least the beta version) has quite a nice integration with WSL. You install it, then there is an option under "Resources > WSL Integration" where you can select which distros you want to use it with. For me the default was already selected.

Thought it would be worth mentioning, as it seems like less trouble than trying than getting the daemon running properly within your WSL instance. I guess the downside is you end up running multiple VMs instead of one, but I'll take that for a pain-free setup.

@tigerwill90
Copy link

@tigerwill90 tigerwill90 commented May 7, 2020

For anyone else who found their way to this issue, I just thought it would be worth mentioning that Docker Desktop (or at least the beta version) has quite a nice integration with WSL. You install it, then there is an option under "Resources > WSL Integration" where you can select which distros you want to use it with. For me the default was already selected.

Thought it would be worth mentioning, as it seems like less trouble than trying than getting the daemon running properly within your WSL instance. I guess the downside is you end up running multiple VMs instead of one, but I'll take that for a pain-free setup.

Awesome ! Thank's

@alexchiri
Copy link

@alexchiri alexchiri commented May 10, 2020

For anyone else who found their way to this issue, I just thought it would be worth mentioning that Docker Desktop (or at least the beta version) has quite a nice integration with WSL. You install it, then there is an option under "Resources > WSL Integration" where you can select which distros you want to use it with. For me the default was already selected.

Thought it would be worth mentioning, as it seems like less trouble than trying than getting the daemon running properly within your WSL instance. I guess the downside is you end up running multiple VMs instead of one, but I'll take that for a pain-free setup.

But if you want to use this for running kind and you care about being able to access your "nodes" by private IP, then this is not possible using the Docker Desktop Edge for Windows with WSL2 backend.

@wail-shudar
Copy link

@wail-shudar wail-shudar commented Aug 4, 2020

I encountered this too - the "fix" was running this in WSL:

sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

Not really a viable fix, but it made my command work.

My environment:

Client: Docker Engine - Community
 Version:           19.03.1
 API version:       1.40
 Go version:        go1.12.5
 Git commit:        74b1e89
 Built:             Thu Jul 25 21:21:05 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.1
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.5
  Git commit:       74b1e89
  Built:            Thu Jul 25 21:19:41 2019
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

The command I was running that caused it to error (interestingly, on my system it seemed to be a system load based error, only under high demand did I see this.

 kind create cluster --config cluster.yaml 

Where cluster.yaml was this (essentially creating 6 k8s nodes):

kind: Cluster
apiVersion: kind.sigs.k8s.io/v1alpha3
nodes:
- role: control-plane
- role: control-plane
- role: control-plane
- role: worker
- role: worker
- role: worke

The real MVP.

@PavelSosin-320
Copy link

@PavelSosin-320 PavelSosin-320 commented Aug 12, 2020

@Tacfarinas . Right! This fix also solves this problem in the "native docker as service" on CentOS7 distro 👍
The hint was gotten from Osuzu post. He referenced Ubuntu.

@mjawath
Copy link

@mjawath mjawath commented Oct 10, 2020

Please fill out the below information:

  • Your Windows build number: 18912
  • What you're doing and what's happening:

Intermittently, Docker CE quits working with a cgroup error. If I run wsl --shutdown then restart Ubuntu, it will work again.

docker run -it --rm -v /home/user/aks-engine:/go/src/github.com/Azure/aks-engine -w /go/src/github.com/Azure/aks-engine  quay.io/deis/go-dev:v1.22.1 bash
Unable to find image 'quay.io/deis/go-dev:v1.22.1' locally
v1.22.1: Pulling from deis/go-dev
9ff7e2e5f967: Pull complete                                                                                                                                                                      59856638ac9f: Pull complete                                                                                                                                                                      6f317d6d954b: Pull complete                                                                                                                                                                      a9dde5e2a643: Pull complete                                                                                                                                                                      692b28bf9c4d: Pull complete                                                                                                                                                                      5075eab28b88: Pull complete                                                                                                                                                                      Digest: sha256:e872262e17cbc9b562a0d98d7abd75c6ee35eee782f13ba6e0ad0ec5e0b92753
Status: Downloaded newer image for quay.io/deis/go-dev:v1.22.1
docker: Error response from daemon: cgroups: cannot find cgroup mount destination: unknown.
Makefile:55: recipe for target 'dev' failed
make: *** [dev] Error 125
  • What's wrong / what should be happening instead:

This should work consistently.

  • Docker Version
$ docker version
Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77
 Built:             Sat May  4 02:35:57 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 01:59:36 2019
  OS/Arch:          linux/amd64
  Experimental:     false

this worked for me too

@Agrover112
Copy link

@Agrover112 Agrover112 commented Oct 12, 2020

Windows build number: 1903 (OS Build 18932.1000)

Docker version:

Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77
 Built:             Sat May  4 02:35:57 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 01:59:36 2019
  OS/Arch:          linux/amd64
  Experimental:     false

Here's some steps to reproduce the bug on my WSL2 / Ubuntu:

Create an image with this Dockerfile:
https://gist.github.com/simmessa/5304c435b62739f4b86946a4f653c1be

docker build . -t wsl2bugimage:latest

docker run -td --name wsl2bug --privileged -p 222:22 wsl2bugimage:latest

At this point running any new container in Docker will result in this error:

docker run -it hello-world
docker: Error response from daemon: cgroups: cannot find cgroup mount destination: unknown.

And you will have to restart wsl just like @PatrickLang explained.

Hope this helps a bit.

Simone.

This guy is a legend

@yyy0921
Copy link

@yyy0921 yyy0921 commented Oct 30, 2020

my wrong information

# docker run hello-world
docker: Error response from daemon: cgroups: cannot find cgroup mount destination: unknown.
ERRO[0000] error waiting for container: context canceled `

# docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:59 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:01:30 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

The solved solution

#service docker stop
# apt-cache madison docker-ce
#apt-get install docker-ce=17.03.1~ce-0~ubuntu-xenial
#docker version
Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Mon Mar 27 17:14:09 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.1-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Mon Mar 27 17:14:09 2017
 OS/Arch:      linux/amd64
 Experimental: false

#service docker start
# service docker status
# docker run hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

successfully。

The reason i guess that in the "apt-get install docker-ce=17.03.1ce-0ubuntu-xenial" process,
docker-ce-cli is unistall,docker-ce-cli is have relationship with daemon(in my install process,daemon has so many problems.)

@Farrukhw
Copy link

@Farrukhw Farrukhw commented Dec 6, 2020

sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

Thank you :). This solution worked for me, for the same issue i.e.

# docker run hello-world
docker: Error response from daemon: cgroups: cannot find cgroup mount destination: unknown.
ERRO[0000] error waiting for container: context canceled `
docker version
Client: Docker Engine - Community
 Version:           19.03.14
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        5eb3275d40
 Built:             Tue Dec  1 19:20:26 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.14
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       5eb3275d40
  Built:            Tue Dec  1 19:18:53 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.9
  GitCommit:        ea765aba0d05254012b0b9e595e995c09186427f
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

@Tiberivs
Copy link

@Tiberivs Tiberivs commented Jan 12, 2021

The "fix":

sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

Works for me on Gentoo Linux with OpenRC insted of systemd.
Seems the problem is common for all systems without systemd.

@Agrover112
Copy link

@Agrover112 Agrover112 commented Jan 12, 2021

Yes this is the fix

@JimmyTzuc
Copy link

@JimmyTzuc JimmyTzuc commented Feb 17, 2021

Hi! I have a fix in my ubuntu
sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

@frankgo81
Copy link

@frankgo81 frankgo81 commented Feb 26, 2021

Hi! I have a fix in my ubuntu
sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

works for me too....big thanks

@therealkenc
Copy link
Collaborator

@therealkenc therealkenc commented May 10, 2021

Create an image with this Dockerfile:

"This Dockerfile" (for some values of 'this' as with the OP) is key here. Some containers expect cgroups to be set up just-so, other containers will not. There are 32 likes on submission as of this writing, so call it feature request "WSL /init could set up cgroups like typical Linux installs expect". More practically, WSL needs a better systemd story LZ #994. Thanks everyone for the work-around suggestions.

@sangnguyen-sunbytes
Copy link

@sangnguyen-sunbytes sangnguyen-sunbytes commented Jun 1, 2021

The "fix":

sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

Works for me on Gentoo Linux with OpenRC insted of systemd.
Seems the problem is common for all systems without systemd.

Thanks you so much, that work for me

@beshoo
Copy link

@beshoo beshoo commented Sep 19, 2021

The "fix":

sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

Works for me on Gentoo Linux with OpenRC insted of systemd.
Seems the problem is common for all systems without systemd.

docker run hello-world
docker: Error response from daemon: cgroups: cannot find cgroup mount destination: unknown.
ERRO[0002] error waiting for container: context canceled


mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd
mount: cgroup is already mounted or /sys/fs/cgroup/systemd busy
       cgroup is already mounted on /sys/fs/cgroup/systemd
       cgroup is already mounted on /sys/fs/cgroup/freezer
       cgroup is already mounted on /sys/fs/cgroup/devices
       cgroup is already mounted on /sys/fs/cgroup/cpu,cpuacct
       cgroup is already mounted on /sys/fs/cgroup/cpuset
       cgroup is already mounted on /sys/fs/cgroup/memory

Same problem

@hershkoy
Copy link

@hershkoy hershkoy commented Nov 7, 2021

I also had this issue in the past. Usually the fix mentioned works fine:

sudo mkdir /sys/fs/cgroup/systemd
sudo mount -t cgroup -o none,name=systemd cgroup /sys/fs/cgroup/systemd

However today, for some reason it didn't.
The fix for me today was to also restart docker service:

sudo service docker stop
sudo service docker start

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet