1.8.0: Systemd can't start docker on Centos 7.1 #15498
Milestone
Comments
|
ping @jfrazelle |
|
did you do an |
|
I cant reproduce on a centos 7 machine I want to say it has something to do with kernel config |
|
can you try running this https://github.com/docker/docker/blob/master/contrib/check-config.sh |
|
@jfrazelle You can ignore the bridge modprobe failure message. That will be printed if the bridge module is compiled into the kernel rather than as a module. We should rather check the docker logs. |
|
Those were the logs :/
|
|
@jfrazelle I just ran check config and yum update. Anything else I can try to help collect more information to debug this? (in the morning I could try the same reproduction on a couple of cloud hosts to see how it works out) |
|
I just tried the same reproduction on a DigitalOcean RH7 droplet and docker worked fine.
Everything looks good there. uname -a: Linux rh7test 3.10.0-123.8.1.el7.x86_64 #1 SMP Mon Sep 22 19:06:58 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Still trying to determine why running in centos 7 minimal with updates inside of a virtualbox is failing. |
|
Same problem with centos 7 inside VMware Player Centos 7 minimal install Logs |
|
When I run docker daemon directly from the command line I have no issue: I added the -D debug option to the options passed through systemd, but I can't find where the additional output is going. Anyone with more experience with systemd that can point me in the right direction to provide additional details? |
|
My guess is firewall rules |
|
I can have firewalld running and then start the docker service one time now.. but it will fail if i restart it |
|
I have verified that I can When running inside a virtualbox VM though:
I am still trying to collect more details and will post them here when I get them. In the meantime the only thing I can think of is that the VM I am using has multiple network interfaces configured (one bridged and one host-only), maybe that is causing some of the issue. When running If I stop firewalld, then docker can startup with no problem. This line in the output looks suspicious to me: Jul 12 09:31:45 cent7 docker[2615]: time="2015-07-12T09:31:45.363185154-04:00" level=warning msg="Running modprobe bridge nf_nat br_netfilter failed with message: modprobe: WARNING: Module br_netfilter not found.\n, error: exit status 1" Is there a module or package that I need to have installed in addition to docker in order for it to work with firewalld? |
|
I am also affected by this. Docker 1.7 from main repo used to function until I updated today to 1.8. I run Centos 7 minimal install inside virtualbox. journalctl shows me this: however if i stop the firewall I can start Docker daemon without problems |
|
@abierbaum @squeaky-pl @KalleDK As a quick check, can you enable debugs (-D) and restart the docker-service ?
|
|
|
|
@KalleDK @squeaky-pl thanks. its clear that docker daemon times out after waiting for 90 seconds on some activity that daemon is busy with. Could you get a I dont see this problem either in my centos7-minimal setup running in virtualbox with or without firewalld enabled. So, am guessing some specific configuration that causes the docker daemon to be busy / waiting on something. |
|
@mavenugo What is the best way to get you the strace output you need? I haven't done this before with docker so I want to make sure I get it right the first time. :) |
|
@mavenugo Not sure if that helps but it seems to be stuck on this system call |
|
@squeaky-pl thanks. but could you enable strace in ExecStart, something like |
|
@mavenugo It seems that running through strace affects it somehow and causes it to fail in a different way here is the strace output https://gist.github.com/squeaky-pl/eb5e175f5fe6d50c42a0 |
|
@squeaky-pl One more thing that you can do is send |
|
@squeaky-pl You can send SIGUSR1 to docker daemon by doing the following |
|
|
@mavenugo Here is my strace output: https://gist.github.com/abierbaum/6bc0bd5039559ab16828 journalctl --unit=docker |
|
as @squeaky-pl indicated, enabling strace in the systemd init isnt working. But the output for @squeaky-pl the output that you provided is not complete. Can you please get the full trace output from |
|
|
@squeaky-pl thanks. it looks like the docker daemon is stuck @ |
|
If docker-selinux is required to be installed, please update the installation guide to avoid probelms: https://docs.docker.com/installation/centos/#install https://docs.docker.com/installation/ubuntulinux/#install |
|
@pravinprabhat the 1.9 release bundles our own SELinux policy; installing the extra package will no longer be needed once 1.9 is released (which is probably next week) |
|
Can confirm, can't start |
|
@simoncpu please read the discussion above; for 1.8.x you shouldn't use |
|
@thaJeztah OK, can confirm that installing |
|
Have seen some startup issues using centos 7.1 with Docker 1.9-rc5 on aws. Setting |
|
@jfarrell This is not related at all. |
|
thanks @cpuguy83, that was the issue, adding |
|
Even after installing docker-selinux I still cannot get the docker service to start. I am using ansible to spin up a Centos 7 AWS instance and I get the following: |
|
@thaJeztah Thanks! Adding |
|
I've tried docker-engine 1.9 on CentOS 7.1 AMI but it didn't work.
|
|
I see the same thing as @EslamElHusseiny. It appears the storage is just too slow in AWS and times out. Running it manually fixes that then starting via systemd works. ERRO[0000] WARNING: No --storage-opt dm.thinpooldev specified, using loopback; this configuration is strongly discouraged for production use DEBU[0002] [deviceset] constructDeviceIdMap() |
|
Same issue, on Azure, Centos 7.1. Following this guide: https://docs.docker.com/engine/installation/centos/ |
|
I'm hitting this issue as well (centos 7.1, docker 1.8.2). In my case, it would seem the docker service is failing to start because its dependency docker-storage-setup is failing to start. |
|
@deusnefum I don't think that's related to this issue, also, I think that package is used in the RedHat packaged version of Docker? if you're using the version that's packaged by RedHat/CentOS/Fedora, please report the issue in their issue tracker |
|
Just an FYI, installing docker-selinux prevents a straightforward update: Update proceeds once docker-selinux is removed. |
|
@simoncpu, seems like you are switching from docker packages to docker-engine packages, so you need to uninstall docker* and then install docker-engine which includes docker-engine-selinux as well. |
|
On a fresh CentOS 7.1.1503 image (from Amazon AMI Behavior is interesting in that the error only happens on a fresh install when started with SystemD. After I started it manually with debug logging & The first time, docker needs to set up the devicemapper storage using loopback device. However, it appears to be too slow to complete this, and it hangs with message After starting manually with I can then stop docker in my terminal and re-start it via SystemD with: The logs for when I manually start So, on |
|
I have the same issue as @trinitronx |
|
Fresh install of Docker hangs when I attempt to start with systemctl start docker, however it works when started as docker daemon. I have the docker-selinux package installed. docker -vDocker version 1.8.2-el7, build a01dc02/1.8.2 uname -aLinux ip-172-29-0-100.blah.local 3.10.0-327.4.4.el7.x86_64 #1 SMP Thu Dec 17 15:51:24 EST 2015 x86_64 x86_64 x86_64 GNU/Linux |
|
@trinitronx @danielwhatmuff the default size for devicemapper has been changed back to 10G in docker 1.10 (see #18314). Also note that the CentOS Extras packages are maintained and supported by Red Hat, so any issue with those packages should be reported in their issue tracker, because unfortunately we can't make changes in those packages |
|
@danielwhatmuff Please open a bugzilla on this issue. |
|
This bug is present again in Nothing is found, there is no socket unit, I can fix it by Docker will not start: The package is broken by commit 04104c3. |
|
@rfkrocktk it's not broken, socket activation has been removed for RPM based installs; if your install has an unmodified unit file, docker should work, but if local modifications are found, yum/dnf won't replace that file; see the steps to resolve this here; https://github.com/docker/docker/blob/master/CHANGELOG.md#1120-2016-07-14 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
and others
Using the default configuration of the latest docker 1.8.0 release, systemd can't start docker.
Workaround:
yum install docker-selinuxUse the commands below to provide key information from your environment:
docker version: 1.8.0-1
docker info: N/A
uname -a: Linux cent7 3.10.0-229.7.2.el7.x86_64 #1 SMP Tue Jun 23 22:06:11 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Provide additional environment details (AWS, VirtualBox, physical, etc.):
Reproduction:
Expected to see docker package install correctly by default and allow using systemd to startup docker on the machine.
This is running inside a fresh install of Centos 7.1 inside virtualbox. Docker 1.7.1 works with no issue.
The text was updated successfully, but these errors were encountered: