Skip to content
This repository has been archived by the owner. It is now read-only.

Docker errors in CoreOS 1029.0.0 #1246

Closed
brandonweeks opened this issue Apr 27, 2016 · 6 comments
Closed

Docker errors in CoreOS 1029.0.0 #1246

brandonweeks opened this issue Apr 27, 2016 · 6 comments

Comments

@brandonweeks
Copy link

@brandonweeks brandonweeks commented Apr 27, 2016

Using coreos-kubernetes to provision a cluster fails with the alpha channel/1029.0.0. Kubelet never starts, it seems that the Docker service has a bug introduced in 1029.0.0 as 1010.1.0 works with the same configuration.

Cluster created with kube-aws from coreos-kubernetes latest (f3f0af1) with releaseChannel: alpha and controllerInstanceType: m3.medium.

MOTD: https://gist.github.com/brandonweeks/02d1e1cdc3fbffd25739c81eb5c571c2
Output from journal -u docker --no-pager: https://gist.github.com/brandonweeks/57e65a91eb9635e5c7312e661b6523f2

@brandonweeks brandonweeks changed the title Docker errors on Docker errors in CoreOS 1029.0.0 Apr 27, 2016
@philips
Copy link

@philips philips commented Apr 27, 2016

@brandonweeks yes, we had to pause the alpha rollout of 1029.0.0 because of this issue.

We are still trying to root cause it but it seems related to the systemd cgroups driver. If you remove the Environment="DOCKER_CGROUPS=--exec-opt native.cgroupdriver=systemd" from the docker.service file it will work (but some other things will break like kubelet, if you are using it).

We will keep this issue updated with details.

@mischief
Copy link

@mischief mischief commented Apr 27, 2016

backporting opencontainers/runc#446 to docker 1.10.3 appears to fix the cpu shares issue, but i need to run kubernetes to be sure everything is ok.

@mischief
Copy link

@mischief mischief commented Apr 27, 2016

fix for this should be in CoreOS alpha 1031.0.0, tomorrow.

@brandonweeks
Copy link
Author

@brandonweeks brandonweeks commented Apr 27, 2016

Thank you @mischief!

@mischief
Copy link

@mischief mischief commented Apr 28, 2016

1032.0.0 is out, so kube-aws should be fine now on alpha.

@mischief
Copy link

@mischief mischief commented Apr 29, 2016

for posterity, a simple reproducer for this bug is:

docker run --rm -ti --cpu-shares=100 busybox
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

4 participants