Skip to content

Server crashes on docker-compose up/docker-compose restart since upgrade to 17.06.0-ce when single-run container present #70

@nrlakin

Description

@nrlakin
  • This is a bug report
  • This is a feature request
  • I searched existing issues before opening this one

Expected behavior

docker-compose up -d cleanly restarts containers.

Actual behavior

Fork bomb? I get the following errors every time I run docker-compose up -d or docker-compose restart:

Jul 28 16:23:23 dockerd[1156]: time="2017-07-28T16:23:22.290755035Z" level=error msg="libcontainerd: failed to receive event from containerd: rpc error: cod
Jul 28 16:23:35 dockerd[1156]: time="2017-07-28T16:23:25.815196029Z" level=info msg="libcontainerd: new containerd process, pid: 2501"
Jul 28 16:24:44 dockerd[1156]: time="2017-07-28T16:24:36.171616370Z" level=info msg="libcontainerd: new containerd process, pid: 2505"
Jul 28 16:32:29 dockerd[1156]: time="2017-07-28T16:32:16.122295964Z" level=info msg="libcontainerd: new containerd process, pid: 2533"
Jul 28 16:36:25 dockerd[1156]: time="2017-07-28T16:34:55.954868904Z" level=info msg="libcontainerd: new containerd process, pid: 2539"
Jul 28 16:39:40 dockerd[1156]: time="2017-07-28T16:39:27.257176662Z" level=info msg="libcontainerd: new containerd process, pid: 2548"
Jul 28 16:42:43 dockerd[1156]: time="2017-07-28T16:42:29.999000255Z" level=info msg="libcontainerd: new containerd process, pid: 2554"
Jul 28 16:46:19 dockerd[1156]: time="2017-07-28T16:46:08.053190101Z" level=info msg="libcontainerd: new containerd process, pid: 2560"
Jul 28 16:47:55 dockerd[1156]: time="2017-07-28T16:47:46.457091066Z" level=info msg="libcontainerd: new containerd process, pid: 2565"
Jul 28 16:50:28 dockerd[1156]: time="2017-07-28T16:50:18.663104714Z" level=info msg="libcontainerd: new containerd process, pid: 2581"
Jul 28 16:52:25 dockerd[1156]: time="2017-07-28T16:52:06.544602390Z" level=info msg="libcontainerd: new containerd process, pid: 2586"
Jul 28 16:54:45 dockerd[1156]: time="2017-07-28T16:54:32.002602317Z" level=info msg="libcontainerd: new containerd process, pid: 2591"
Jul 28 16:57:59 dockerd[1156]: time="2017-07-28T16:57:40.092116997Z" level=info msg="libcontainerd: new containerd process, pid: 2597"

[*** Repeat many times ***]
Out of memory error, system crash.

Steps to reproduce the behavior

I have had this issue every time I've used docker-compose up/restart to restart my containers since upgrading to 17.06.0-ce. I'm running Docker on Ubuntu 16.04. The "fork bomb" doesn't always occur, but the first two lines are always present:

Jul 28 16:23:23 dockerd[1156]: time="2017-07-28T16:23:22.290755035Z" level=error msg="libcontainerd: failed to receive event from containerd: rpc error: cod
Jul 28 16:23:35 dockerd[1156]: time="2017-07-28T16:23:25.815196029Z" level=info msg="libcontainerd: new containerd process, pid: 2501"

And the end result is always a crash. Seems to work fine after reboot, with all containers running.

EDIT: This behavior only occurs when a container is present that runs a single task and exits. There is a container that builds some static files and then exits. Since updating to 17.06.0-ce, this container seems to stay up--I see it in the output of docker ps well after the task is done. Removing the static build container from my compose file resolves the issue.

Output of docker version:

Client:
 Version:      17.06.0-ce
 API version:  1.30
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:23:31 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.06.0-ce
 API version:  1.30 (minimum version 1.12)
 Go version:   go1.8.3
 Git commit:   02c1d87
 Built:        Fri Jun 23 21:19:04 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:

Containers: 6
 Running: 6
 Paused: 0
 Stopped: 0
Images: 17
Server Version: 17.06.0-ce
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 71
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-1020-aws
Operating System: Ubuntu 16.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 990.9MiB
Name: voltapp-development-1
ID: RGYB:FXWD:Z54C:SVHD:CXHD:P4IO:YU4A:Q22J:JXDX:F3J6:ZWIH:ZJFJ
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
 provider=amazonec2
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.)
Server is an AWS t2.micro instance, created with docker-machine and the AWS driver:

docker-machine version

docker-machine version 0.12.2, build 9371605

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions