Skip to content

runtime/cgo: pthread_create failed: Resource temporarily unavailable #21298

@onorua

Description

@onorua

Output of docker version:

Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 15:59:07 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 15:59:07 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 28
 Running: 11
 Paused: 0
 Stopped: 17
Images: 18
Server Version: 1.10.3
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Plugins: 
 Volume: local
 Network: bridge null host
Kernel Version: 4.4.0-11-generic
Operating System: Ubuntu 16.04
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.66 GiB
Name: ip-172-16-236-27
ID: 34U2:FZKU:MWEF:4NNW:XDLK:YT5T:CZK2:GK4X:J4LS:IXLH:U6HT:47Y4
WARNING: No swap limit support

I'm running it on XEN VM, inside the kubernetes cluster. I have like 20 machines like this, but suddenly docker just crash with following message:

Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: runtime/cgo: pthread_create failed: Resource temporarily unavailable
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: SIGABRT: abort
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: PC=0x7fd0a51d9227 m=239
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: goroutine 0 [idle]:
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: goroutine 1 [chan receive]:
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: main.(*DaemonCli).CmdDaemon(0xc820222420, 0xc82000a290, 0xb, 0xb, 0x0, 0x0)
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]:         /usr/src/docker/docker/daemon.go:305 +0x20a7
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: reflect.callMethod(0xc82068ae40, 0xc820c6bc78)
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]:         /usr/local/go/src/reflect/value.go:628 +0x1fc
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: reflect.methodValueCall(0xc82000a290, 0xb, 0xb, 0x1, 0xc82068ae40, 0x0, 0x0, 0xc82068ae40, 0x0, 0x478734, ...)
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]:         /usr/local/go/src/reflect/asm_amd64.s:29 +0x36
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: github.com/docker/docker/cli.(*Cli).Run(0xc82068adb0, 0xc82000a280, 0xc, 0xc, 0x0, 0x0)
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]:         /usr/src/docker/.gopath/src/github.com/docker/docker/cli/cli.go:89 +0x383
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: main.main()
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]:         /usr/src/docker/docker/docker.go:63 +0x43c
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: goroutine 17 [syscall, locked to thread]:
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]: runtime.goexit()
Mar 17 13:19:23 ip-172-16-236-27 docker[2476]:         /usr/local/go/src/runtime/asm_amd64.s:1721 +0x1

Originally issues started with this message:

Mar 17 12:45:07 ip-172-16-236-27 docker[25022]: time="2016-03-17T12:45:07.147053465Z" level=fatal msg="Error starting daemon: Error initializing network controller: could not delete 
the default bridge network: network bridge has active endpoints"

I've restarted server and got message above. I have logs for both before and after restart. Attaching logs after restart, but let me know if you need before the restart.
docker.log.txt

P.S.
The docker daemon started to work in couple of hours of the crashes.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions