docker - runtime/cgo: pthread_create failed: Resource temporarily unavailable

[jcmartins]

go version
go version go1.4.3 linux/amd64

loader --postings /dgraph/p --rdfgzips /data/rdf-films.gz, /data/names.gz --stw_ram_mb 7000
loader --postings /dgraph/p --rdfgzips /data/rdf-films.gz, /data/names.gz --stw_ram_mb 7000
INFO[0000] Empty size: [16] EmptyPosting size: [24] package=posting
INFO[0000] Set max procs to num cpus num_cpu=4 package=loader_main prev_maxprocs=1
INFO[0000] Handling... package=loader_main path=/data/rdf-films.gz

3 merged=2732887 package=posting pending=13836
INFO[0451] List Merge counters added=2746995 clean=3865 dirtysize=0 mapsize=1625 merged=2741105 package=posting pending=5890
INFO[0452] Merged lists. Calling GC. package=posting
INFO[0456] Trying to free OS memory package=posting
INFO[0459] Memory Usage after calling GC. allocated_MB=1486 package=posting
INFO[0460] Counters ignored=0 len_cnq=9971 package=loader parsed=1065104 pending=12971 processed=1052133 read=1075108
runtime/cgo: pthread_create failed: Resource temporarily unavailable
SIGABRT: abort
PC=0x7efdfe2a8107

goroutine 0 [idle]:

goroutine 1 [chan receive, 7 minutes]:
github.com/dgraph-io/dgraph/loader.HandleRdfReader(0x7efdfef4b0f8, 0xc20805a000, 0x1, 0x7efdfef4b0f8, 0x0, 0x0)
/go/src/github.com/dgraph-io/dgraph/loader/loader.go:179 +0x2d3
main.main()
/go/src/github.com/dgraph-io/dgraph/server/loader/main.go:88 +0xd2c

goroutine 17 [syscall, 7 minutes, locked to thread]:
runtime.goexit()
/usr/src/go/src/runtime/asm_amd64.s:2232 +0x1
goroutine 1812945 [chan receive]:
github.com/dgraph-io/dgraph/posting.(*counters).periodicLog(0xc2a68c0bc0)
/go/src/github.com/dgraph-io/dgraph/posting/lists.go:45 +0x68
created by github.com/dgraph-io/dgraph/posting.MergeLists
/go/src/github.com/dgraph-io/dgraph/posting/lists.go:245 +0x1ef

rax 0x0
rbx 0x7f7607fa8708
rcx 0x7f7607c39107
rdx 0x6
rdi 0x1f6
rsi 0x1f7
rbp 0x642d12
rsp 0x7f7606986798
r8 0xa
r9 0x7f7606987700
r10 0x8
r11 0x206
r12 0x7f75fc033990
r13 0x7f7608919060
r14 0x0
r15 0x7f7606987700
rip 0x7f7607c39107
rflags 0x206
cs 0x33
fs 0x0
gs 0x0

[jcmartins]

In my local machine (not docker) run go version 1.5.3 it`s run ok!

[manishrjain]

This is a cgo bug, which was resolved in 1.4.3, but then came back in 1.5.1. When you saw it in 1.4.3, did you run the loader in a docker instance, or directly on local machine?

If 1.5.3 runs fine, we might switch to it.

[jcmartins]

The bug only occurs in docker with go version 1.4.1

In my local notebook with archlinux and running go v1.5.3 this bug does not occurred.

[manishrjain]

This is a known issue with Go. I've created #22 to switch to Go 1.5.3 in v0.2, which we're expecting to launch by end February.

[jcmartins]

Why not wait to go 1.6 ?

[manishrjain]

Go 1.6 hasn't yet released. Following from the bugs that I encountered in Go 1.5.1, I intend to keep DGraph on a version which has been released for at least 3 months, so it has chance to get any bugs fixed. Go 1.5.3 has that going for it.