Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime error with tsuru unit-remove #1204

Closed
vmalaga opened this issue May 22, 2015 · 8 comments
Closed

runtime error with tsuru unit-remove #1204

vmalaga opened this issue May 22, 2015 · 8 comments

Comments

@vmalaga
Copy link

vmalaga commented May 22, 2015

Hey all

after add some units to app, if I try to remove any unit, the tsr daemon crack with runtime error.
I send the trace at the tsr

tsuru unit-remove 1 -a itapps

2015-05-23T01:38:28.708126276+02:00 DELETE /apps/itapps/units 0 in 5.282663ms
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x20 pc=0x511a54]

goroutine 184 [running]:
github.com/tsuru/tsuru/provision/docker.(*segregatedScheduler).GetRemovableContainer(0x0, 0xc208139562, 0x6, 0xc208055920, 0x0, 0x0, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/provision/docker/scheduler.go:140 +0x94
github.com/tsuru/tsuru/provision/docker.(*dockerProvisioner).RemoveUnits(0xc20804cf00, 0x7f759b449210, 0xc20819db80, 0x1, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/provision/docker/provisioner.go:629 +0x630
github.com/tsuru/tsuru/app.func·020()
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/app/app.go:427 +0xcd
created by github.com/tsuru/tsuru/app.(*App).RemoveUnits
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/app/app.go:444 +0x3df

goroutine 1 [IO wait]:
net.(*pollDesc).Wait(0xc2081d1640, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081d1640, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2081d15e0, 0x0, 0x7f759b441e50, 0xc2081c6db0)
        /usr/lib/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc20804e390, 0xc2081fb238, 0x0, 0x0)
        /usr/lib/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc20804e390, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc2081001e0, 0x7f759b448f50, 0xc20804e390, 0x0, 0x0)
        /usr/lib/go/src/net/http/server.go:1728 +0x92
gopkg.in/tylerb/graceful%2ev1.(*Server).Serve(0xc2081d3f80, 0x7f759b448f50, 0xc20804e390, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/tylerb/graceful.v1/graceful.go:207 +0x535
gopkg.in/tylerb/graceful%2ev1.(*Server).ListenAndServe(0xc2081d3f80, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/tylerb/graceful.v1/graceful.go:110 +0x211
github.com/tsuru/tsuru/api.RunServer(0xc208073a00, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/api/server.go:357 +0x5d86
main.(*apiCmd).Run(0xc20807bb20, 0xc20800ca00, 0xc20800ca50, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/cmd/tsr/api.go:35 +0x1d1
main.(*tsrCommand).Run(0xc2080d48a0, 0xc20800ca00, 0xc20800ca50, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/cmd/tsr/command.go:60 +0x448
github.com/tsuru/tsuru/cmd.(*Manager).Run(0xc2080b2180, 0x0, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/cmd/cmd.go:192 +0x1872
main.main()
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/cmd/tsr/main.go:60 +0x9c

goroutine 6 [chan receive]:
github.com/tsuru/tsuru/db/storage.retire(0xc20804c100)
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/db/storage/storage.go:108 +0x7f
created by github.com/tsuru/tsuru/db/storage.init·1
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/db/storage/storage.go:103 +0x61

goroutine 9 [syscall]:
os/signal.loop()
        /usr/lib/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
        /usr/lib/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 11 [chan receive]:
main.func·001()
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/cmd/tsr/main.go:50 +0x5c
created by main.listenSignals
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/cmd/tsr/main.go:53 +0x18e

goroutine 18 [select]:
github.com/tsuru/tsuru/provision/docker.(*containerHealer).runContainerHealer(0xc20818ffe0)
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/provision/docker/healer_container.go:27 +0xfc
created by github.com/tsuru/tsuru/provision/docker.(*dockerProvisioner).initDockerCluster
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/tsuru/provision/docker/provisioner.go:125 +0x791

goroutine 12 [select]:
gopkg.in/mgo%2ev2.(*mongoCluster).syncServersLoop(0xc208144780)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/cluster.go:379 +0x606
created by gopkg.in/mgo%2ev2.newCluster
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/cluster.go:73 +0x1d0

goroutine 15 [IO wait]:
net.(*pollDesc).Wait(0xc2081a3330, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081a3330, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081a32d0, 0xc208181e60, 0x24, 0x24, 0x0, 0x7f759b441e50, 0xc2081c6b78)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e6d0, 0xc208181e60, 0x24, 0x24, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
gopkg.in/mgo%2ev2.fill(0x7f759b448648, 0xc20804e6d0, 0xc208181e60, 0x24, 0x24, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:501 +0x75
gopkg.in/mgo%2ev2.(*mongoSocket).readLoop(0xc20818cb60)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:518 +0x126
created by gopkg.in/mgo%2ev2.newSocket
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:165 +0x3d3

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
        /usr/lib/go/src/runtime/asm_amd64.s:2232 +0x1

goroutine 16 [sleep]:
gopkg.in/mgo%2ev2.(*mongoServer).pinger(0xc20818ca80, 0x1)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/server.go:295 +0x169
created by gopkg.in/mgo%2ev2.newServer
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/server.go:88 +0x196

goroutine 19 [select]:
github.com/tsuru/docker-cluster/cluster.(*Cluster).runActiveMonitoring(0xc208055920, 0xb2d05e00)
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/docker-cluster/cluster/cluster.go:280 +0x4c6
created by github.com/tsuru/docker-cluster/cluster.(*Cluster).StartActiveMonitoring
        /build/buildd/tsuru-server-0.11.0/src/github.com/tsuru/docker-cluster/cluster/cluster.go:215 +0x79

goroutine 20 [select]:
gopkg.in/mgo%2ev2.(*mongoCluster).syncServersLoop(0xc208144960)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/cluster.go:379 +0x606
created by gopkg.in/mgo%2ev2.newCluster
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/cluster.go:73 +0x1d0

goroutine 50 [IO wait]:
net.(*pollDesc).Wait(0xc2081d0d80, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081d0d80, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081d0d20, 0xc2081cbad0, 0x24, 0x24, 0x0, 0x7f759b441e50, 0xc2081c6c90)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e1b0, 0xc2081cbad0, 0x24, 0x24, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
gopkg.in/mgo%2ev2.fill(0x7f759b448648, 0xc20804e1b0, 0xc2081cbad0, 0x24, 0x24, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:501 +0x75
gopkg.in/mgo%2ev2.(*mongoSocket).readLoop(0xc20818c380)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:518 +0x126
created by gopkg.in/mgo%2ev2.newSocket
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:165 +0x3d3

goroutine 24 [IO wait]:
net.(*pollDesc).Wait(0xc2081a3aa0, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081a3aa0, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081a3a40, 0xc2081cabd0, 0x24, 0x24, 0x0, 0x7f759b441e50, 0xc2081f27e8)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e768, 0xc2081cabd0, 0x24, 0x24, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
gopkg.in/mgo%2ev2.fill(0x7f759b448648, 0xc20804e768, 0xc2081cabd0, 0x24, 0x24, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:501 +0x75
gopkg.in/mgo%2ev2.(*mongoSocket).readLoop(0xc20818d260)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:518 +0x126
created by gopkg.in/mgo%2ev2.newSocket
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:165 +0x3d3

goroutine 23 [IO wait]:
net.(*pollDesc).Wait(0xc2081a3790, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081a3790, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081a3730, 0xc2081ca840, 0x24, 0x24, 0x0, 0x7f759b441e50, 0xc2081c6c48)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e750, 0xc2081ca840, 0x24, 0x24, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
gopkg.in/mgo%2ev2.fill(0x7f759b448648, 0xc20804e750, 0xc2081ca840, 0x24, 0x24, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:501 +0x75
gopkg.in/mgo%2ev2.(*mongoSocket).readLoop(0xc20818cfc0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:518 +0x126
created by gopkg.in/mgo%2ev2.newSocket
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:165 +0x3d3

goroutine 79 [IO wait]:
net.(*pollDesc).Wait(0xc20815ca00, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20815ca00, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20815c9a0, 0xc208297440, 0x24, 0x24, 0x0, 0x7f759b441e50, 0xc2081f2938)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e228, 0xc208297440, 0x24, 0x24, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
gopkg.in/mgo%2ev2.fill(0x7f759b448648, 0xc20804e228, 0xc208297440, 0x24, 0x24, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:501 +0x75
gopkg.in/mgo%2ev2.(*mongoSocket).readLoop(0xc20818c540)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:518 +0x126
created by gopkg.in/mgo%2ev2.newSocket
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:165 +0x3d3

goroutine 72 [IO wait]:
net.(*pollDesc).Wait(0xc208165090, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208165090, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208165030, 0xc20822e000, 0x1000, 0x1000, 0x0, 0x7f759b441e50, 0xc2082a7448)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e9d8, 0xc20822e000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc208191e48, 0xc20822e000, 0x1000, 0x1000, 0x2, 0x0, 0x0)
        /usr/lib/go/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc208249cc0, 0xc20822e000, 0x1000, 0x1000, 0x2, 0x0, 0x0)
        /usr/lib/go/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc208054de0)
        /usr/lib/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208054de0, 0xc20824fb0a, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc208054de0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc208239530, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc208239530, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc208054de0, 0xc208299380, 0x0, 0x0)
        /usr/lib/go/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc208191e00, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc208191e00)
        /usr/lib/go/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
        /usr/lib/go/src/net/http/server.go:1751 +0x35e

goroutine 65 [IO wait]:
net.(*pollDesc).Wait(0xc2081d1b10, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081d1b10, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081d1ab0, 0xc208205000, 0x1000, 0x1000, 0x0, 0x7f759b441e50, 0xc2082163c8)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e440, 0xc208205000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f759b4489e8, 0xc20804e440, 0xc20807d658, 0xc208205000, 0x1000, 0x1000, 0x9c6760, 0x0, 0x0)
        /usr/lib/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081094e0, 0xc208205000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
        <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208101c80)
        /usr/lib/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208101c80, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20807d600)
        /usr/lib/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:660 +0xc9f

goroutine 40 [IO wait]:
net.(*pollDesc).Wait(0xc2081d0140, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081d0140, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081d00e0, 0xc208199000, 0x1000, 0x1000, 0x0, 0x7f759b441e50, 0xc2081f3728)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e088, 0xc208199000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f759b4489e8, 0xc20804e088, 0xc20807d238, 0xc208199000, 0x1000, 0x1000, 0x9c6760, 0x0, 0x0)
        /usr/lib/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081c46a0, 0xc208199000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
        <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081d2b40)
        /usr/lib/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081d2b40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20807d1e0)
        /usr/lib/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:660 +0xc9f

goroutine 44 [IO wait]:
net.(*pollDesc).Wait(0xc2081d0370, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081d0370, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081d0310, 0xc2081ca630, 0x24, 0x24, 0x0, 0x7f759b441e50, 0xc20807b5f0)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e0a0, 0xc2081ca630, 0x24, 0x24, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
gopkg.in/mgo%2ev2.fill(0x7f759b448648, 0xc20804e0a0, 0xc2081ca630, 0x24, 0x24, 0x0, 0x0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:501 +0x75
gopkg.in/mgo%2ev2.(*mongoSocket).readLoop(0xc20818c1c0)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:518 +0x126
created by gopkg.in/mgo%2ev2.newSocket
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/socket.go:165 +0x3d3

goroutine 42 [IO wait]:
net.(*pollDesc).Wait(0xc2081d0060, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081d0060, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081d0000, 0xc208161000, 0x1000, 0x1000, 0x0, 0x7f759b441e50, 0xc2081f3408)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e098, 0xc208161000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f759b4489e8, 0xc20804e098, 0xc20807d188, 0xc208161000, 0x1000, 0x1000, 0x9c6760, 0x0, 0x0)
        /usr/lib/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081c4700, 0xc208161000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
        <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081d2ba0)
        /usr/lib/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081d2ba0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20807d130)
        /usr/lib/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:660 +0xc9f

goroutine 38 [IO wait]:
net.(*pollDesc).Wait(0xc2081d0450, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081d0450, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081d03f0, 0xc208127000, 0x1000, 0x1000, 0x0, 0x7f759b441e50, 0xc2081f3178)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e070, 0xc208127000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f759b4489e8, 0xc20804e070, 0xc20807d0d8, 0xc208127000, 0x1000, 0x1000, 0x9c6760, 0x0, 0x0)
        /usr/lib/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081c45e0, 0xc208127000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
        <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081d2a20)
        /usr/lib/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081d2a20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20807d080)
        /usr/lib/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:660 +0xc9f

goroutine 43 [select]:
net/http.(*persistConn).writeLoop(0xc20807d130)
        /usr/lib/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:661 +0xcbc

goroutine 41 [select]:
net/http.(*persistConn).writeLoop(0xc20807d1e0)
        /usr/lib/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:661 +0xcbc

goroutine 34 [select]:
gopkg.in/mgo%2ev2.(*mongoCluster).syncServersLoop(0xc208144c30)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/cluster.go:379 +0x606
created by gopkg.in/mgo%2ev2.newCluster
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/cluster.go:73 +0x1d0

goroutine 61 [select]:
gopkg.in/tylerb/graceful%2ev1.(*Server).manageConnections(0xc2081d3f80, 0xc208100360, 0xc208100480, 0xc208100660, 0xc208100720)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/tylerb/graceful.v1/graceful.go:243 +0x3da
created by gopkg.in/tylerb/graceful%2ev1.(*Server).Serve
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/tylerb/graceful.v1/graceful.go:193 +0x2f9

goroutine 39 [select]:
net/http.(*persistConn).writeLoop(0xc20807d080)
        /usr/lib/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:661 +0xcbc

goroutine 37 [sleep]:
gopkg.in/mgo%2ev2.(*mongoServer).pinger(0xc20818d180, 0x1)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/server.go:295 +0x169
created by gopkg.in/mgo%2ev2.newServer
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/server.go:88 +0x196

goroutine 47 [sleep]:
gopkg.in/mgo%2ev2.(*mongoServer).pinger(0xc20818c0e0, 0x1)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/server.go:295 +0x169
created by gopkg.in/mgo%2ev2.newServer
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/mgo.v2/server.go:88 +0x196

goroutine 48 [IO wait]:
net.(*pollDesc).Wait(0xc2081d0a70, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081d0a70, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081d0a10, 0xc208141000, 0x1000, 0x1000, 0x0, 0x7f759b441e50, 0xc208217c20)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e148, 0xc208141000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f759b4489e8, 0xc20804e148, 0xc20807cf78, 0xc208141000, 0x1000, 0x1000, 0x9c6760, 0x0, 0x0)
        /usr/lib/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081c52e0, 0xc208141000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
        <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081d3140)
        /usr/lib/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081d3140, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20807cf20)
        /usr/lib/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:660 +0xc9f

goroutine 49 [select]:
net/http.(*persistConn).writeLoop(0xc20807cf20)
        /usr/lib/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:661 +0xcbc

goroutine 62 [chan receive]:
gopkg.in/tylerb/graceful%2ev1.(*Server).handleInterrupt(0xc2081d3f80, 0x7f759b448f50, 0xc20804e390)
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/tylerb/graceful.v1/graceful.go:268 +0x41
created by gopkg.in/tylerb/graceful%2ev1.(*Server).Serve
        /build/buildd/tsuru-server-0.11.0/src/gopkg.in/tylerb/graceful.v1/graceful.go:203 +0x505

goroutine 66 [select]:
net/http.(*persistConn).writeLoop(0xc20807d600)
        /usr/lib/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:661 +0xcbc

goroutine 69 [IO wait]:
net.(*pollDesc).Wait(0xc2081d1fe0, 0x72, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2081d1fe0, 0x0, 0x0)
        /usr/lib/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2081d1f80, 0xc208218000, 0x1000, 0x1000, 0x0, 0x7f759b441e50, 0xc208217408)
        /usr/lib/go/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc20804e4b0, 0xc208218000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
        /usr/lib/go/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f759b4489e8, 0xc20804e4b0, 0xc20807d708, 0xc208218000, 0x1000, 0x1000, 0x9c6760, 0x0, 0x0)
        /usr/lib/go/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2080f9800, 0xc208218000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
        <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080f4b40)
        /usr/lib/go/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080f4b40, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
        /usr/lib/go/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20807d6b0)
        /usr/lib/go/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:660 +0xc9f

goroutine 70 [select]:
net/http.(*persistConn).writeLoop(0xc20807d6b0)
        /usr/lib/go/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
        /usr/lib/go/src/net/http/transport.go:661 +0xcbc
Opening config file: /etc/tsuru/tsuru.conf
@andrewsmedina
Copy link
Contributor

@vmalaga can you see the tsuru log output. By default tsuru send log to syslog (/var/log/syslog)

@cezarsa
Copy link
Member

cezarsa commented May 23, 2015

@vmalaga I think you found a bug in the latest release. The default scheduler isn't working anymore, this default scheduler isn't recommended for production environments but it should either be removed or remain working.
Good news is it should be an easy fix. Please, try setting in you tsuru.conf:

docker:
    segregate: true

This will enable the segregate scheduler and should get you past this error.

@vmalaga
Copy link
Author

vmalaga commented May 24, 2015

@cezarsa you're right, with the segregate scheduler there is not any problem.
ok I use segregate from now.

thanks

@vmalaga vmalaga closed this as completed May 24, 2015
@vmalaga
Copy link
Author

vmalaga commented May 29, 2015

@cezarsa I am testing the units scale and see that the segregate scheduler fails removing units from system.

Now don't generate a runtime error, but the unit is not removed from pool

root@tsrmgr:~# tsuru app-info -a qhelpdesk
Application: qhelpdesk
Repository: git@tsrmgr.qdqmedia.com:qhelpdesk.git
Platform: python
Teams: cmsteam
Address: qhelpdesk.tsuru.qdqmedia.com
Owner: xxxx@qdqmedia.com
Team owner: cmsteam
Deploys: 43
Pool: prod
Units: 2
+------------+---------+--------------------+-------+--------------+
| Unit       | State   | Host               | Port  | IP           |
+------------+---------+--------------------+-------+--------------+
| 370483ca65 | started | tsr01.qdqmedia.com | 32829 | 172.17.0.102 |
| a26eaf42fc | started | tsr03.qdqmedia.com | 32833 | 172.17.0.115 |
+------------+---------+--------------------+-------+--------------+

Service instances: 1
+----------+--------------+
| Service  | Instance     |
+----------+--------------+
| postgres | qhelpdesk_db |
+----------+--------------+

App Plan:
+---------------+--------+------+-----------+--------+---------+
| Name          | Memory | Swap | Cpu Share | Router | Default |
+---------------+--------+------+-----------+--------+---------+
| autogenerated | 0 MB   | 0 MB | 100       |        | false   |
+---------------+--------+------+-----------+--------+---------+

root@tsrmgr:~# tsuru unit-remove 1 -a qhelpdesk
Units successfully removed!

and at syslog i see that, but the node choosen to remove the unit is invalid

May 29 11:21:36 tsrmgr tsr[11584]: [scheduler] Possible nodes for remove a container: []string{"tsr02.qdqmedia.com", "tsr03.qdqmedia.com", "tsr01.qdqmedia.com"}
May 29 11:21:36 tsrmgr tsr[11584]: [scheduler] Chosen node for remove a container: "http://tsr02.qdqmedia.com:2375" Count: 6

@vmalaga vmalaga reopened this May 29, 2015
@cezarsa
Copy link
Member

cezarsa commented May 29, 2015

@vmalaga This was caused by another bug in the unit-remove scheduling algorithm, we'll release 0.11.1 probably later today and it will fix this issue.

@vmalaga
Copy link
Author

vmalaga commented May 29, 2015

If I can help you testing any thing say me

@andrewsmedina
Copy link
Contributor

this issue is fixed since 0.11.1. @vmalaga can we close this issue?

@vmalaga
Copy link
Author

vmalaga commented Feb 13, 2016

yes, sorry

@vmalaga vmalaga closed this as completed Feb 13, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cezarsa @andrewsmedina @vmalaga