Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

panic: emitter is closed #853

Closed
kishansagathiya opened this issue Jul 25, 2019 · 2 comments

Comments

@kishansagathiya
Copy link
Member

commented Jul 25, 2019

Additional information:

  • OS: Linux
  • IPFS Cluster version: master, ef8f2cb
  • Installation method: built from source

Describe the bug:

My port for rest api was occupied, when I ran tests go test -v ./api/rest. Test failed as expected. I freed the port. On running the test again I saw this error

➜  ipfs-cluster ✗ go test -v ./api/rest 
=== RUN   TestLoadEmptyJSON
--- PASS: TestLoadEmptyJSON (0.00s)
=== RUN   TestLoadJSON
--- PASS: TestLoadJSON (0.00s)
=== RUN   TestApplyEnvVars
--- PASS: TestApplyEnvVars (0.00s)
=== RUN   TestLibp2pConfig
panic: emitter is closed

goroutine 71 [running]:
github.com/libp2p/go-eventbus.(*emitter).Emit(0xc000191200, 0xc81540, 0xc0000b7980)
	/home/kishan/code/pkg/mod/github.com/libp2p/go-eventbus@v0.0.3/basic.go:33 +0x6f
github.com/libp2p/go-libp2p/p2p/host/basic.(*BasicHost).RemoveStreamHandler(0xc0000d62c0, 0xd4501d, 0xc)
	/home/kishan/code/pkg/mod/github.com/libp2p/go-libp2p@v0.2.1/p2p/host/basic/basic_host.go:419 +0x112
github.com/libp2p/go-libp2p-gostream.(*listener).Close(0xc000046400, 0xc00014add0, 0x10)
	/home/kishan/code/pkg/mod/github.com/libp2p/go-libp2p-gostream@v0.1.2/listener.go:39 +0x53
net/http.(*onceCloseListener).close(...)
	/usr/local/go/src/net/http/server.go:3286
sync.(*Once).Do(0xc0000b7570, 0xc00014ae00)
	/usr/local/go/src/sync/once.go:44 +0xb3
net/http.(*onceCloseListener).Close(0xc0000b7560, 0x0, 0x55)
	/usr/local/go/src/net/http/server.go:3282 +0x54
net/http.(*Server).Serve(0xc0003684e0, 0xf91f80, 0xc000046400, 0xf815e0, 0xc0000b2210)
	/usr/local/go/src/net/http/server.go:2879 +0x4b0
github.com/ipfs/ipfs-cluster/api/rest.(*API).runLibp2pServer(0xc00012b050, 0xf95700, 0xc000362980)
	/home/kishan/code/ipfs-cluster/api/rest/restapi.go:477 +0x322
created by github.com/ipfs/ipfs-cluster/api/rest.(*API).run
	/home/kishan/code/ipfs-cluster/api/rest/restapi.go:449 +0x8a
FAIL	github.com/ipfs/ipfs-cluster/api/rest	0.094s

Though, looking at the logs it seems like it is happening from go-eventbus

After that tests ran fine.

@kishansagathiya

This comment has been minimized.

Copy link
Member Author

commented Jul 25, 2019

Just happened again,

hsanjuan added a commit that referenced this issue Jul 25, 2019
Cancelling its context before closing the listeners and de-registering
protocols is ground for panics on libp2p.
@hsanjuan

This comment has been minimized.

Copy link
Collaborator

commented Jul 25, 2019

I had seen this before but I thought it was related to the other go-eventbus panic. Seems not though.

In that test:

  • SetClient() is never called
  • runLibp2pServer hangs waiting for it
  • The API is Shutdown()
  • This cancels the context before the libp2p listener is closed
  • cancelling the context effectively kills the libp2p host
  • that closes the eventbus
  • Right after cancelling the context api.rpcReady is closed and the listener further below
  • The runLibp2pServer, which had been waiting on that channel, moves forward and calls server.Serve on the libp2p listener.
  • In the meantime, because context was cancelled, go-eventbus emmitters are being closed.
  • If Shutdown called Close() on the listener before the server started, when it starts it internally that call Accept which errors because the listener was closed already and this makes an additional late Close() on the listener.
  • That late listener.Close() made from http.Server is the one that panics as it usually arrives after the emitters have finished closing. The previous Close() in Shutdown() usually arrives before and does not panic (but it can be made to with a Sleep()).

Handled in #855 . Regression reported on : https://github.com/libp2p/go-libp2p/issues/691

@hsanjuan hsanjuan referenced this issue Jul 25, 2019
hsanjuan added a commit that referenced this issue Jul 26, 2019
Cancelling its context before closing the listeners and de-registering
protocols is ground for panics on libp2p.
@hsanjuan hsanjuan closed this in 7a78620 Jul 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.