Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upConsul SD shutdown race condition resulting in crash #2200
Comments
juliusv
added
area/stability
component/service discovery
help wanted
kind/bug
low hanging fruit
priority/P1
labels
Nov 18, 2016
fabxc
added a commit
that referenced
this issue
Nov 18, 2016
fabxc
referenced this issue
Nov 18, 2016
Merged
discovery: terminate senders before closing channel #2201
juliusv
closed this
in
#2201
Nov 18, 2016
This comment has been minimized.
This comment has been minimized.
lock
bot
commented
Mar 24, 2019
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
lock
bot
locked and limited conversation to collaborators
Mar 24, 2019
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
juliusv commentedNov 18, 2016
As reported in https://groups.google.com/forum/#!topic/prometheus-developers/NbouN8u1S8Y, Prometheus 1.2.1 crashes with a panic due to nil-channel send upon shutdown in the Consul SD code. Looking at the code, the same problem seems to still exist in master.
The
selectinwatch():https://github.com/prometheus/prometheus/blob/master/retrieval/discovery/consul/consul.go#L302-L306
...may try to write into
chat the moment whenchis closed byRun()(which spawns thewatch()es as goroutines).Run()should wait for thewatch()goroutines to terminate before returning and closingch.