Add `/-/healthy` and `/-/ready` endpoints

[discordianfish]

Prometheus should provide an health endpoint which returns 200 if prometheus is ready to response to queries (recovery is done etc).

The officially recommendation on high availability is to run two instances behind a load balance. In such scenario we need a health endpoint to avoid hitting a prometheus server that got restarted and isn't ready to serve traffic.

[beorn7]

From the discussion in prometheus/pushgateway#105 , I'd say what you want here is a /-/ready endpoint. The other suggested endpoint is /-/healthy, which returns 200 for as long as Prometheus doesn't want to get restarted.

In other words:

Prometheus should bind to /-/healthy immediately after startup and keep responding 200 for as long as it doesn't wish to get killed.

Prometheus should start responding 200 to /-/ready once it is ready to accept queries.

[discordianfish]

Ah nice, yes that totally makes sense 👍

[yonkeltron]

Just wanted to throw in my two cents for this one as it would be awesome to have this for Kubernetes to use as well.

[beorn7]

@gouthamve I just saw you pinged me on IRC about this. I think all my thoughts around this have already been dumped into my last comment above. Please let me know what you are missing, and I'll try to go into more details where needed.

[emluque]

Hi, I have been working on this on my local. I have arrived at a simple working solution. I think I could send a PR before end of week. If nobody else is working on this I would like to claim the issue.

[gouthamve]

Sure @emluque, looking forward to the PR! Incase you get stuck somewhere, feel free to hit us up on IRC or comment here.

EDIT: Hmm, I am not able to assign you to this issue, sorry. Just comment below if you stop working on this.

[SimenB]

#2997 is merged, should this be closed?

[grobie]

Resolved with #2997.

[julianvmodesto]

Is it possible to backport this to 1.7.x? Related to coreos/prometheus-operator#658

[beorn7]

This is part of the 1.x line of Prometheus. It will in fact be released tomorrow (if nothing goes wrong). It will be 1.8.0, however, as this is a new feature and not a bug fix.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.