Missing health check endpoint

[tothandras]

The proxy doesn't have a health check endpoint, which is very handy in a Kubernetes environment for readiness and liveness probes.

(I'm happy to contribute with a PR, when the details are discussed.)

[minrk]

PR sounds great! What should a health check look like?

I would put it at /api/health/

[tothandras]

@minrk It has to be exposed on the proxy itself (proxy-public port), because some limitations of GCE Ingress. I would go with /healthz. The health-check handler could ping the db for example. It is also a problem that load balancing multiple proxy instances (to have HA) with memory store is not possible.

[minrk]

If it has to be on the public port, I think it has to be something highly unlikely for any proxied application to serve, such as /_chp_health.

[tothandras]

I think /healthz is quite unlikely and common for health checks at the same time.

[rgbkrk]

I'm a fan of the leading _ style.

[tothandras]

I think we won't be able to agree on this detail, so let's make it configurable. 🙂

[willingc]

Naming is hard 🤷‍♀️

Why don't we start with _healthz as the name? The rationale: healthz is used in K8 for probes and prefixing with _ satisfies IMHO @minrk's and @rgbkrk's preference for a something unlikely.

[rgbkrk]

The rationale: healthz is used in K8 for probes

That does make it tempting to align with if it makes deployment easier. It does look like the value used for probes is configurable.

[minrk]

Why don't we start with _healthz as the name?

That sounds great! I still would prefer a CHP prefix, though, so maybe /_chp_healthz?

I think /healthz is quite unlikely and common for health checks at the same time.

If it's common, then it's likely to also be exposed by something behind CHP. For instance, if JupyterHub implemented this standard endpoint and deployed with CHP in a single container (the default behavior), this should be proxied to the Hub and not served by CHP itself.

Since CHP is a proxy, we have a special case because all URLs are in a shared namespace with proxied applications, so anything CHP-specific should be on a CHP-specific URL.

Let's go with non-configurable /_chp_healthz.

[yuvipanda]

Can't we have the health check only on the api port and not the proxy port? I'd much prefer that than overloading the proxying port :)

This is what traefik does for example.

[minrk]

@yuvipanda if that's possible, let's definitely do that. I thought @tothandras said this wasn't possible, though.

[yuvipanda]

Ah I see. I don't think kubernetes has that limitation - it's possible that GCE has it for their ingress, however.