Link to the Kubernetes documentation when discussing startup probes

[victorherraiz]

Even after the changes for #28432, the documentation does not offer a clear recommendation for dealing with projects with possible long start-up times.

Current text:

If an application takes longer to start than the configured liveness period, Kubernetes mentions the "startupProbe" as a possible solution. Generally speaking, the "startupProbe" is not necessarily needed here, as the "readinessProbe" fails until all startup tasks are done. This means your application will not receive traffic until it is ready. However, if your application takes a long time to start, consider using a "startupProbe" to make sure that Kubernetes won’t kill your application while it is in the process of starting. See the section that describes how probes behave during the application lifecycle.

According to K8S documentation, liveness probes start at the same time as readiness probes. And configuring initialDelaySeconds could delay the availability of the service. Even the same service could take an arbitrary time to start, depending on several factors, and starting the liveness probes could cause some restarts.

To avoid this problem, I normally configure the startup probe pointing to the readiness endpoint. And I think it could be nice to mention that in the documentation or create a specific endpoint for startup covering all the scenarios with a complete solution. I prefer the latter option.

The main purpose of this proposal is to make the container available as soon as possible and prevent unnecessary restarts.

[wilkinsona]

I don't think I understand what you're looking for here with "a specific endpoint for startup covering all the scenarios with a complete solution". Kubernetes' own documentation recommends that a startup probe uses the same endpoint as the liveness probe. This is on the docs page to which we already link. We could perhaps also link to https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-startup-probes but I'm not seeing the need for another endpoint.

[victorherraiz]

Thank you, @wilkinsona, for your prompt answer. I use the Spring readiness endpoint because of the lifecycle Spring Boot probes: the service is ACCEPTING_TRAFFIC after being CORRECT.

But after reading the Kubernetes documentation, it makes complete sense to point to liveness; the container is not at risk of restarting and has the same benefit of making the service reachable as soon as possible.

It could be useful to point to that documentation: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-startup-probes

I still think that exposing a /actuator/health/startup endpoint following that recommendation (i.e., the same as liveness) could help developers to have a great default.

What do you think?

[bclozel]

Sorry but I don't understand your point either. The documentation doesn't state that you need a separate endpoint for startup. Thé k8s startup probe is just a way to configure a separate probe configuration for startup if needed. If your application is quick enough to start the liveness is plenty enough.

Adding another endpoint copying the same defaults as liveness would confuse developers. Right now they can add health checks to the liveness group so they should also add the same to the startup group? Or a different set? Why would it matter when the result should be the same and only the k8s timeout really changes, which is not an application concern.

I understand that there are a lot of subtleties in the k8s docs but I would rather focus on application concerns and not require all developers to know all the complexities of distributed systems management in platforms.

[victorherraiz]

Thanks, @bclozel, for your response; I will try to explain my point better.

Some applications do not have a constant startup time; it could depend on many factors: the node resources that may be different between environments or during high-demand hours, the amount of data to load or process at the startup, etc.

Just setting up a liveness probe with a fixed configuration will risk restarts in some scenarios; it could delay restarts or just delay the readiness in others. Getting this right is quite complex. The K8s solution is the startup probe: delay readiness and liveness probing until the startup event.

Adding this link to the documentation will be quite helpful: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#define-startup-probes . It made me reconsider using a readiness probe for startup.

Also, adding a dedicated probe endpoint for startup in the actuator-specific k8s section could be beneficial to developers, in my opinion. They do not have to deal with the nuances and subtleties of k8s docs.

Adding the same groups, by default, as liveness to the startup endpoint makes total sense. In the end, what I try to accomplish is what I mention in my first comment: make the container available as soon as possible and prevent unnecessary restarts.

If you consider that this is too much trouble to solve this problem, maybe adding a comment to configure the liveness endpoint as a startup probe will suffice.