Improve probe setup #183
Merged
Improve probe setup #183
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Liveness probe is now a TCP probe. This has the effect that if the server is overloaded and does not respond to HTTP requests in time that it will not be killed as long as the server still listens on the TCP port (i.e. the server is actually alive). Therefore it can finish the requests (this shouldn't take forever due to gunicorn request timeouts). The readiness probe now is an HTTP probe, so if the server is overloaded, it won't receive any new requests. Instead of initialDelaySeconds, a startup HTTP probe is used which queries the service every second, so that it's marked as available as soon as it's actually available. This is vital for one of our use cases where we scale the service to zero and only activate it on the first request, i.e. users wait for the service to actually become available.
ividito
approved these changes
Mar 10, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Liveness probe is now a TCP probe. This has the effect that if the server is overloaded and does not respond to HTTP requests in time that it will not be killed as long as the server still listens on the TCP port (i.e. the server is actually alive). Therefore it can finish the requests (this shouldn't take forever due to gunicorn request timeouts).
The readiness probe now is an HTTP probe, so if the server is overloaded, it won't receive any new requests.
Some http health endpoints also check related services like the DB. So in case the DB is down, you usually wouldn't want the service to be killed. This is achieved in this setup since only the readiness probe fails.
Instead of initialDelaySeconds, a startup HTTP probe is used which queries the service every second, so that it's marked as available as soon as it's actually available. This is vital for one of our use cases where we scale the service to zero and only activate it on the first request, i.e. users wait for the service to actually become available.
We have used a setup like this successfully, but in other use cases also other configurations could be useful, so this could also be made configurable via values. This setup is probably a reasonable default configuration though.