Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Start doing the minimum in /status checks #984

Merged
merged 4 commits into from Aug 22, 2023
Merged

Conversation

Jozzey
Copy link
Contributor

@Jozzey Jozzey commented Aug 22, 2023

https://eaflood.atlassian.net/browse/WATER-4096

AWS ELBs require an endpoint they can hit to confirm whether an app is running. They are commonly referred to as health checks and are used to determine whether the ELB should route traffic through to an app instance. In our apps, it is the /status endpoint.

The endpoint in all our repos currently reads in the package.json file to get the app's version number. This information is then used to support the /service-status page in the water-abstraction-ui. Some of the repos also include a test query to the DB to confirm it can connect.

Having checks that confirm you can connect to dependent services (databases, other apps etc) is a good thing. But the ELB health checks are made multiple times a second across all instances. They only care whether an app is up or not. So if, for example, you include querying your DB in /status you're hitting your DB with multiple connections per second, multiplied by the number of server instances you have running.

Including reading a file from disk each time means we're adding an unnecessary load on a service that already has performance and resource usage issues.

We've already added a new /health/info endpoint to each repo and we do DB connection checks elsewhere. So, we can reduce the work of our /status endpoint across all the repos to the bare minimum; returning a static { "status": "alive" } response.

This issue was originally raised in DEFRA/water-abstraction-team#67

https://eaflood.atlassian.net/browse/WATER-4096

AWS ELBs require an endpoint they can hit to confirm whether an app is running. They are commonly referred to as health checks and are used to determine whether the ELB should route traffic through to an app instance. In our apps, it is the `/status` endpoint.

The endpoint in all our repos currently reads in the `package.json` file to get the app's version number. This information is then used to support the `/service-status` page in the water-abstraction-ui. Some of the repos also include a test query to the DB to confirm it can connect.

Having checks that confirm you can connect to dependent services (databases, other apps etc) is a good thing. But the ELB health checks are made multiple times a second across all instances. They only care whether an app is up or not. So if, for example, you include querying your DB in  `/status` you're hitting your DB with multiple connections per second, multiplied by the number of server instances you have running.

Including reading a file from disk each time means we're adding an unnecessary load on a service that already has performance and resource usage issues.

We've already added a new `/health/info` endpoint to each repo and we do DB connection checks elsewhere. So, we can reduce the work of our `/status` endpoint across all the repos to the bare minimum; returning a static `{ "status": "alive" }` response.
@Jozzey Jozzey added the housekeeping Refactoring, tidying up or other work which supports the project label Aug 22, 2023
@Jozzey Jozzey self-assigned this Aug 22, 2023
@Jozzey Jozzey marked this pull request as ready for review August 22, 2023 15:33
@Jozzey Jozzey merged commit 5677597 into main Aug 22, 2023
5 checks passed
@Jozzey Jozzey deleted the update-/status-endpoint branch August 22, 2023 16:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
housekeeping Refactoring, tidying up or other work which supports the project
Projects
None yet
3 participants