Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide health check for Docker image #550

Closed
novoj opened this issue Apr 30, 2024 · 4 comments · Fixed by #554
Closed

Provide health check for Docker image #550

novoj opened this issue Apr 30, 2024 · 4 comments · Fixed by #554
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@novoj
Copy link
Collaborator

novoj commented Apr 30, 2024

To reliably run evitaDB on production systems, we need to provide a health check for the Docker container. The container should be healthy if no OutOfMemoryError occurred and the endpoint responds within 2s with 2 retries.

@novoj novoj added the enhancement New feature or request label Apr 30, 2024
@novoj novoj added this to the Beta milestone Apr 30, 2024
@novoj novoj self-assigned this Apr 30, 2024
@lukashornych
Copy link
Collaborator

Doesn't this solve the alive endpoints we provide in GQL and REST endpoint? Should this do something more?

@novoj
Copy link
Collaborator Author

novoj commented May 1, 2024

Yes, I'm implementing it on the "system" endpoint so we don't have to rely on APIs that could be disabled at any time. I will also detect the situations when queues are full and requests are rejected, also when GC cannot free memory and the system will crash due to insufficient memory, and also when there are a lot of internal errors that might signal some problem. This is not covered by our "liveness" probes in the GQL / REST APIs.

@novoj
Copy link
Collaborator Author

novoj commented May 2, 2024

We should also prepare readiness probe. See Kubernetes documentation

novoj added a commit that referenced this issue May 3, 2024
Readiness probe implemented and tested.
novoj added a commit that referenced this issue May 3, 2024
novoj added a commit that referenced this issue May 3, 2024
Liveness done, but not manually tested.
novoj added a commit that referenced this issue May 3, 2024
novoj added a commit that referenced this issue May 4, 2024
Full implementation manually tested.
novoj added a commit that referenced this issue May 4, 2024
@novoj novoj linked a pull request May 4, 2024 that will close this issue
@novoj novoj closed this as completed in #554 May 4, 2024
@novoj
Copy link
Collaborator Author

novoj commented May 4, 2024

novoj added a commit that referenced this issue May 4, 2024
Added link between readiness and liveness probe. Allow to configure system API port and start delay in Dockerfile using environment variables.
novoj added a commit that referenced this issue May 5, 2024
Fixed work with service loader and probe instantiation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants