Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: mark as unqueryable if status is not_ready/not_healthy #7297

Conversation

MichaHoffmann
Copy link
Contributor

@MichaHoffmann MichaHoffmann commented Apr 22, 2024

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

Added a field to Info RPC to indicate if Prometheus is down. If prometheus is down we should not fan out to the Store API of the sidecar of that prometheus.

Verification

@MichaHoffmann MichaHoffmann force-pushed the mhoffmann/mark-sidecar-as-unqueryable-if-prometheus-is-down branch 4 times, most recently from 1f8c83e to 1df444c Compare April 22, 2024 18:16
@yeya24
Copy link
Contributor

yeya24 commented Apr 22, 2024

I am wondering if we can know a Prometheus is down without introducing a new field? Would be good if we can avoid indroducing a field for one component only.

@MichaHoffmann
Copy link
Contributor Author

I am wondering if we can know a Prometheus is down without introducing a new field? Would be good if we can avoid indroducing a field for one component only.

We could return an error in the info callback too, I was a bit torn between the two.

@MichaHoffmann MichaHoffmann force-pushed the mhoffmann/mark-sidecar-as-unqueryable-if-prometheus-is-down branch from 1df444c to 964f5f8 Compare April 23, 2024 16:43
@MichaHoffmann MichaHoffmann changed the title Sidecar: mark as unqueryable if prometheus is down *: mark as unqueryable if status is not_ready/not_healthy Apr 23, 2024
If the prometheus that belongs to a sidecar is down we dont need to
query the sidecar. This PR makes it so that we take the sidecar out of
the endpoint set then. We do the same for all other store APIs by
retuning an error in the info/Info gRPC call if they are marked as not
ready.

Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>
@MichaHoffmann MichaHoffmann force-pushed the mhoffmann/mark-sidecar-as-unqueryable-if-prometheus-is-down branch from 964f5f8 to 713e5e3 Compare April 23, 2024 16:56
Copy link
Contributor

@yeya24 yeya24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Make sense to me

@MichaHoffmann MichaHoffmann merged commit 57016bd into main Apr 24, 2024
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants