Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #3204 after it has been inadvertently removed #3979

Merged
merged 1 commit into from
Mar 26, 2021

Commits on Mar 26, 2021

  1. mixin: Use sidecar's metric timestamp for healthcheck (thanos-io#3204)

    During prometheus updates the alert was firing because the metric was
    initialized with a value of '0' before the first heartbeat was sent. As
    such, the evaluation of the alert results into actually taking just the
    value of time() into consideration which led to misleading information
    about the health of the sidecar.
    
    As the thanos_sidecar_last_heartbeat_success_time_seconds metric is
    effectively just a timestamp that resets on new deployments, we can
    simply wrap it around the timestamp() function which should return
    almost the same value of the metric itself with the added benefit that
    heartbeat resets will be ignored.
    
    This also refactors the relevant tests and drops the timeout to 4
    minutes in order to ensure that we do not get hit by stale data if
    the sidecar takes longer to start.
    
    Signed-off-by: Markos Chandras <markos@chandras.me>
    Signed-off-by: Damien Grisonnet <dgrisonn@redhat.com>
    hwoarang authored and dgrisonnet committed Mar 26, 2021
    Configuration menu
    Copy the full SHA
    ce79e2f View commit details
    Browse the repository at this point in the history