Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add rule for critical distributor inflight push request alert #408

Merged
merged 6 commits into from
Oct 22, 2021

Conversation

treid314
Copy link
Contributor

@treid314 treid314 commented Oct 15, 2021

What this PR does: This PR adds a critical alert for cortex_distributor_inflight_push_requests reaching a 90% of the set max_inflight_push_requests limits.

Checklist

  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
  • Runbook changes

cortex-mixin/alerts/alerts.libsonnet Outdated Show resolved Hide resolved
cortex-mixin/alerts/alerts.libsonnet Outdated Show resolved Hide resolved
cortex-mixin/docs/playbooks.md Outdated Show resolved Hide resolved
Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a CHANGELOG entry too, please?

cortex-mixin/alerts/alerts.libsonnet Outdated Show resolved Hide resolved
cortex-mixin/alerts/alerts.libsonnet Outdated Show resolved Hide resolved
cortex-mixin/docs/playbooks.md Outdated Show resolved Hide resolved
cortex-mixin/docs/playbooks.md Outdated Show resolved Hide resolved
cortex-mixin/docs/playbooks.md Outdated Show resolved Hide resolved
cortex-mixin/docs/playbooks.md Outdated Show resolved Hide resolved
Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (modulo the nits). Thanks!

cortex-mixin/alerts/alerts.libsonnet Outdated Show resolved Hide resolved
cortex-mixin/docs/playbooks.md Outdated Show resolved Hide resolved
cortex-mixin/docs/playbooks.md Outdated Show resolved Hide resolved
@treid314 treid314 marked this pull request as ready for review October 22, 2021 19:42
@treid314 treid314 requested a review from a team as a code owner October 22, 2021 19:42
Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@treid314 treid314 merged commit 92f3b64 into grafana:main Oct 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants