Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import cortex mixin from upstream #373

Merged
merged 535 commits into from
Oct 19, 2021
Merged

Import cortex mixin from upstream #373

merged 535 commits into from
Oct 19, 2021

Conversation

simonswine
Copy link
Contributor

@simonswine simonswine commented Oct 18, 2021 •
edited
Loading

What this PR does:

Imports cortex mixin from upstream including history and placing it under jsonnet/mimir-mixin

Which issue(s) this PR fixes:

This allows to diverge with alerts and runbooks from the Cortex project.

Continued from #366

Checklist

  • Tests CI updated
  • Documentation added
  • [ ] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

pracucci and others added 30 commits January 27, 2021 13:05
@pracucci
Increased CortexAllocatingTooMuchMemory alert threshold
fbab4f2 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Merge pull request grafana/cortex-jsonnet#256 from grafana/raise-inge…
c7a4115 
…ster-memory-alert-threshold

Increased CortexAllocatingTooMuchMemory alert threshold
@gouthamve
Add alert for etcd memory limits close
bbc5c34 
Signed-off-by: Goutham Veeramachaneni <gouthamve@gmail.com>
@gouthamve
Merge pull request grafana/cortex-jsonnet#261 from gouthamve/add-etcd…
0ff411b 
…-memory-alert

Add alert for etcd memory limits close
@replay
the distributor now supports push via GRPC (grafana/cortex-jsonnet#266)
d5ad43a 
Signed-off-by: Mauro Stettler <mauro.stettler@gmail.com>
@pracucci
Fixed CortexQuerierHighRefetchRate alert
adb82b6 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Fixed label matcher
667ca36 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Sort legend descending in the CPU/memory panels
5c75158 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Merge pull request grafana/cortex-jsonnet#271 from grafana/resources-…
97806d9 
…tooltip-decrescent-sorting

Sort legend descending in the CPU/memory panels.
@pracucci
Merge branch 'main' into fix-refetch-alert
353f403
@pracucci
Merge pull request grafana/cortex-jsonnet#268 from grafana/fix-refetc…
d281bd6 
…h-alert

Fixed CortexQuerierHighRefetchRate alert
@pracucci
Add slow queries dashboard
a7bdd2e 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Added tenant ID field to the table
419eaba 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Merge pull request grafana/cortex-jsonnet#272 from grafana/add-slow-q…
c616398 
…ueries-dashboard

Add slow queries dashboard
@tomwilkie
Add recording rules to calculate Cortex scaling
0774870 
- Update dashboard so it only shows under provisioned services and why
- Add sizing rules based on limits.
- Add some docs to the dashboard.

Signed-off-by: Tom Wilkie <tom@grafana.com>
@tomwilkie
Merge pull request grafana/cortex-jsonnet#278 from grafana/scaling-rules
fe87d90 
Add recording rules to calculate Cortex scaling
@pracucci
Increased CortexRequestErrors alert severity
8486777 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Fixed "Disk Writes" and "Disk Reads" panels
1c4dec6 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Merge pull request grafana/cortex-jsonnet#280 from grafana/fix-node-d…
aaaefee 
…isk-panels

Fixed "Disk Writes" and "Disk Reads" panels
@pracucci
Pre-compute aggregations to optimize scaling recording rules
4ff33a3 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Removed 5m step from subquery
7ba8424 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Merge pull request grafana/cortex-jsonnet#284 from grafana/optimize-r…
f1fb713 
…ecording-rules

Pre-compute aggregations to optimize scaling recording rules
@pracucci
Add function to customize compactor statefulset
5c36a63 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Use the job name in compactor alerts too
d7fbc23 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Merge pull request grafana/cortex-jsonnet#287 from grafana/add-functi…
2715796 
…on-to-create-compactor-statefulset

Add function to customize compactor statefulset
@pracucci
Fixed CortexCompactorRunFailed threshold
73c6770 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Merge pull request grafana/cortex-jsonnet#288 from grafana/fix-compac…
5862286 
…tor-alert

Fixed CortexCompactorRunFailed threshold
@pracucci
Added Cortex Rollout progress dashboard
a103a95 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Merge pull request grafana/cortex-jsonnet#289 from grafana/add-rollou…
b0d4587 
…t-progress-dashboard

Added Cortex Rollout progress dashboard
@pracucci
Fix 'Unhealthy pods' in Cortex Rollout dashboard
efa84d4 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
pracucci and others added 7 commits October 14, 2021 09:45
@pracucci
Merge pull request grafana/cortex-jsonnet#404 from grafana/improve-Co…
4de2e29 
…rtexIngesterReachingSeriesLimit

Fine tune CortexIngesterReachingSeriesLimit alert
@pracucci
Merge pull request grafana/cortex-jsonnet#405 from grafana/alert-on-s…
fd975db 
…tuck-rollout

Add CortexRolloutStuck alert
@pracucci
Added CortexFailingToTalkToConsul alert
dcb3306 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Fixed alert message
d79b304 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Update alert to be generic to KV stores
d02bb6b 
Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci
Merge pull request grafana/cortex-jsonnet#406 from grafana/alert-on-c…
859efc9 
…onsul-failures

Added CortexFailingToTalkToConsul alert
@simonswine
Merge commit '859efc9' into 20211014_import-cortex-mixin
1680cc8
@simonswine simonswine changed the title 20211018 import cortex mixin Import cortex mixin from upstream Oct 18, 2021
@simonswine simonswine mentioned this pull request Oct 18, 2021
Closed
3 tasks
@simonswine
Copy link
Contributor Author

@56quarters: Another thing to note is that it would be nice to standardize where mixins are in a repository between Mimir, Loki, and Tempo.

This is a good point, which of the two do we prefer. (I have a slight preference towards operations, how do others feel?)

@simonswine
Copy link
Contributor Author

@56quarters: I notice this doesn't import the jsonnet for running Mimir with Tanka, should it?

I think eventually yes, but for now I would like to make a start with adding some alerts and runbook content for query-sharding only. I think the operational jsonnet manifests can be added separtely.

@56quarters
Copy link
Contributor

@56quarters: Another thing to note is that it would be nice to standardize where mixins are in a repository between Mimir, Loki, and Tempo.

This is a good point, which of the two do we prefer. (I have a slight preference towards operations, how do others feel?)

I prefer "operations" as well

pracucci
pracucci reviewed Oct 18, 2021
View reviewed changes
Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job!

I know you didn't do it, but let me say saying it: I wouldn't do any "Cortex -> Mimir" renaming in alerts / dashboards at this stage. We can do it later on, in separate PRs (mixing it with this import PR would make review more difficult).

pracucci
pracucci approved these changes Oct 19, 2021
View reviewed changes
Copy link
Collaborator

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job! I didn't check every single command in Makefile but, as far as CI passes, I'm good :) I think we can eventually reiterate on it if we find any issue in the future.

@pracucci pracucci merged commit 025285e into main Oct 19, 2021
@pracucci pracucci deleted the 20211018_import-cortex-mixin branch October 19, 2021 16:06
pracucci added a commit that referenced this pull request Oct 19, 2021
@pracucci
Revert "Import cortex mixin from upstream (#373)"
0b67b87 
This reverts commit 025285e.
@pracucci pracucci restored the 20211018_import-cortex-mixin branch October 19, 2021 16:08
@simonswine simonswine mentioned this pull request Oct 19, 2021
Merged
2 tasks
@pracucci pracucci deleted the 20211018_import-cortex-mixin branch October 19, 2021 16:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pracucci pracucci pracucci approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.