Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track custom anonymous usage statistics from ingester and querier #2685

Merged
merged 3 commits into from
Aug 10, 2022

Conversation

pracucci
Copy link
Collaborator

@pracucci pracucci commented Aug 10, 2022

What this PR does

In this PR I'm adding some custom metrics tracked by the anonymous usage stats tracker, that could help us better understand how Mimir is used (and its scale in the wild). The following custom metrics are tracked:

  • Querier
    • querier_remote_read_requests
    • querier_instant_query_requests
    • querier_range_query_requests
    • querier_exemplars_query_requests
    • querier_labels_query_requests
    • querier_series_query_requests
    • querier_metadata_query_requests
    • querier_cardinality_query_requests
  • Ingester
    • ingester_replication_factor
    • ingester_inmemory_series
    • ingester_inmemory_tenants
    • ingester_appended_samples
    • ingester_appended_exemplars
  • All
    • blocks_storage_backend

The GetInt() and GetCounter() logic has been copied from Loki and I would like to keep it as closest as possible, to be able to move it to a common library in a follow up work.

Known issue: if a report fails to send, we need to try to send the same exact report, because counters are reset each time we build a new one. I will do some refactoring to fix in a follow up PR.

Which issue(s) this PR fixes or relates to

Part of #1815.

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

Signed-off-by: Marco Pracucci <marco@pracucci.com>
@github-actions

This comment has been minimized.

Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci pracucci requested a review from colega August 10, 2022 12:03
@pracucci pracucci marked this pull request as ready for review August 10, 2022 12:03
@github-actions

This comment has been minimized.

pkg/ingester/ingester.go Outdated Show resolved Hide resolved
Copy link
Contributor

@colega colega left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! This was straightforward.

Signed-off-by: Marco Pracucci <marco@pracucci.com>
@pracucci pracucci enabled auto-merge (squash) August 10, 2022 16:01
@github-actions
Copy link
Contributor

Helm <> Jsonnet Diff

⚠️ A difference was detected between the Helm chart and the Jsonnet library.

  1. Use make check-helm-jsonnet-diff to reproduce the output locally.
  2. This test is experimental while we gather feedback about its usefulness.
  3. It does not block your PR from being merged, but we would appreciate you trying to keep feature parity between the Helm chart and Jsonnet library if possible.

If you get stuck on this step and the Mimir maintainers aren't able to help, feel free to merge without making this step pass and tag @Logiraptor so the Mimir maintainers can gather feedback later.

Please see the contribution docs here for more info.

Expand to see the output

Output of https://github.com/grafana/mimir/actions/runs/2834030252

Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
diff -r -u -N scratch/./helm/07-config/ingester-MimirConfig.yml scratch/./jsonnet/08-config/ingester-MimirConfig.yml
--- scratch/./helm/07-config/ingester-MimirConfig.yml	2022-08-10 16:05:30.469191258 +0000
+++ scratch/./jsonnet/08-config/ingester-MimirConfig.yml	2022-08-10 16:05:38.073325427 +0000
@@ -553,7 +553,7 @@
     max_fetched_chunks_per_query: 2000000 (default)
     max_fetched_series_per_query: 0 (default)
     max_global_exemplars_per_user: 0 (default)
-    max_global_series_per_metric: 20000 (default)
+    max_global_series_per_metric: 0
     max_global_series_per_user: 150000 (default)
     max_label_name_length: 1024 (default)
     max_label_names_per_series: 30 (default)
diff -r -u -N scratch/./helm/07-config/overrides-exporter-MimirConfig.yml scratch/./jsonnet/08-config/overrides-exporter-MimirConfig.yml
--- scratch/./helm/07-config/overrides-exporter-MimirConfig.yml	2022-08-10 16:05:30.477191399 +0000
+++ scratch/./jsonnet/08-config/overrides-exporter-MimirConfig.yml	2022-08-10 16:05:38.077325499 +0000
@@ -498,7 +498,7 @@
     max_fetched_chunks_per_query: 2000000 (default)
     max_fetched_series_per_query: 0 (default)
     max_global_exemplars_per_user: 0 (default)
-    max_global_series_per_metric: 20000 (default)
+    max_global_series_per_metric: 0
     max_global_series_per_user: 150000 (default)
     max_label_name_length: 1024 (default)
     max_label_names_per_series: 30 (default)

@pracucci pracucci merged commit a247ea5 into main Aug 10, 2022
@pracucci pracucci deleted the track-custom-anonymous-usage-stats branch August 10, 2022 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants