Skip to content

TSDB: add bucket store metrics in querier #1996

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 27 commits into from
Jan 21, 2020
Merged

TSDB: add bucket store metrics in querier #1996

merged 27 commits into from
Jan 21, 2020

Conversation

pstibrany
Copy link
Contributor

@pstibrany pstibrany commented Jan 17, 2020 •
edited
Loading

This PR adds export of aggegate metrics from TSDB's store.BucketStore and storecache.InMemoryIndexCache, used by Querier component.

This PR uses same approach as #1990 and #1983 before that. It builds on #1990, and moves code related to aggegating and sending metrics into pkg/util/metrics_helper.go file.

Checklist

  • Tests updated
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@pstibrany pstibrany requested a review from pracucci January 17, 2020 14:08
@pstibrany
Moved MetricFamiliesPerUser type to util package.
57a3b0c 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Added prometheus registry to block stores.
c27da49 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Expose first set of metrics from TSDB bucket store.
4785568 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Support for summaries with labels.
b93dd89 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Send data directly to channel to avoid allocating extra slices/maps j…
2834f8d 
…ust to return results.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Added more summaries.
520911f 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Added remaining metrics from TSDB Bucket Store.
7e3a2d5 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Extracted TSDB bucket store metrics into separate type
db6808d 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Added test for bucket_store_metrics_test.
88a118d 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Added test for bucket_store_metrics_test.
23efddb 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Gather and report metrics from Thanos' storecache.InMemoryIndexCache.
821b95b 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Extracted common code that builds MetricFamiliesPerUser
d146805 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Added benchmarks
a9274b2 
We may want to optimize this in the future.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Updated CHANGELOG.md
2ec4fa1 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
pracucci
pracucci reviewed Jan 20, 2020
View reviewed changes
Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job @pstibrany ! The design looks good. I just left few comments here and there.

@pstibrany
Added tests to sum and getMetricsWithLabelNames functions.
9b44603 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Fixes.
eee9e51 
Fixed sum for nil input.
Extracted common code for sum of counters/gauges with labels.
Fixed gauge output (was reported as counter)

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Replaced cortex_bucket_store prefix with cortex_querier_bucket_store.
ece88df 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Removed uninteresting cortex_querier_bucket_store_sent_chunk_size_byt…
2ff0240 
…es metric.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Replaced cortex_store_index_cache prefix with cortex_querier_blocks_i…
92050ab 
…ndex_cache

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Group metrics registration, and register only into non-nil registry.
abaf8cb 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Make message generic.
8857202 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Ignore result and error and make lint happy.
6e803b8 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Copy link
Contributor Author

Thank you Marco for your review. I believe (I will double check) that I've addressed all your feedback now.

@pstibrany
Added test for getMetricsWithLabelNames with no labels.
d5f9693 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Comment about missing m1 in test output.
020ae23 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
pracucci
pracucci reviewed Jan 21, 2020
View reviewed changes
@pstibrany
Fixed duplicate entry in CHANGELOG.md
02d54c4 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
@pstibrany
Use single call.
7741f31 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
pracucci
pracucci reviewed Jan 21, 2020
View reviewed changes
@pstibrany
Typo
a9a95e0 
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
pracucci
pracucci approved these changes Jan 21, 2020
View reviewed changes
Copy link
Contributor

@pracucci pracucci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good job! LGTM 🎉

@pracucci pracucci merged commit ea1a045 into cortexproject:master Jan 21, 2020
@pstibrany pstibrany deleted the tsdb-block-store-metrics branch January 22, 2020 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
2 more reviewers

@pracucci pracucci pracucci approved these changes

@gouthamve gouthamve gouthamve approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@pstibrany @pracucci @gouthamve