Query: Trim very long external labels and add cmd flag to optionally specify metric labels to collect #5785

utukJ · 2022-10-13T07:55:58Z

I added CHANGELOG entry for this change.
Change is not relevant to the end user.

Changes

trimmed external label
added []string flag query.metric-label for optionally specifying labels to collect
added unit tests

Verification

Wrote unit test

Signed-off-by: utukj <utukphd@gmail.com>

bwplotka

Amazing work! I gave some detailed comments to make this PR even better. Feel free to disagree with anything 🤗

Almost ready to be merged 💪🏽

cmd/thanos/query.go

pkg/query/endpointset.go

pkg/query/endpointset_test.go

fpetkovski · 2022-10-13T10:48:11Z

Thanks for this PR. I am not sure whether we should even keep external labels in metrics. I would consider just dropping them completely and avoiding complications.

full sentence fix Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com>

utukJ · 2022-10-13T13:06:29Z

Thanks for this PR. I am not sure whether we should even keep external labels in metrics. I would consider just dropping them completely and avoiding complications.

Thanks for the feedback @fpetkovski . Dropping external labels was discussed. Checkout the discussion here.

fpetkovski · 2022-10-13T13:25:03Z

I wasn't aware of that, thanks for pointing it out :)

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com>

Signed-off-by: utukj <utukphd@gmail.com>

…uery-labels

Signed-off-by: utukj <utukphd@gmail.com>

…-labels

Signed-off-by: utukj <utukphd@gmail.com>

pkg/query/endpointset_test.go

Signed-off-by: utukj <utukphd@gmail.com>

Signed-off-by: GitHub <noreply@github.com> Signed-off-by: GitHub <noreply@github.com> Co-authored-by: yeya24 <yeya24@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com>

…5673) * Create a PathOrContent reloader Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add docs to staticPathContent.Rewrite Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Run goimports Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Properly cancel the context in the test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Watch parent directory of file This helps handling deletes and other situations. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless ctx.Done() Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add a debounce timer to config reload It helps managing situations where a create event is followed by a write or when a big file write is sent by the fsnotify backend as many write events. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix event.Op bitmask check Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update lastReload Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix debouncer for path content reloader Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Improve documentation of the PathContentRealoder Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Dain reload timer before resetting Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Run tests in parallel Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Simplify debouncing logic Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add more tests to file reloader Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Simplify condition for triggering reload Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Use absolute path to config file Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Get rid of parallel test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Put back 2s wait between fs operations Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless sleep Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Stop reloadTimer when context cancelled Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove unused fucntion Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add missing copyright to test file Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Auto-reload tenant limit config on file changes Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Wrap error when reloading config Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Move limiter config reloader and update logs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Get rid of useless types and allocations Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove errorChan from config reload starter Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Use UnRegisterer in the Limiter To ensure that limit reloads will be able to re-register their metrics. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Better guard against nil registerer in the limiter Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove wrong nil guard Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com>

…-io#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix some linter warnings Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless logs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Refactor query tests Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove TODO comment Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix query tests Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Reformat query docs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless return Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Put back old query docs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update query docs again Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix e2e env name Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add missing copyright notice. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Clean up test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update changelog Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix broken changelog after merge Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove extra empty line Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Co-authored-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com>

Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: utukj <utukphd@gmail.com>

bwplotka · 2022-10-21T09:44:08Z

Checks are failing.

Signed-off-by: utukj <utukphd@gmail.com>

GiedriusS

Looks solid, solves a problem for us! 💪 awesome work! Let's wait for @bwplotka to see if he wants to do another review.

pkg/query/endpointset_test.go

cmd/thanos/query.go

bwplotka

LGTM, mod 2 minor nits!

pkg/query/endpointset_test.go

bwplotka

.

Signed-off-by: bwplotka <bwplotka@gmail.com>

…nto query-labels

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com>

…uery-labels

…-labels

Signed-off-by: utukj <utukphd@gmail.com>

bwplotka

2 suggestions are still unresolved, ping (: Almost there!

…ross them." This reverts commit 96b1545. Signed-off-by: utukj <utukphd@gmail.com>

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com>

Signed-off-by: utukj <utukphd@gmail.com>

…uery-labels

GiedriusS

CircleCI has some kind of bug but tests pass locally so I think we can merge this because all comments were addressed. 👍 💪

…specify metric labels to collect (thanos-io#5785) * added unit tests for long labels and no external labels Signed-off-by: utukj <utukphd@gmail.com> * trimmed too long external labels Signed-off-by: utukj <utukphd@gmail.com> * added optional label selection Signed-off-by: utukj <utukphd@gmail.com> * added cmd flag for choosing metric labels Signed-off-by: utukj <utukphd@gmail.com> * updated docs Signed-off-by: utukj <utukphd@gmail.com> * Update pkg/query/endpointset.go full sentence fix Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com> * Update pkg/query/endpointset.go Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com> * minor fixes from code review Signed-off-by: utukj <utukphd@gmail.com> * fixed code comments Signed-off-by: utukj <utukphd@gmail.com> * used enum for labels Signed-off-by: utukj <utukphd@gmail.com> * updated query docs Signed-off-by: utukj <utukphd@gmail.com> * cleaned up tests Signed-off-by: utukj <utukphd@gmail.com> * Updates busybox SHA (thanos-io#5793) Signed-off-by: GitHub <noreply@github.com> Signed-off-by: GitHub <noreply@github.com> Co-authored-by: yeya24 <yeya24@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com> * Receive: Reload tenant limit configuration on file change (thanos-io#5673) * Create a PathOrContent reloader Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add docs to staticPathContent.Rewrite Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Run goimports Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Properly cancel the context in the test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Watch parent directory of file This helps handling deletes and other situations. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless ctx.Done() Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add a debounce timer to config reload It helps managing situations where a create event is followed by a write or when a big file write is sent by the fsnotify backend as many write events. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix event.Op bitmask check Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update lastReload Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix debouncer for path content reloader Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Improve documentation of the PathContentRealoder Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Dain reload timer before resetting Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Run tests in parallel Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Simplify debouncing logic Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add more tests to file reloader Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Simplify condition for triggering reload Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Use absolute path to config file Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Get rid of parallel test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Put back 2s wait between fs operations Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless sleep Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Stop reloadTimer when context cancelled Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove unused fucntion Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add missing copyright to test file Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Auto-reload tenant limit config on file changes Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Wrap error when reloading config Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Move limiter config reloader and update logs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Get rid of useless types and allocations Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove errorChan from config reload starter Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Use UnRegisterer in the Limiter To ensure that limit reloads will be able to re-register their metrics. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Better guard against nil registerer in the limiter Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove wrong nil guard Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com> * Query: add query metrics to calls going through the Store API (thanos-io#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix some linter warnings Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless logs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Refactor query tests Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove TODO comment Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix query tests Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Reformat query docs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless return Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Put back old query docs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update query docs again Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix e2e env name Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add missing copyright notice. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Clean up test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update changelog Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix broken changelog after merge Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove extra empty line Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Co-authored-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com> * docs: mark me as shepherd for next release (thanos-io#5797) Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: utukj <utukphd@gmail.com> * Revert "docs: mark me as shepherd for next release (thanos-io#5797)" This reverts commit ea646a6. Signed-off-by: utukj <utukphd@gmail.com> * Revert "Query: add query metrics to calls going through the Store API (thanos-io#5741)" This reverts commit eec4fd0. Signed-off-by: utukj <utukphd@gmail.com> * Revert "Receive: Reload tenant limit configuration on file change (thanos-io#5673)" This reverts commit 24e1cc0. Signed-off-by: utukj <utukphd@gmail.com> * Revert "Updates busybox SHA (thanos-io#5793)" This reverts commit 9474c00. Signed-off-by: utukj <utukphd@gmail.com> * Updates busybox SHA (thanos-io#5793) Signed-off-by: GitHub <noreply@github.com> Signed-off-by: GitHub <noreply@github.com> Co-authored-by: yeya24 <yeya24@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com> * Receive: Reload tenant limit configuration on file change (thanos-io#5673) * Create a PathOrContent reloader Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add docs to staticPathContent.Rewrite Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Run goimports Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Properly cancel the context in the test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Watch parent directory of file This helps handling deletes and other situations. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless ctx.Done() Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add a debounce timer to config reload It helps managing situations where a create event is followed by a write or when a big file write is sent by the fsnotify backend as many write events. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix event.Op bitmask check Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update lastReload Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix debouncer for path content reloader Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Improve documentation of the PathContentRealoder Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Dain reload timer before resetting Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Run tests in parallel Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Simplify debouncing logic Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add more tests to file reloader Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Simplify condition for triggering reload Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Use absolute path to config file Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Get rid of parallel test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Put back 2s wait between fs operations Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless sleep Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Stop reloadTimer when context cancelled Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove unused fucntion Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add missing copyright to test file Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Auto-reload tenant limit config on file changes Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Wrap error when reloading config Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Move limiter config reloader and update logs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Get rid of useless types and allocations Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove errorChan from config reload starter Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Use UnRegisterer in the Limiter To ensure that limit reloads will be able to re-register their metrics. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Better guard against nil registerer in the limiter Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove wrong nil guard Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com> * Query: add query metrics to calls going through the Store API (thanos-io#5741) * Implement granular query performance metrics for Thanos Query These are grabbed from the data returned by multiple Store APIs after execution of a query. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix some linter warnings Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless logs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Refactor query tests Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix long function definition (newQuerier) Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove TODO comment Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix query tests Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Reformat query docs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove useless return Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Put back old query docs Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update query docs again Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix e2e env name Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Add missing copyright notice. Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Bump wait time to twice scrape interval Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Attempt to fix randomly failing test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Checking more metrics to ensure the store is ready Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Clean up test Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Do not record store api metrics when didn't touch series or samples Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Retrigger CI Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Also skip store api metrics on zero chunks touched Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Update changelog Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Fix broken changelog after merge Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Remove extra empty line Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Refactor names and (un)exported types and fields Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Start listing metrics exported by Thanos Query Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Rename pkg/store/metrics -> pkg/store/telemetry Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> * Get rid of the pkg/store/telemetry package Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Co-authored-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com> * docs: mark me as shepherd for next release (thanos-io#5797) Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: utukj <utukphd@gmail.com> * Revert "docs: mark me as shepherd for next release (thanos-io#5797)" This reverts commit c509c0e. Signed-off-by: utukj <utukphd@gmail.com> * Revert "Updates busybox SHA (thanos-io#5793)" This reverts commit ad11a03. Signed-off-by: utukj <utukphd@gmail.com> * Revert "Query: add query metrics to calls going through the Store API (thanos-io#5741)" This reverts commit 7a77769. Signed-off-by: utukj <utukphd@gmail.com> * Revert "Receive: Reload tenant limit configuration on file change (thanos-io#5673)" This reverts commit 32ca327. Signed-off-by: utukj <utukphd@gmail.com> * fixed lint issue Signed-off-by: utukj <utukphd@gmail.com> * added unit test for truncate and clean up Signed-off-by: utukj <utukphd@gmail.com> * fixed truncate label func and added more tests Signed-off-by: utukj <utukphd@gmail.com> * removed name from truncate test Signed-off-by: utukj <utukphd@gmail.com> * reorganized test cases and removed redundant comments Signed-off-by: utukj <utukphd@gmail.com> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com> * Update pkg/query/endpointset.go Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com> * fixed failing checks Signed-off-by: utukj <utukphd@gmail.com> * e2e: Adding test for querier with two stores loadbalancing across them. Signed-off-by: bwplotka <bwplotka@gmail.com> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com> * dumped long expected output in unittest Signed-off-by: utukj <utukphd@gmail.com> * Revert "e2e: Adding test for querier with two stores loadbalancing across them." This reverts commit 96b1545. Signed-off-by: utukj <utukphd@gmail.com> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com> * Update pkg/query/endpointset_test.go Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com> * moved label definition to endpointset Signed-off-by: utukj <utukphd@gmail.com> Signed-off-by: utukj <utukphd@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com> Signed-off-by: GitHub <noreply@github.com> Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Signed-off-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: bwplotka <bwplotka@gmail.com> Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: yeya24 <yeya24@users.noreply.github.com> Co-authored-by: Douglas Camata <159076+douglascamata@users.noreply.github.com> Co-authored-by: Matej Gera <38492574+matej-g@users.noreply.github.com> Co-authored-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

utukJ added 4 commits October 13, 2022 08:09

added unit tests for long labels and no external labels

8c76dab

Signed-off-by: utukj <utukphd@gmail.com>

trimmed too long external labels

c5f1536

Signed-off-by: utukj <utukphd@gmail.com>

added optional label selection

745e729

Signed-off-by: utukj <utukphd@gmail.com>

added cmd flag for choosing metric labels

72ce975

Signed-off-by: utukj <utukphd@gmail.com>

pull-request-size bot added the size/L label Oct 13, 2022

utukJ changed the title ~~Query: Trim very long external labels and add cmd flag to optionally choose metric labels to collect~~ Query: Trim very long external labels and add cmd flag to optionally specify metric labels to collect Oct 13, 2022

updated docs

2c08e6d

Signed-off-by: utukj <utukphd@gmail.com>

bwplotka reviewed Oct 13, 2022

View reviewed changes

Update pkg/query/endpointset.go

9fe8ea7

full sentence fix Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com>

utukJ and others added 7 commits October 13, 2022 19:29

Update pkg/query/endpointset.go

2107140

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com>

minor fixes from code review

606cba5

Signed-off-by: utukj <utukphd@gmail.com>

Merge branch 'query-labels' of https://github.com/utukj/thanos into q…

c581cc7

…uery-labels

fixed code comments

a07e974

Signed-off-by: utukj <utukphd@gmail.com>

used enum for labels

03b5289

Signed-off-by: utukj <utukphd@gmail.com>

Merge branch 'main' of https://github.com/thanos-io/thanos into query…

4b07af3

…-labels

updated query docs

be3ca53

Signed-off-by: utukj <utukphd@gmail.com>

sonatype-lift bot reviewed Oct 18, 2022

View reviewed changes

pkg/query/endpointset_test.go Outdated Show resolved Hide resolved

sonatype-lift bot reviewed Oct 18, 2022

View reviewed changes

pkg/query/endpointset_test.go Outdated Show resolved Hide resolved

utukJ force-pushed the query-labels branch from 3e9cfe1 to 735ac7d Compare October 18, 2022 15:36

utukJ and others added 5 commits October 18, 2022 16:51

cleaned up tests

32a0a98

Signed-off-by: utukj <utukphd@gmail.com>

Updates busybox SHA (thanos-io#5793)

9474c00

Signed-off-by: GitHub <noreply@github.com> Signed-off-by: GitHub <noreply@github.com> Co-authored-by: yeya24 <yeya24@users.noreply.github.com> Signed-off-by: utukj <utukphd@gmail.com>

docs: mark me as shepherd for next release (thanos-io#5797)

ea646a6

Let's release the RC on Friday. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com> Signed-off-by: utukj <utukphd@gmail.com>

utukJ force-pushed the query-labels branch from 735ac7d to ea646a6 Compare October 18, 2022 15:52

pull-request-size bot added size/XXL and removed size/L labels Oct 18, 2022

fixed failing checks

56ef7cb

Signed-off-by: utukj <utukphd@gmail.com>

GiedriusS previously approved these changes Oct 21, 2022

View reviewed changes

bwplotka reviewed Oct 21, 2022

View reviewed changes

pkg/query/endpointset_test.go Outdated Show resolved Hide resolved

bwplotka reviewed Oct 21, 2022

View reviewed changes

cmd/thanos/query.go Outdated Show resolved Hide resolved

bwplotka previously approved these changes Oct 21, 2022

View reviewed changes

bwplotka reviewed Oct 21, 2022

View reviewed changes

pkg/query/endpointset_test.go Outdated Show resolved Hide resolved

bwplotka reviewed Oct 21, 2022

View reviewed changes

pkg/query/endpointset_test.go Outdated Show resolved Hide resolved

bwplotka reviewed Oct 21, 2022

View reviewed changes

bwplotka and others added 3 commits October 21, 2022 14:32

e2e: Adding test for querier with two stores loadbalancing across them.

96b1545

Signed-off-by: bwplotka <bwplotka@gmail.com>

Merge branch 'utuk-lb-query' of https://github.com/thanos-io/thanos i…

fb38b75

…nto query-labels

Update pkg/query/endpointset_test.go

edaf16e

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com>

utukJ dismissed stale reviews from bwplotka and GiedriusS via edaf16e October 25, 2022 14:41

utukJ added 3 commits October 25, 2022 15:47

Merge branch 'query-labels' of https://github.com/utukj/thanos into q…

f11cb35

…uery-labels

Merge branch 'main' of https://github.com/thanos-io/thanos into query…

9dfa81b

…-labels

dumped long expected output in unittest

14ee774

Signed-off-by: utukj <utukphd@gmail.com>

bwplotka reviewed Oct 26, 2022

View reviewed changes

Revert "e2e: Adding test for querier with two stores loadbalancing ac…

f963ac1

…ross them." This reverts commit 96b1545. Signed-off-by: utukj <utukphd@gmail.com>

utukJ force-pushed the query-labels branch from e01509f to f963ac1 Compare October 26, 2022 06:03

utukJ and others added 5 commits October 26, 2022 07:05

Update pkg/query/endpointset_test.go

a34f356

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com>

Update pkg/query/endpointset_test.go

c4e0f89

Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com> Signed-off-by: Uwakmfon Utuk <41128987+utukJ@users.noreply.github.com>

moved label definition to endpointset

c26cf74

Signed-off-by: utukj <utukphd@gmail.com>

Merge branch 'query-labels' of https://github.com/utukj/thanos into q…

c68bdcd

…uery-labels

Merge branch 'main' into query-labels

f34506d

GiedriusS approved these changes Oct 26, 2022

View reviewed changes

GiedriusS merged commit 989efe5 into thanos-io:main Oct 27, 2022

yeya24 mentioned this pull request Dec 19, 2022

gRPC connections metric has long label values #5540

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Query: Trim very long external labels and add cmd flag to optionally specify metric labels to collect #5785

Query: Trim very long external labels and add cmd flag to optionally specify metric labels to collect #5785

utukJ commented Oct 13, 2022 •

edited

bwplotka left a comment

fpetkovski commented Oct 13, 2022

utukJ commented Oct 13, 2022

fpetkovski commented Oct 13, 2022

bwplotka commented Oct 21, 2022

GiedriusS left a comment

bwplotka left a comment

bwplotka left a comment

bwplotka left a comment

GiedriusS left a comment

Query: Trim very long external labels and add cmd flag to optionally specify metric labels to collect #5785

Query: Trim very long external labels and add cmd flag to optionally specify metric labels to collect #5785

Conversation

utukJ commented Oct 13, 2022 • edited

Changes

Verification

bwplotka left a comment

Choose a reason for hiding this comment

fpetkovski commented Oct 13, 2022

utukJ commented Oct 13, 2022

fpetkovski commented Oct 13, 2022

bwplotka commented Oct 21, 2022

GiedriusS left a comment

Choose a reason for hiding this comment

bwplotka left a comment

Choose a reason for hiding this comment

bwplotka left a comment

Choose a reason for hiding this comment

bwplotka left a comment

Choose a reason for hiding this comment

GiedriusS left a comment

Choose a reason for hiding this comment

utukJ commented Oct 13, 2022 •

edited