feat: optional per-query duration and row count metrics#996
Merged
burningalchemist merged 1 commit intoMay 12, 2026
Merged
Conversation
Adds an opt-in `global.enable_query_metrics` flag. When enabled, every
query emits two extra gauges per scrape:
- `query_duration_seconds{query=...}` — wall-clock time of the last run
- `query_rows_returned{query=...}` — number of rows the database returned
Both inherit the target's constant labels, so they aggregate the same
way as `up` and `scrape_duration_seconds`. The feature is off by default
to keep the existing metric surface unchanged.
Owner
|
Hi @rassie, thank you for your contribution! 👍 I'll have a look at the PR this week and get back to you. Overall, sounds good! 👍 |
burningalchemist
approved these changes
May 12, 2026
Owner
burningalchemist
left a comment
burningalchemist
left a comment
There was a problem hiding this comment.
Looks good to me! 🚀 Thanks again! 👍
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds an opt-in
global.enable_query_metricsflag. When set, every configured query emits two extra gauges per scrape:query_duration_seconds{query="<name>"}— wall-clock time of the query during the last scrape, including row scanning. Emitted even when the query errors so spikes around failures stay visible.query_rows_returned{query="<name>"}— number of rows the database returned during the last scrape.Both inherit the same constant labels as
upandscrape_duration_seconds(notablytargetin jobs mode).Motivation
Issue #303 added debug logging for per-query timing; this exposes the same information as first-class metrics so slow queries can be identified directly from PromQL.
Notes
ENABLE_QUERY_METRICS=true.quantile_over_timeworks the same in practice (caveat: biased withmin_interval > 0).Disclosure: this change was prepared with the help of an LLM-based coding assistant. All design choices were reviewed and the diff was tested locally before submission.