store-gateway: report touched postings & series instead of fetched #4671
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The store-gateway sends statistics about each request in its response to the querier. My understanding is that the purpose of these stats is to help gauge the cost of a query. Currently, the store-gateway reports its fetched bytes. I propose to report touched bytes instead. __Fetched index bytes__ are based on the number of bytes fetched from the bucket. This excludes bytes fetched from the cache and includes bytes overfetched because of an incorrect size estimation or when joining adjacent ranges from the index object. __Touched bytes__ are the sum of bytes that were directly necessary in order to serve the request. This means that they exclude bytes that we overfetched (e.g. when joining adjacent regions of an object in the object store or when incorrectly estimating the size of a series). The number of touched bytes should not generally change between query executions and will give a better picture of how expensive the query was and not depend on cache hit rates or the adjacency of requested regions in the index. Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
dimitarvdimitrov
force-pushed
the
dimitar/store-gateway-index-response-stats
branch
from
6c3601d
to
30e6b68
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does
The store-gateway sends statistics about the involved index bytes
in each request in its response to the querier. These statistics
are later combined in the query-frontend and included in
query statslogs.My understanding is that the purpose of these stats is to
help gauge the cost of a query. Currently, the store-gateway reports
its fetched bytes. I propose to report touched bytes instead.
Fetched index bytes are based on the number of bytes fetched from
the bucket. This excludes bytes fetched from the cache and includes bytes overfetched
because of an incorrect size estimation or when joining adjacent ranges
from the index object.
Touched bytes are the sum of bytes that were directly necessary in
order to serve the request. This means that they exclude bytes that we
overfetched (e.g. when joining adjacent regions of
an object in the object store or when incorrectly estimating the size of
a series).
The number of touched bytes should not generally change between query
executions and will give a better picture of how expensive the query was
and not depend on cache hit rates or the adjacency of requested regions
in the index.
Signed-off-by: Dimitar Dimitrov dimitar.dimitrov@grafana.com
Which issue(s) this PR fixes or relates to
Fixes #
Checklist
CHANGELOG.mdupdated - the order of entries should be[CHANGE],[FEATURE],[ENHANCEMENT],[BUGFIX]