streaming store-gateway: add tests for index cache with query sharding #3728
Conversation
Adds tests for when running the streaming implementation wth query sharding and index cache. This is a followup of PR 3687 Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
pkg/storegateway/series_refs.go
Outdated
| type cachedSeriesHasher struct { | ||
| cache *hashcache.BlockSeriesHashCache | ||
| } | ||
|
|
||
| func (b cachedSeriesHasher) Fetch(seriesID storage.SeriesRef) (uint64, bool) { | ||
| return b.cache.Fetch(seriesID) |
There was a problem hiding this comment.
I just realised we were/are not tracking stats.seriesHashCacheRequests and stats.seriesHashCacheHits in filterPostingsByCachedShardHash(), but I think we should.
There was a problem hiding this comment.
we are. here
are we talking about the same place 🤔
There was a problem hiding this comment.
and now i've changed it in 7453753 so that the stats are passed as a pointer to filterPostingsByCachedShardHash and it passes it down to cachedSeriesHasher, which in turn updates the stats
pkg/storegateway/bucket.go
Outdated
| blockSeriesHashCache, | ||
| cachedSeriesHasher{blockSeriesHashCache}, |
There was a problem hiding this comment.
Why we need both? Can we only pass one of them?
There was a problem hiding this comment.
filterPostingsByCachedShardHash can only look at the cache. I kept them as two so that I don't change the default implementation.
This commit makes them one 7453753 and changes the default implementaiton
…iesCaching Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
This reverts commit fd30b11.
Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
dimitarvdimitrov
force-pushed
the
dimitar/streaming-index-cache-tests
branch
from
16f75cf
to
7453753
Compare
grafana#3728) * streaming store-gateway: add tests for index cache with query sharding Adds tests for when running the streaming implementation wth query sharding and index cache. This is a followup of PR 3687 Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Remove unnecessary short circuit Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Remove unnecessary seriesHashCache interface Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Make a note for scope of TestOpenBlockSeriesChunkRefsSetsIterator_SeriesCaching Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> * Revert "Remove unnecessary seriesHashCache interface" This reverts commit fd30b11. * Pass a single struct to calculate and cache hashes of series Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com> Signed-off-by: Dimitar Dimitrov <dimitar.dimitrov@grafana.com>
Adds tests for when running the streaming implementation with query
sharding and index cache. This is a follow-up of #3687
This PR also includes some changes to interfaces so that it's easier to mock the hash of a series.
Signed-off-by: Dimitar Dimitrov dimitar.dimitrov@grafana.com