Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Optimize storage.LabelQuerier.LabelValues implementations #525

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

aknuds1
Copy link
Contributor

@aknuds1 aknuds1 commented Aug 25, 2023

Try to make storage.LabelQuerier.LabelValues implementations faster by, in labelValuesWithMatchers, making PostingsForMatchers intersect matched series with series containing the label name (through a special new match type MatchSet). This eliminates the need to after the fact find series containing the label name and intersect with the PostingsForMatchers result.

I dropped the maxExpandedPostings optimization, since I don't see it being relevant after my changes, but please point out if I'm wrong.

Benchmark results are promising, geometric mean CPU time improvement looking to be ~59%, geometric mean memory usage reduction ~68%. I do see some of the benchmark cases looking much worse, I guess we'll have to look into those.

It doesn't look as if I've broken anything, considering the test suite passes, but please look carefully. I'm new to this code :)

Benchmark results for blocks (go test -bench=BenchmarkQuerier/Block/labelValuesWithMatchers -benchmem -run='^$' -timeout=30m -count 6 ./tsdb):
benchstat-blocks.txt

Benchmark results for heads (go test -bench=BenchmarkQuerier/Head [benchstat-head.txt](https://github.com/grafana/mimir-prometheus/files/12440956/benchstat-head.txt) /labelValuesWithMatchers -benchmem -run='^$' -timeout=30m -count 6 ./tsdb):
benchstat-head.txt

Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com>
@aknuds1 aknuds1 changed the title WIP: Optimize storage.LabelQuerier.LabelValues implementations Optimize storage.LabelQuerier.LabelValues implementations Aug 25, 2023
@aknuds1 aknuds1 marked this pull request as ready for review August 25, 2023 15:34
@aknuds1 aknuds1 requested a review from replay August 25, 2023 15:34
@aknuds1 aknuds1 added the enhancement New feature or request label Aug 25, 2023
@aknuds1
Copy link
Contributor Author

aknuds1 commented Aug 25, 2023

Converting to draft since the modified algorithm performs much worse for some cases.

@aknuds1 aknuds1 marked this pull request as draft August 25, 2023 15:57
@aknuds1 aknuds1 changed the title Optimize storage.LabelQuerier.LabelValues implementations WIP: Optimize storage.LabelQuerier.LabelValues implementations Aug 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant