Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Boltdb shipper query performance improvements #2770

Merged

Conversation

sandeepsukhani
Copy link
Contributor

What this PR does / why we need it:
This PR includes a couple of improvements, all in a separate commit:

  1. Uses new QueryWithCursor method which can reuse the same cursor to do all the index queries instead of having to start a new transaction for each query.
  2. A new type called IndexDeduper which acts as a middleware for deduping the index entries which significantly improves the CPU usage and query performance.
  3. Use Fifocache(an in-memory cache) in ingester which also significantly reduces the load on ingesters by caching the index in memory for serving subsequent queries. It also helps improve the query performance.
  4. Snapshotting of dbs in ingesters for reads. This is again an optimization to improve query performance in ingesters. It would create a new snapshot of dbs in ingesters every minute for reads.
  5. Ingesters now ship the files faster and retain fewer of them which reduces the amount of index ingesters have to deal with during queries. This also helps reduce the load on ingesters during queries.

All the changes complement each other to enhance query performance and reduce CPU usage. I have also fine-tuned some configs which have a supporting comment to explain why the change has been done.

Checklist

  • Tests updated

@sandeepsukhani sandeepsukhani changed the title Boltdb shipper query improvements Boltdb shipper query performance improvements Oct 16, 2020
@sandeepsukhani sandeepsukhani force-pushed the boltdb-shipper-query-improvements branch from 0151424 to ce6f93b Compare October 16, 2020 10:34
Copy link
Collaborator

@slim-bean slim-bean left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@slim-bean slim-bean merged commit f60554c into grafana:master Oct 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants