Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip preloading of irrelevant postings #154

Closed
fabxc opened this issue Dec 20, 2017 · 0 comments
Closed

Skip preloading of irrelevant postings #154

fabxc opened this issue Dec 20, 2017 · 0 comments

Comments

@fabxc
Copy link
Collaborator

fabxc commented Dec 20, 2017

It just occurred to me that once we have blocks from many sources, it can rather easily happen that we preload a lot of irrelevant postings.

Suppose you get up{job="A"}. The up metric will be present in virtually every single block for the given time range. However, job="A" will probably be present in just a few. Currently if there's no postings list of job="A" for a block, we will return an EmptyPostings transparently and still register the postings list for __name__="up" for preloading for all blocks.

If we were more explicit about the non-existant postings, we could skip that work. That however would go pretty deep down into index fetching internals and we can only infer this by looking at the operations between postings (intersection vs. merge).

Probably not something to optimize quite right now yet. Especially since we've ongoing upstream work in that area.
But we should keep it in mind in case we run into performance issues later.

@fabxc fabxc closed this as completed Dec 27, 2017
rabenhorst added a commit to rabenhorst/thanos that referenced this issue May 4, 2023
* Replaced summary in extprom metrics with histogram

* Reverted newDefaultMetrics
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant