-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: add post-mortem for avoid global sort and move it #7130
base: main
Are you sure you want to change the base?
Conversation
7fe2034
to
49e5d0f
Compare
Long overdue TODO task for me - adding a post-mortem for the avoid global sort proposal and move it to the "proposals-done" directory. Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
49e5d0f
to
5b6a3f0
Compare
|
||
Theoretically, we could address this in the distant future by modifying the ordering rules within the TSDB, but I'm uncertain if that will ever materialize. | ||
|
||
Nevertheless, despite the initial challenges faced in implementation and subsequent reverting, we have made significant improvements to the querying internals. For example, we now use a ProxyHeap in a multitude of components, rather than repeating the same logic across each individual component. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean we are still avoiding global sort by stripping out replica labels on stores, we just have to perform resorting in stores afterwards, but the sorting is now not done in global context but on the store APIs right? So its not full revert I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mhm, why do we https://github.com/thanos-io/thanos/blob/main/pkg/store/proxy_heap.go#L757-L761 force resorting here? Shouldn't we check whether the server supports without replica labels
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is a good question, stores should always return sorted responses.
Edit: Ah, I added that, its because this is used for block clients in bucket store. We remove labels in the block clients and then put them onto a proxy heap; if we wouldnt resort there we would have the same issues we have in querier in the bucket store because it acts like a proxy heap for blocks. So we need to make them return sorted responses after removal of the external labels. We cannot rely on sorted responses there. Maybe it would have been more elegant to do the sorting in the block series client because technically it is the one violating the sorting constraint for Store APIs I think.
Long overdue TODO task for me - adding a post-mortem for the avoid global sort proposal and move it to the "proposals-done" directory.