Docs: add initial descriptions of Loki use of hash rings #4900
Conversation
trevorwhitney
left a comment
trevorwhitney
left a comment
There was a problem hiding this comment.
LGTM! I added some suggestions for what the query-scheduler and compactor rings do, though might want @slim-bean to confirm what I said about the query-scheduler.
There was a problem hiding this comment.
The query schedulers use the information in their key-value store to allow enable "service discovery" of the schedulers. This allows query workers to connect to all available schedulers, and scheduler to connect to all available query frontends, effectively creating a single queue that query load can be sharded across.
There was a problem hiding this comment.
Compactors use the information in the key-value store to identify a single instance that will be responsible for compaction, and the compactor is only enabled on that instance, despite the target being enabled on multiple instances.
sandeepsukhani
left a comment
sandeepsukhani
left a comment
There was a problem hiding this comment.
Added some minor suggestions but other than that it looks good to me.
There was a problem hiding this comment.
The png is not referenced correctly
|  | |
|  |
There was a problem hiding this comment.
@sandeepsukhani : this image callout is correct as is.
There was a problem hiding this comment.
| Each node has a key-value store that holds communication information | |
| Each node has a key-value pair in the store that holds communication information |
There was a problem hiding this comment.
I'm leaving this as is. My prose works top down, and it states that each node in the ring has a store. It is also true that each node is represented as a key-value pair within the store, but that isn't what I wished to convey.
There was a problem hiding this comment.
This looks out of the context because gossip(memberlist) is one of the supported stores for the ring. You can maybe say each node periodically keeps updating its info in the store at the configured heartbeat interval.
There was a problem hiding this comment.
I don't understand your comment. To me, there are 2 different issues. Issue 1: which store is used. Issue 2: the gossip protocol is used to propagate info.
gossip(memberlist) is a particular implementation detail and not about how the rings work (in general).
Are you telling me that a different algorithm is used to propagate info if a store type other than memberlist is used? If yes, then I will change the prose.
My prose is there to explain to readers the basics of how the ring works. I'm trying to do this independent of which configuration/implementation they choose for their key-value store.
There was a problem hiding this comment.
Yes, I think not all available stores use the gossip protocol (ex: etcd and inmemory).
KMiller-Grafana
force-pushed
the
docs/document-rings
branch
from
814fabb to
41ea2c7
Compare
5 participants
This new reference info on consistent hash rings is incomplete. Please use comments to suggest the missing pieces and identify incorrect information.