-
Notifications
You must be signed in to change notification settings - Fork 11.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ROCKETMQ-67] Consistent Hash allocate strategy #67
Conversation
2 similar comments
package org.apache.rocketmq.common.consistenthash; | ||
|
||
/** | ||
* Created by jaskeylin on 2017/2/20. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, please remove the author info.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry , i didn'it notice that. Just a minute
2 similar comments
2 similar comments
2 similar comments
@zhouxinyu @lizhanhui @shroman @vongosling Since I think consistent hash can be also applied to order message sharding, the classes in this pr can be reused. I will submit a new pr for that sharding message queue selector after this pr is merged : https://issues.apache.org/jira/browse/ROCKETMQ-136?jql=project%20%3D%20ROCKETMQ |
@Jaskey Can you explain more about this feature? Having a better description on JIRA would help a lot review this PR -- especially for new functionalities. Since the description is very general, I have the following questions. What is the advantage of introducing this strategy? |
This is feature to give choice to users who care more about latency stabilization and messages duplication. As you know, the default And that will cause
This is especially significant when they have tens of consumer instances and scale-up or deployment is often. Consistent Hash strategy to allocate queue is a good choice for these users. |
@Jaskey This is a very good description, and now reviewers will understand well your intentions :) If you clearly state your intentions in JIRA, or discuss in the ml, it saves much time, and good for the record. Sorry, it's not a code review yet, just wanted understand your intentions. |
@shroman |
@shroman @zhouxinyu @lizhanhui @vongosling I heard that consistent hash strategy will be accomplished in 4.1.x , while this pr has been open for more than a month, can we accelerate it, would you please help review the implementations? |
for (int i = 0; i < 4; i++) { | ||
h <<= 8; | ||
h |= ((int) digest[i]) & 0xFF; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
digest is 128 bits, but the generated h only use 32 bits, other 96 bits are ignored, will it be better use ^ method
JIRA: https://issues.apache.org/jira/browse/ROCKETMQ-67
Consitent hash alogrithm to allocate message queue to prevent "thundering herd".
Core implementation:
Define a ConsitentHashRouter to hash clientId to a hash ring. And try to find the nearest clientId for every message queue.
High scalability considered, there are several new class to support as many scenarios as possible when user needs to use consitent hash
Node
interface.