-
Notifications
You must be signed in to change notification settings - Fork 151
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consistent hashing is different than Python #335
Comments
the issue where that python code is introduced is graphite-project/carbon#196 (comment) |
Here I'm showing there is a drift between the current implementation and Python one making the two implementations incompatible. |
but in your PR which fixes this issue, you claim to introduce the equivalent for graphite-project/carbon@024f9e6#diff-1486787206e06af358b8d935577e76f5 which is the solution to graphite-project/carbon#196 so how it is possible that we're not aiming to solve graphite-project/carbon#196 here ? |
Because we aim to solve #335 does not mean we aim to solve graphite-project/carbon#196. It may do, but that's a non goal. I never claimed the issues are the same. I don't know why graphite-project/carbon#196 appeared in this conversation. I just pointed the commit on The issue we are discussing here is incompatible algorithm between |
I have multiple carbon-caches receiving metrics from a relay-ng route in consitentHashing mode. Thanks to the parameter CARBONLINK_HOSTS, the Graphite webapp may query the caches for data that has not yet been persisted. |
so @hamelg you can independently confirm that the behavior from this patch is the right one? i will bump this on my priority list then. |
my bad ! |
so @hamelg to be clear you're saying then the code in master works fine and we should not merge this PR? |
If that can help here is some code that illustrate the issue: |
Finally, I have succeeded in reproducing the issue with the specific setup provided by vmiszczak-teads. carbon-relay-ng.conf
carbonate.conf
Without his patch, the hashing result is different on these nodes : After applying the patch, carbonate and relay-ng give the same result. |
I found out that Carbon in Python has different behavior than this project
carbon_ch
.This is a severe issue because a tool like
carbonate
does not sees the same world thancarbon-relay-ng
. I almost messed a 5TB cluster wanting to purge metrics after a rebalance.Using server list :
gives ring positions including:
Python gives:
Notice
(59418, ('10.20.34.131', None))
exists in Python and not in this implementation.Having a metric that hashes to
59418
gives destination10.20.40.126
with carbon-relay-ng and10.20.34.131
with Python/Carbonate.If you filter out metrics with
carbon-sieve
to delete after a rebalance, you may delete metrics that you should not.The text was updated successfully, but these errors were encountered: