Closed
Conversation
ghost
assigned
src/riak_core_claim.erl
Outdated
Contributor
There was a problem hiding this comment.
In the riak_core.app file, and in default_choose_params/1 this is target_n_val is 4 (
riak_core/src/riak_core_claim.erl
Line 247 in a73fcfd
Contributor
Author
There was a problem hiding this comment.
We should probably set at 4 until we have a filter on the fallback preflists to ensure node uniqueness as long as possible.
Contributor
|
A few comments about comments and surprising debug output above. Tested using the provided eqc tests, and a bit of prodding by hand. Wondering if we want to add metrics for time to claim, and maybe the claim "metrics" (diversified / diagonal / violations etc)? All works for me +1 to merge. |
added 7 commits
June 9, 2012 17:20
Claim V3 - unlike the v1/v2 algorithms, v3 treats claim as an optimization problem. In it's current form it creates a number of possible claim plans and evaluates them for violations, balance and diversity, choosing the 'best' plan. Violations are a count of how many partitions owned by the same node are within target-n of one another. Lower is better, 0 is desired if at all possible. Balance is a measure of the number of partitions owned versus the number of partitions wanted. Want is supplied to the algorithm by the caller as a list of node/counts. The score for deviation is the RMS of the difference between what the node wanted and what it has. Lower is better, 0 if all wants are mets. Diversity measures how often nodes are close to one another in the preference list. The more diverse (spread of distances apart), the more evenly the responsibility for a failed node is spread across the cluster. Diversity is calculated by working out the count of each distance for each node pair (currently distances are limited up to target N) and computing the RMS on that. Lower diversity score is better, 0 if nodes are perfectly diverse.
diversify is still used if claimv3 cannot find a plan without violations. Redudece lager:debug output and made it more readable.
metadata changed by the claim function is not persisted to the raw ring. Also added a force_claim environment variable to trigger a one time reclaim.
Contributor
Author
|
rebased and merged |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Claim V3 - unlike the v1/v2 algorithms, v3 treats claim as an optimization problem. In it's current form it creates a number of possible claim plans and evaluates them for violations, balance and diversity, choosing the 'best' plan.
Violations are a count of how many partitions owned by the same node are within target-n of one another. Lower is better, 0 is desired if at all possible.
Balance is a measure of the number of partitions owned versus the number of partitions wanted. Want is supplied to the algorithm by the caller as a list of node/counts. The score for deviation is the RMS of the difference between what the node wanted and what it has. Lower is better, 0 if all wants are mets.
Diversity measures how often nodes are close to one another in the preference list. The more diverse (spread of distances apart), the more evenly the responsibility for a failed node is spread across the cluster. Diversity is calculated by working out the count of each distance for each node pair
(currently distances are limited up to target N) and computing the RMS on that. Lower diversity score is better, 0 if nodes are perfectly diverse.
For testing and playing around, you may want to look at running riak_core_claim_sim:commission(). which will run comparisons against the v1/v2 claim algorithm for some preset combinations of ring size, number of nodes, and target n.
You could also play with riak_core_claim_sim:print_failure_analysis(Ring, TargetN, NumFailures) which prints reports on the console for how a ring does given a target N and number of nodes to fail.
This code is based on the jdm-claim-sim branch - conflicts with the cluster management changes will need to be resolved between the three branches before we can merge.