New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crush: update choose_args when items are added/removed #15311

Merged
merged 4 commits into from May 31, 2017

Conversation

Projects
None yet
2 participants
@ghost

ghost commented May 26, 2017

@ghost ghost added core feature labels May 26, 2017

@ghost ghost requested a review from liewegas May 26, 2017

@ghost ghost changed the title from crush: update weight_set when items are added/removed to DNM: crush: update weight_set when items are added/removed May 26, 2017

@ghost

This comment has been minimized.

ghost commented May 26, 2017

will work on move/reweight now

ldachary added some commits May 26, 2017

crush: wrap crush_bucket_{remove,add}_item
Signed-off-by: Loic Dachary <loic@dachary.org>
crush: allow item addition/removal when choose_args exists
Signed-off-by: Loic Dachary <loic@dachary.org>

@ghost ghost changed the title from DNM: crush: update weight_set when items are added/removed to DNM: crush: update choose_args when items are added/removed May 26, 2017

@ghost ghost changed the title from DNM: crush: update choose_args when items are added/removed to crush: update choose_args when items are added/removed May 26, 2017

@ghost

This comment has been minimized.

ghost commented May 26, 2017

@liewegas does that sound reasonable ? I think it addresses what we discussed today.

crush_weight_set *weight_set = &arg->weight_set[j];
weight_set->weights = (__u32*)realloc(weight_set->weights, new_size * sizeof(__u32));
assert(weight_set->size + 1 == new_size);
weight_set->weights[weight_set->size] = weight;

This comment has been minimized.

@liewegas

liewegas May 30, 2017

Member

the commit message needs updating? this is setting it to weight arg, not 0

This comment has been minimized.

@ghost
crush_choose_arg *arg = &arg_map.args[-1-bucket->id];
for (__u32 j = 0; j < arg->weight_set_size; j++) {
crush_weight_set *weight_set = &arg->weight_set[j];
weight_set->weights[position] = weight;

This comment has been minimized.

@liewegas

liewegas May 30, 2017

Member

it might be worth doing the proportional thing here: calculate float ratio = weight / old_weight, and use that ratio to adjust this value.

This comment has been minimized.

@ghost

ghost May 30, 2017

The ratio idea is probably correct when the weight of an item is adjusted for the first weight set in a bucket. But the ratio is going to be different for the second (and successive) weight set. We know from experience that the second weight set is going to lower the weight of the items that have the highest target weight and vice versa. It follows that if a target weight changes and becomes higher than another, the ratio needs to be inverted. That's why I think better to just stick with the target weight until rebalancing gets a chance to run.

Am I making things more complicated than they should be ?

if (choose_args.size() > 0) {
ldout(cct, 1) << "swap_bucket not implemented when choose_args is not empty" << dendl;
return -EDOM;
}

This comment has been minimized.

@liewegas

liewegas May 30, 2017

Member

i think we can actually support this without much trouble. we're completely swapping the contents, so we can just swap the choose_args for the buckets too. and then call adjust for the bucket itself at the end to account for any net bucket weight change.

This comment has been minimized.

@ghost
@liewegas

This comment has been minimized.

Member

liewegas commented May 30, 2017

what's here so far looks good!

@liewegas

This comment has been minimized.

Member

liewegas commented May 30, 2017

ldachary added some commits May 26, 2017

crush: update choose_args when items are added/removed
When items are added:

- the value in the weight set is set to the target weight. It is assumed
  that weight set are updated on a regular basis and will eventually be
  set to a value that prevents excessive over/under filling.

- the value in the id list is set to the item id.

When items are removed, their weight / ids in all choose_args are
removed.

Signed-off-by: Loic Dachary <loic@dachary.org>
crush: update choose_args on adjust_item_weight
The osd_crush_update_weight_set (true by default) can be used to disable
the update of the weights.

Signed-off-by: Loic Dachary <loic@dachary.org>
@ghost

This comment has been minimized.

ghost commented May 31, 2017

@liewegas the commit message was updated, swap_bucket does not need modification (it already updates the parents weights and that will also update their weight set) and I kept the target weight instead of trying to apply the ratio. Did I miss anything ?

@liewegas

This comment has been minimized.

Member

liewegas commented May 31, 2017

@liewegas liewegas merged commit bf82eb5 into ceph:master May 31, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment