Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

crush: add devices class that rules can use as a filter #13444

Merged
18 commits merged into from Mar 8, 2017
Merged

crush: add devices class that rules can use as a filter #13444

18 commits merged into from Mar 8, 2017

Commits on Mar 1, 2017

  1. crush: parse "class XXX" after device 

    Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    eb5a7c3 View commit details
    Browse the repository at this point in the history

  2. crush: parse "class XXX" after step take 

    Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    6c9cb09 View commit details
    Browse the repository at this point in the history

  3. crush: parse "class XXX" after bucket id 

    Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    09e26cb View commit details
    Browse the repository at this point in the history

  4. crush: item_exists is const 

    Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    0ca0b24 View commit details
    Browse the repository at this point in the history

  5. crush: _bucket_is_in_use does not use cct 

    Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    3d51246 View commit details
    Browse the repository at this point in the history

  6. crush: emacs comment is obsolete 

    Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    df33e70 View commit details
    Browse the repository at this point in the history

  7. crush: implement remove_unused_root 

    Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    93c0e10 View commit details
    Browse the repository at this point in the history

  8. crush: get_item_weight must not fail on root 

    Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    fa27369 View commit details
    Browse the repository at this point in the history

  9. crush: implement device_class_clone 

    Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    b3c50b1 View commit details
    Browse the repository at this point in the history

  10. crush: implement split_id_class 

    Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    09e4d40 View commit details
    Browse the repository at this point in the history

  11. crush: implement trim_roots_with_class 

    Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    c68c0c6 View commit details
    Browse the repository at this point in the history

  12. crush: a bucket is also busy if a derived bucket uses it 

    The step take of a rule may reference a bucket that is cloned
from the original one, with the device class added. In this
case both buckets are busy because they are taken by the rule.

Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    ad51404 View commit details
    Browse the repository at this point in the history

  13. crush: implement {populate,cleanup}_classes 

    Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    9cf9d81 View commit details
    Browse the repository at this point in the history

  14. crush: do not erase non existent name 

    Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    fb111a8 View commit details
    Browse the repository at this point in the history

  15. crush: clear bucket class ids when removed 

    Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    57e66c6 View commit details
    Browse the repository at this point in the history

  16. crush: update device classes where relevant 

    The device classes are implemented by modifying:

- the argument of step TAKE in rules
- cloning bucket trees when required by a rule step

This happens (via populate_classes):

- before compiling a rule step TAKE

When the crush map is encoded, the device class information is stored
with it, independently from the rules and the buckets, as a map of
classes for each device & bucket and a map of classes for each rule step
TAKE.

The extra buckets created but not used by any rule do not need to be
preserved and they are removed (via cleanup_classes):

- before decompilation
- after compilation
- after decoding

The client and daemons that are not aware of the device classes are
compatible because the crushmap modified with the new buckets is fully
functional. The invalid names used in the for the generated
buckets (bucket~class) can be CrushWrapper::decode by any existing
client because there is no verification of the name validity during
decoding. It can also be CrushWrapper::dump or CrushCompiler::decompile
via ceph osd dump or crushtool. It cannot, however, be compiled again
because CrushCompiler::compile will try to set the name with
CrushWrapper::set_item_name and it will fail with EINVAL because of the
~.

Fixes: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    f9eb206 View commit details
    Browse the repository at this point in the history

  17. crush: document the class qualifier 

    The class qualifier is documented for devices and step take
only. Although it shows in the buckets and could be set by the user, it
would be very error prone to do so.

Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    154f3d3 View commit details
    Browse the repository at this point in the history

  18. crush: device classes mini cluster sanity check 

    Run a mini cluster and verify that modifying the crushmap,
sending it to the cluster produces the intended effect.

Refs: http://tracker.ceph.com/issues/18943

Signed-off-by: Loic Dachary <ldachary@redhat.com>
    @ldachary
    ldachary committed Mar 1, 2017
    Copy the full SHA
    c7311e1 View commit details
    Browse the repository at this point in the history