Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mgr/balancer: mgr module to automatically balance PGs across OSDs #16272

Merged
merged 15 commits into from Sep 8, 2017
Merged

mgr/balancer: mgr module to automatically balance PGs across OSDs #16272

merged 15 commits into from Sep 8, 2017

Commits on Sep 6, 2017

  1. mgr/PyModules: add 'pg_status' dump 

    This is summary info, same as what's in 'ceph status'.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    85b5b80 View commit details
    Browse the repository at this point in the history

  2. mgr/PyModules: add 'pg_dump' get 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    bfb9286 View commit details
    Browse the repository at this point in the history

  3. mgr: add trivial OSDMap wrapper class 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    2ef0051 View commit details
    Browse the repository at this point in the history

  4. pybind/mgr/mgr_module: add default arg to get_config 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    39c42dd View commit details
    Browse the repository at this point in the history

  5. pybind/mgr/balancer: add balancer module 

    - wake up every minute
- back off when unknown, inactive, degraded
- throttle against misplaced ratio
- apply some optimization step
  - initially implement 'upmap' only

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    0d9685c View commit details
    Browse the repository at this point in the history

  6. pybind/mgr/balancer: do upmap by pool, in random order 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    028a66d View commit details
    Browse the repository at this point in the history

  7. crush/CrushWrapper: refactor get_rule_weight_osd_map to work with roo… 

    …ts too

Allow us to specify a root node in the hierarchy instead of a rule.
This way we can use it in conjunction with find_takes().

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    69454e0 View commit details
    Browse the repository at this point in the history

  8. crush/CrushWrapper: fix output arg for find_{takes,roots}() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    60b9cfa View commit details
    Browse the repository at this point in the history

  9. crush/CrushWrapper: rule_has_take 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    ef140de View commit details
    Browse the repository at this point in the history

  10. mgr/PyOSDMap: get_crush, find_takes, get_take_weight_osd_map 

    These let us identify distinct CRUSH hierarchies that rules distribute
data over, and create relative weight maps for the OSDs they map to.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    3b8a276 View commit details
    Browse the repository at this point in the history

  11. mgr/PyOSDMap: OSDMap.map_pool_pgs_up, CRUSHMap.get_item_name 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    a928bf6 View commit details
    Browse the repository at this point in the history

  12. pybind/mgr/balancer: rough framework 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    d5e5c68 View commit details
    Browse the repository at this point in the history

  13. pybind/mgr/balancer: make 'crush-compat' sort of work 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    7a00e02 View commit details
    Browse the repository at this point in the history

  14. src/pybind/mgr/balancer/module.py: improve scoring method 

    * score lies in [0, 1), 0 being perfect distribution
* use shifted and scaled cdf of normal distribution
  to prioritize highly over-weighted device.
* consider only over-weighted devices to calculate score

Signed-off-by: Spandan Kumar Sahu <spandankumarsahu@gmail.com>
    @SpandanKumarSahu @liewegas
    SpandanKumarSahu authored and liewegas committed Sep 6, 2017
    Copy the full SHA
    c09308c View commit details
    Browse the repository at this point in the history

  15. pybind/mgr/balancer: make auto mode work 

    (with upmap at least)

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Sep 6, 2017
    Copy the full SHA
    ef1a3be View commit details
    Browse the repository at this point in the history