Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

osd: initial minimal efforts to clean up PG interface #17708

Merged
merged 72 commits into from
Oct 9, 2017
Merged

osd: initial minimal efforts to clean up PG interface #17708

merged 72 commits into from
Oct 9, 2017

Commits on Oct 6, 2017

  1. osd: use PG::get_pgid() instead of info.pgid 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    23cabca View commit details
    Browse the repository at this point in the history

  2. osd/PG: add get_last_user_version() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    487225d View commit details
    Browse the repository at this point in the history

  3. osd/PG: inline lock_suspend_timeout 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    c1893d2 View commit details
    Browse the repository at this point in the history

  4. osd/PG: add get_history() accessor 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    58684c5 View commit details
    Browse the repository at this point in the history

  5. osd/PG: add set_last_scrub_stamp 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    3ca232e View commit details
    Browse the repository at this point in the history

  6. osd/PG: get_acting() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    9ecd628 View commit details
    Browse the repository at this point in the history

  7. osd/PG: protect everything that doesn't have to be public 

    This lets us see what the *actual* public interface is that we are using.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    1 Configuration menu
    Copy the full SHA
    7c1cba8 View commit details
    Browse the repository at this point in the history

  8. osd/PG: clean up intrusive_ptr helpers 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    7ebf2a7 View commit details
    Browse the repository at this point in the history

  9. osd/PG: is_deleting() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    2bfaa8b View commit details
    Browse the repository at this point in the history

  10. osd/PG: start grouping public interface at top of class 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    a7e1a73 View commit details
    Browse the repository at this point in the history

  11. osd/PG: clean up set_force_{recovery,backfill} interface 

    - update state under pg lock
- clean up PG interface
- log only when we adjust the state

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    2482b4c View commit details
    Browse the repository at this point in the history

  12. osd/PG: move C_PG_ActivateCommitted into header 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    0591585 View commit details
    Browse the repository at this point in the history

  13. osd/PG: move C_PG_FinishRecovery into header 

    Less code than delcaring it in 2 places (by making it a friend).

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    ca4ba9b View commit details
    Browse the repository at this point in the history

  14. osd/PG: expose static methods for ceph-objectstore-tool 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    c3b300d View commit details
    Browse the repository at this point in the history

  15. osd/PG: collection more of public at top of class 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    931fc92 View commit details
    Browse the repository at this point in the history

  16. osd/ECBackend: use Listener instead of PG interface to check undersized 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    50a6f3c View commit details
    Browse the repository at this point in the history

  17. osd/PG: add is_forced_recovery_or_backfill() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    7abb015 View commit details
    Browse the repository at this point in the history

  18. osd/PG: protect pg state accessors 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    e70067c View commit details
    Browse the repository at this point in the history

  19. osd/PG: cleanup and move ref counting public bits 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    8921301 View commit details
    Browse the repository at this point in the history

  20. osd/PG: protect get_role() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    c4a9fdf View commit details
    Browse the repository at this point in the history

  21. osd/PG: protect get_last_peering_reset 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    49aeb04 View commit details
    Browse the repository at this point in the history

  22. osd/PG: reorder some members 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    9ab4eeb View commit details
    Browse the repository at this point in the history

  23. osd: use pg->pg_id const member, not get_pgid() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    329eb60 View commit details
    Browse the repository at this point in the history

  24. osd: use is_deleting() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    923ca75 View commit details
    Browse the repository at this point in the history

  25. osd/PG: make osr public 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    83a3513 View commit details
    Browse the repository at this point in the history

  26. osd: drop legacy infos object cleanup 

    This won't be there post-luminous

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    72d50e4 View commit details
    Browse the repository at this point in the history

  27. osd/PG: make get_watchers public, hide locking 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    5cef6ca View commit details
    Browse the repository at this point in the history

  28. osd/PG: add dump_pgstate_history() to public interface 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    7908142 View commit details
    Browse the repository at this point in the history

  29. osd/PG: add get_pg_stats() to public interface 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    385cda1 View commit details
    Browse the repository at this point in the history

  30. osd/PG: add get_num_ref() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    1b81002 View commit details
    Browse the repository at this point in the history

  31. osd/PG: on_shutdown -> shutdown in public interface 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    80233f2 View commit details
    Browse the repository at this point in the history

  32. osd/PG: make init() public 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    3b70a35 View commit details
    Browse the repository at this point in the history

  33. osd/PG: drop bufferlist passed from peek_map_epoch to read_state/read… 

    …_info

This is no longer used or needed.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    9dfa724 View commit details
    Browse the repository at this point in the history

  34. osd/PG: simplify/streamline upgrade process 

    There's no reason for this to be exposed to the OSD.  Just do it directly
in read_state().

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    48f4126 View commit details
    Browse the repository at this point in the history

  35. osd/PG: make read_state, peek_map_epoch public 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    662bb8f View commit details
    Browse the repository at this point in the history

  36. osd/PG: move load_pgs cruft into read_state() 

    This only happens during initial load; move it there.

Also, since update_store_with_options

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    342d681 View commit details
    Browse the repository at this point in the history

  37. osd: use PG::is_deleting() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    6022038 View commit details
    Browse the repository at this point in the history

  38. osd: remove build_past_intervals_parallel 

    This was added in the dumpling era to handle the case where we have lots
of PGs with stale or missing past_intervals.  These days, we update
PastIntervals as we inject new makes (via advance_map), when we
learn about new PGs via notify with get the PastIntervals at the same time,
and during PGCreate handling we have a separate path in
build_initial_pg_history.  This code is obsolete.

It also reaches deep into the PG class where it shouldn't!

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    ac9e2f7 View commit details
    Browse the repository at this point in the history

  39. osd/PG: hide peering_queue from OSD 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    43265b2 View commit details
    Browse the repository at this point in the history

  40. osd/PG: move some osd logic into handle_pg_trim 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    ea169c1 View commit details
    Browse the repository at this point in the history

  41. osd/PG: move more recovery logic into PG 

    I suspect we eventually want to move the create_context and
dispatch_context into OSDService (if it isn't there already) and move
even more of this logic into PG.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    6493f8e View commit details
    Browse the repository at this point in the history

  42. osd: bit less verbose while loading pgs 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    f542b20 View commit details
    Browse the repository at this point in the history

  43. osd/PG: get_need_up_thru(), get_same_interval_since() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    0438163 View commit details
    Browse the repository at this point in the history

  44. osd/PG: move write_if_dirty inside handle_{loaded,create} 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    fa7818a View commit details
    Browse the repository at this point in the history

  45. osd/PG: move some more split machinery into PG 

    and move other split pieces into the public interface.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    81f9372 View commit details
    Browse the repository at this point in the history

  46. osd/PG: move a few assertions, flag change into start_recovery_ops() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    b3b8119 View commit details
    Browse the repository at this point in the history

  47. osd/PG: drop unneeded write_if_dirty 

    Nothing in start_recovery_ops or stuck_on_unfound dirties state.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    82077e7 View commit details
    Browse the repository at this point in the history

  48. osd/PG: simplify unfound recovery case; rename method 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    e5405fe View commit details
    Browse the repository at this point in the history

  49. osd/PG: simplify start_recovery_ops return value meaning 

    Instead of complex check in caller, return simple bool indicating whether
find_unfound() should be called with a rctx.

There is one remaining condition that can probably be simplified:

  if (!recovering.empty() ||
      work_in_progress || recovery_ops_active > 0 || deferred_backfill)
    return !work_in_progress && have_unfound();

but we will leave it for now.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    7a4f30d View commit details
    Browse the repository at this point in the history

  50. osd: use accessors for pg resurrection 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    d36c468 View commit details
    Browse the repository at this point in the history

  51. osd/PG: define public interface for heartbeat peers 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    b8f33aa View commit details
    Browse the repository at this point in the history

  52. osd/PG: move bits of delete_dir (pg removal) into PG 

    We don't want to expose things like SnapMapper, OSDriver.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    79e7dbb View commit details
    Browse the repository at this point in the history

  53. osd/PG: change 'debug dump_missing' 

    - json
- move into PG class

No users in the qa suite.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    2a3a3fd View commit details
    Browse the repository at this point in the history

  54. osd/PGBackend: remove scrub_supported() 

    Everybody supports scrub.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    d14cc56 View commit details
    Browse the repository at this point in the history

  55. osd/PG: get_must_scrub() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    88c1330 View commit details
    Browse the repository at this point in the history

  56. osd/PG: make sched_scrub() public 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    24af955 View commit details
    Browse the repository at this point in the history

  57. osd: use pg_id.pool() for pg pool id 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    070181a View commit details
    Browse the repository at this point in the history

  58. osd/PG: add get_pool() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    17e6d40 View commit details
    Browse the repository at this point in the history

  59. osd/PG: make get_primary() public 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    c4616ee View commit details
    Browse the repository at this point in the history

  60. osd/PG: make on_removal() public 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    25769b2 View commit details
    Browse the repository at this point in the history

  61. osd/PG: make update_snap_mapper_bits public 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    7debd61 View commit details
    Browse the repository at this point in the history

  62. osd: drop redundant is_active() check before scrub_sched() 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    f6d3077 View commit details
    Browse the repository at this point in the history

  63. osd/PG: drop OSD friend, finally! 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    05d5206 View commit details
    Browse the repository at this point in the history

  64. osd: print summary for forced backfill/recovery to debug 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    7816ab5 View commit details
    Browse the repository at this point in the history

  65. ceph-objectstore-tool: print pg_info_t to stdoutt 

    Helpful for debugging

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    10a5a83 View commit details
    Browse the repository at this point in the history

  66. ceph-objectstore-tool: do not clear same_interval_since and PastInter… 

    …vals

We dropped the OSD recalculation code for this.  Import it.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    0f654d0 View commit details
    Browse the repository at this point in the history

  67. ceph-objectstore-tool: remove rm-past-intervals op 

    The OSD doesn't rebuild this on demand anymore.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    b6a5c09 View commit details
    Browse the repository at this point in the history

  68. ceph-objectstore-tool: refuse to import PG older that OSD's oldest_epoch 

    We don't have a way to construct a valid PastIntervals history.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    3681711 View commit details
    Browse the repository at this point in the history

  69. ceph-objectstore-tool: drop unused biginfo_oid 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    3131279 View commit details
    Browse the repository at this point in the history

  70. ceph-objectstore-tool: hide DIAGNOSTIC stuff behind debug/verbose mode 

    Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    366a3d5 View commit details
    Browse the repository at this point in the history

  71. ceph-objectstore-tool: bring past_intervals up to date on import 

    We can't blinding fast-forward the pg epoch to the latest map without
also updating the pg history and PastIntervals.  Do that at import time.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    2347ecb View commit details
    Browse the repository at this point in the history

  72. osd/PGLog: drop old compat coll/oid args to read_log_and_missing 

    The oldest version we care about is 10, and these were for <8.

Signed-off-by: Sage Weil <sage@redhat.com>
    @liewegas
    liewegas committed Oct 6, 2017
    Configuration menu
    Copy the full SHA
    582ee8d View commit details
    Browse the repository at this point in the history