Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DNM]reef: rgw: sync fairness bidding #52550

Open
wants to merge 8 commits into
base: reef
Choose a base branch
from
Open

[DNM]reef: rgw: sync fairness bidding #52550

wants to merge 8 commits into from

Commits on Jul 19, 2023

  1. rgw: add BidManager for sync fairness via watch/notify 

    Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 40c0179)
    @cbodley
    cbodley authored and Shilpa Jagannath committed Jul 19, 2023
    Copy the full SHA
    6a7c3c0 View commit details
    Browse the repository at this point in the history

  2. rgw/sync: add a BidManager to metadata sync 

    Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 43e95e6)
    @cbodley
    cbodley authored and Shilpa Jagannath committed Jul 19, 2023
    Copy the full SHA
    7707bf1 View commit details
    Browse the repository at this point in the history

  3. rgw: multisite metadata sync fairness 

        multisite metadata sync fairness

    The approach of this commit is to allow multiple RGWs to participate in the multisite metadata sync.
    Before this commit only single RGW has caught the all the sync locks.
    This feature is using bidding algorithm.
    For each lock, RGW is randomizing a number from 0 to shard count and for each shard is picking randomally one number and giving it as the bid_amount.
    each one of those vectors each RGW handles are being sent using watch notify (based on RADOS watch notify).
    Each time the RGW tries to lock it will compare its bid for the lock and the bids of other rgws, if the current RGW has the highest bid it will try to acquire the lock.

    Important configs:

    rgw_sync_work_period - For how long the RGW will sync until it will send unlock (very important in the beggining, because in the beginning only single RGW holds the locks)
    rgw_sync_lease_period - not new to this commit but affecting it, For how many seconds the RGW will request from the RADOS to keep the lock, mainly important in case of failure, so automatically the RGW will lose a lock if it's down

Fixes: https://tracker.ceph.com/issues/41230
Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
Signed-off-by: Or Friedmann <ofriedma@ibm.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
(cherry picked from commit 38fa043)
    Or Friedmann authored and Shilpa Jagannath committed Jul 19, 2023
    Copy the full SHA
    0807088 View commit details
    Browse the repository at this point in the history

  4. rgw/multisite: add a timer coroutine to call notify_cr() at intervals 

    Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 6c99d33)
    Shilpa Jagannath committed Jul 19, 2023
    Copy the full SHA
    44c962f View commit details
    Browse the repository at this point in the history

  5. rgw/multisite: periodically check if we still have the highest bid. 

    If we are outbid, exit after updating sync status.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 8ee42ab)
    Shilpa Jagannath committed Jul 19, 2023
    Copy the full SHA
    70cddde View commit details
    Browse the repository at this point in the history

  6. rgw/multisite: clean up 

    Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 31f129f)
    Shilpa Jagannath committed Jul 19, 2023
    Copy the full SHA
    7e718d0 View commit details
    Browse the repository at this point in the history

  7. rgw/multisite: - add a lost_bid variable to handle is_highest_bidder(… 

    …) control flow

- check for is_highest_bidder() before even attempting to take the lock
- don't block on RGWMetaSyncShardNotifyCR()
- other minor fixes

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 14c4713)
    Shilpa Jagannath committed Jul 19, 2023
    Copy the full SHA
    1f389fc View commit details
    Browse the repository at this point in the history

  8. rgw/multisite: use intrusive_ptr instead of raw ptr for notify_stack 

    Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
(cherry picked from commit 410246f)
    Shilpa Jagannath committed Jul 19, 2023
    Copy the full SHA
    ad88f0c View commit details
    Browse the repository at this point in the history