Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mds: multiple mds scrub support #35749

Merged
merged 19 commits into from Nov 17, 2020
Merged

mds: multiple mds scrub support #35749

merged 19 commits into from Nov 17, 2020

Commits on Nov 16, 2020

  1. mds: remove ScrubStack::scrubstack 

    comment says it's hack for dout. I don't see any reason it's needed.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    afb2e12 View commit details
    Browse the repository at this point in the history

  2. mds: make both CInode and CDir as entities of scrub 

    Making CDir as entity of scrub is preparetion for scrubbing across
multiple mds. When subtree bound is encountered, scrub should be
forwarded to subtree's auth mds. The auth mds adds CDir to scrub stack.

Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    547c1ed View commit details
    Browse the repository at this point in the history

  3. mds: change scrub traverse from post-order to breadth-first search 

    After using breadth-first search, scrubing a dir inode does not need
to wait until all of its descendant difrags/inodes are scrubbed. This
simplfies scrub code a lot. The downside is that a scrubbed dir inode
no longer implies corresponding subtree has been fully scrubbed. It
makes later scrub (without force option) less efficient.

Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @simon-rock @ukernel
    simon-rock authored and ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    b43af15 View commit details
    Browse the repository at this point in the history

  4. mds: prevent dirfrag scrub/fragment from running at the same time 

    When a CDir is in scrub stack, mds should not split/merge it.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    a4c9bfc View commit details
    Browse the repository at this point in the history

  5. mds: remove object can't be scrubbed immediately from scrub stack 

    This avoid checking objects that can't be scrubbed repeatly.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    38c6fc8 View commit details
    Browse the repository at this point in the history

  6. include/frag: add encode/decode functions for fragset_t 

    Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    458e7d6 View commit details
    Browse the repository at this point in the history

  7. mds: multiple mds scrub support 

    If a non-auth object is encountered during scrubbing, forward scrub
to the object's auth mds.

Fixes: https://tracker.ceph.com/issues/12274
Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @simon-rock @ukernel
    simon-rock authored and ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    e5c4e8c View commit details
    Browse the repository at this point in the history

  8. mds: rdlock file/nest lock when accumulating stats of subtree dirfrags 

    Signed-off-by: Simon Gao <simon29rock@gmail.com>
Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @simon-rock @ukernel
    simon-rock authored and ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    c8c3ba4 View commit details
    Browse the repository at this point in the history

  9. mds: auth pin CInode when validating its disk state 

    Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    f01ebad View commit details
    Browse the repository at this point in the history

  10. Continuation: don't delete self while there are in-processing stages 

    Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    86fb5b4 View commit details
    Browse the repository at this point in the history

  11. mds: remove on_finish from {CInode,CDir}::scrub_info_t 

    A CInode/CDir is scrubbed no longer means corresponding subtree is fully
scrubbed. The on_finish in {CInode,CDir}::scrub_info_t become useless.
This patch also removes code that flushs journal if scrub has repaired
anything.  Later patch will add the code back at different place.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    03908aa View commit details
    Browse the repository at this point in the history

  12. mds: track scrub status in multiple mds 

    Scrubs are always initialized from mds.0. So mds.0 can ensure that scrub
tags are unique globally. mds.0 periodically gathers scrubs running in
itself and in other mds. A scrub is finished only if it's not running in
any mds.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    ad5471f View commit details
    Browse the repository at this point in the history

  13. mds: abort/pause/resume scrubs in multiple mds 

    Limit scrub abort/pause/resume commands to mds.0. mds.0 sends messages
to other mds, asks them to abort/pause/resume scrubs.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    ff10bdb View commit details
    Browse the repository at this point in the history

  14. mds: don't skip validating disk state of symlink 

    we can check if backtrace is valid and check if inode number is in-use

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    fbac6bc View commit details
    Browse the repository at this point in the history

  15. qa/cephfs: update existing scrub test cases 

    Now scrub is always async.

Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    31deca1 View commit details
    Browse the repository at this point in the history

  16. qa/cephfs: add tests for multimds scrub 

    Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    904c959 View commit details
    Browse the repository at this point in the history

  17. qa/cephfs: Add more tests for multimds scrub 

    Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
    @sidharthanup @ukernel
    sidharthanup authored and ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    ba9b85c View commit details
    Browse the repository at this point in the history

  18. qa/cephfs: log-ignorelist scrub errors 

    Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 16, 2020
    Configuration menu
    Copy the full SHA
    c42570a View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2020

  1. Update Release notes for multimds scrub 

    Signed-off-by: "Yan, Zheng" <zyan@redhat.com>
    @ukernel
    ukernel committed Nov 17, 2020
    Configuration menu
    Copy the full SHA
    11a1997 View commit details
    Browse the repository at this point in the history