ScrubStack and "tag path" command #5662
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@ukernel, please review this so we can get some external eyes on pieces of fsck. :) |
|
I still don't fully understand the code. the code that traverse and scrub subtree generally looks good. should I kick the merge button? |
| CInode *in = i->second->get_projected_linkage()->get_inode(); | ||
| if (in && in->is_dir()) | ||
| scrub_infop->directories_to_scrub.insert(i->first); | ||
| else if (in) |
There was a problem hiding this comment.
do we need scrub remote linkage here?
|
It seems the scrub code can't handle busy directories.(fragment a directory or rename a directory while scrub) Am I missing something? |
Use colon initialization for all the members we can, instead of assignment in the function body. The split is confusing and doesn't seem helpful. Signed-off-by: Greg Farnum <greg@inktank.com>
Wrap up checks for whether the dentry has actually been initialized to contain any data. Signed-off-by: Greg Farnum <gfarnum@redhat.com>
The ValidationContinuation is wrapping an MDR, but other MDSContinuations might not. Remove that bit of the interface. Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Signed-off-by: Greg Farnum <greg@inktank.com>
The local scrub bits are just for the data stored in the dirfrag, like checking rstats match the actual directory contents and such. The recursive stats include actual scrubs on all the descendant files and directories too! Signed-off-by: Greg Farnum <gfarnum@redhat.com>
This is set up so that the coming-shortly scrub system can initialize scrubbing on a directory and perform the actual scrubs, but rely on the directory to keep track of its internal scrub state and selection of dentries. Note that initialize_scrub() and scrub_local() can only be called while the directory is complete. Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
…sk_state() Add another set of scrub stamps for the local scrub, which map onto the fnode local scrub members. Use it in CInode::validate_disk_state() instead of the call to check_rstats, because this way we bump the scrub stats, and also in scrub_initialize() because we know we'll have everything we need in memory at that time, while we might not later on. Signed-off-by: Greg Farnum <gfarnum@redhat.com>
The CDentry is going to be the point of entry for scrubbing code, and is how we maintain our cache size. Therefore, it gets a scrub_parent pointer and a new PIN_SCRUBQUEUE in addition to the basic scrub_info_t struct and scrub_initialize() and scrub_finish() functions. Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Like the CDir stuff, this has data structures and helpers to track its own internal scrub state, and -- in the case of directory inodes -- to select the next dirfrag you need to handle. Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Signed-off-by: Greg Farnum <gfarnum@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
A place for scrubbed dentries to learn about the params that started the scrub off, such as the tag. Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
...now that I grok how long the dir is meant to be complete Signed-off-by: John Spray <john.spray@redhat.com>
Signed-off-by: John Spray <john.spray@redhat.com>
| // FIXME: the magic-constructing scrub_info() is going to leave | ||
| // an unneeded scrub_infop lying around here | ||
| *finally_done = !dn->scrub_info()->dentry_scrubbing; | ||
| if (!*finally_done) { |
There was a problem hiding this comment.
dentry_scrubbing is set to true on initiating. (no space updates its value). this means scrub_dir_dentry_final() can call in->validate_disk_state() multiple times
Signed-off-by: Yan, Zheng <zyan@redhat.com>
ukernel
force-pushed
the
wip-scrub-basic
branch
from
5f56d59
to
9fdbc1e
Compare
call CDir::scrub_finished() after finishing scrub a dirfrag Signed-off-by: Yan, Zheng <zyan@redhat.com>
Signed-off-by: Yan, Zheng <zyan@redhat.com>
Also set the on finish context when starting scrub a dentry, this avoids deleting context for skipped dentry. Signed-off-by: Yan, Zheng <zyan@redhat.com>
Signed-off-by: Yan, Zheng <zyan@redhat.com>
Signed-off-by: Yan, Zheng <zyan@redhat.com>
Signed-off-by: Yan, Zheng <zyan@redhat.com>
ukernel
force-pushed
the
wip-scrub-basic
branch
from
9fdbc1e
to
b65ace2
Compare
|
@jcsp @gregsfortytwo Can we merge this? |
jcsp
added a commit
that referenced
this pull request
Nov 18, 2015
ScrubStack and "tag path" command Reviewed-by: John Spray <john.spray@redhat.com> Reviewed-by: Yan, Zheng <zyan@redhat.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the stuff from my branch, minus the orphan-finding stuff in data-scan (which will come later after #5545)
I would like to come back later and refactor the scrub_info_p stuff, in order to avoid putting all the methods directly onto the CDir/CInode/CDentry classes, and to avoid the slightly odd implicit construction of the info object from scrub_info().
Getting the guts of this landed (albeit probably a bit rough around the edges) is desirable to enable the other work (DamageTable, orphan finding in cephfs-data-scan) to proceed.
TestForwardScrub.test_apply_tag (from wip-scrub-jcs ceph-qa-suite branch) verifies the basics of this.