New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

osd/OSD.h: requeue the scrub job with higher priority to shorten the blocking time of related requests #15552

Merged
merged 1 commit into from Jun 16, 2017

Conversation

Projects
None yet
5 participants
@TsaiJin
Contributor

TsaiJin commented Jun 7, 2017

When the current chunk is in the state of scrub, all requests targeted at this chunk will be blocked.

So when the scrub job of current chunk is requeued, it should be assigned a higher priority so that it can be processed as soon as possible to shorten the blocking time of related requetes.

Signed-off-by: Jin Cai caijin.caij@alibaba-inc.com

@TsaiJin TsaiJin changed the title from osd/OSD.h: requeue the scrub job with higher priority to osd/OSD.h: requeue the scrub job with higher priority to shorten the blocking time of related requests Jun 7, 2017

@ceph-jenkins

This comment has been minimized.

Collaborator

ceph-jenkins commented Jun 7, 2017

Build finished.

@tchaikov tchaikov self-requested a review Jun 7, 2017

@gregsfortytwo

We requeue scrubs for a lot of different reasons, not all of which involve client IO. I think we need to parameterize it so the caller can indicate a desire for prioritization.

@TsaiJin

This comment has been minimized.

Contributor

TsaiJin commented Jun 8, 2017

@gregsfortytwo, yes, we requeue scrubs for a lot of different reasons such as starting a scrub of a new chunk or comparing the scrub maps of a chunk among all replicas.
But only in the case of a chunk(scrubber.start != scrubber.end) in scrub, all IO requests which are targeted at the corresponding chunk will be blocked. So we need to requeue this type of scrub with higher priority.

@TsaiJin

This comment has been minimized.

Contributor

TsaiJin commented Jun 8, 2017

@gregsfortytwo I add a boolean type parameter to determine whether requeue the scrub job with high priority in requeue_scrub() function.

@gregsfortytwo

This comment has been minimized.

Member

gregsfortytwo commented Jun 9, 2017

Maybe I'm just being dense, but I don't really see how these tests imply that a client IO is waiting. Can't we do a direct query on one of the lists of blocked IOs instead of trying to guess based on the scrubber state?

osd/OSD: There are requests that will be blocked by the chunk, queue …
…this scrub job with higher priority

Signed-off-by: Jin Cai <caijin.caij@alibaba-inc.com>
@TsaiJin

This comment has been minimized.

Contributor

TsaiJin commented Jun 13, 2017

@gregsfortytwo make sense. I modify the code with your suggestion: only requeue the scrub job with higher priority when there are some requests blocked by this job rather than guessing by the state of chunky scrub.

There are some performance data I tested based on this patch:

test environment

five machines, 2 OSDs per machine, each OSD has one 2T NVMe device.
fio job: numjobs=8, iodepth=32, bs=16k, direct=1
osd_scrub_chunk_min=1,  osd_scrub_chunk_max=3

scrub-1
scrub-2

@gregsfortytwo

Perfect!

Reviewed-by: Greg Farnum gfarnum@redhat.com

@yuriw yuriw merged commit cecd1af into ceph:master Jun 16, 2017

3 of 4 checks passed

arm64 make check arm64 make check started
Details
Signed-off-by all commits in this PR are signed
Details
Unmodified Submodules submodules for project are unmodified
Details
make check make check succeeded
Details

@TsaiJin TsaiJin deleted the TsaiJin:wip-queue-scrub-with-higher-priority branch Jun 17, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment