Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

osd: make ec overwrites ready to use #14496

Merged
merged 11 commits into from Apr 20, 2017

Conversation

Projects
None yet
2 participants
@jdurgin
Copy link
Member

jdurgin commented Apr 13, 2017

Enable ec overwrites tests for scrubbing and rados model, make the parameter no longer marked 'hacky' or experimental, and document how to enable it.

Try to stop people from enabling overwrites with filestore, since it performs much worse and more importantly won't work with deep scrub.

@jdurgin

This comment has been minimized.

Copy link
Member Author

jdurgin commented Apr 13, 2017

@tchaikov tchaikov self-requested a review Apr 13, 2017

@jdurgin

This comment has been minimized.

Copy link
Member Author

jdurgin commented Apr 13, 2017

retest this please

@jdurgin jdurgin force-pushed the jdurgin:wip-ec-overwrites-cleanup branch from 8ddb67a to 83144d3 Apr 18, 2017

@liewegas liewegas changed the title make ec overwrites ready to use osd: make ec overwrites ready to use Apr 18, 2017

@liewegas
Copy link
Member

liewegas left a comment

yay!

@jdurgin jdurgin force-pushed the jdurgin:wip-ec-overwrites-cleanup branch from 83144d3 to 48080a9 Apr 18, 2017

@jdurgin

This comment has been minimized.

Copy link
Member Author

jdurgin commented Apr 18, 2017

just fixed a new use of 'debug_white_box_testing...', will merge if jenkins succeeds

@jdurgin

This comment has been minimized.

Copy link
Member Author

jdurgin commented Apr 18, 2017

retest this please

1 similar comment
@jdurgin

This comment has been minimized.

Copy link
Member Author

jdurgin commented Apr 19, 2017

retest this please

@jdurgin jdurgin force-pushed the jdurgin:wip-ec-overwrites-cleanup branch from 48080a9 to 3b2b6a4 Apr 19, 2017

jdurgin added some commits Feb 14, 2017

OSDMonitor: get stripe_width via stripe_unit in ec profile
With bluestore, making the smallest write match min_alloc_size avoids
write amplification. With EC pools this is the stripe unit, or
stripe_width / num_data_chunks. Rather than requiring people to divide
by k to get the smallest ec write, allow it to be specified directly
via stripe_unit. Store it in the ec profile so changing a monitor
config option isn't necessary to set it.

This is particularly important for ec overwrites since they allow random i/o
which should match bluestore's checksum granularity (aka min_alloc_size).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
osd, OSDMonitor, qa: mark ec overwrites non-experimental
Keep the pool flag around so we can distinguish between a pool that
should maintain hashes for each chunk, and a missing one is a bug, vs
an overwrites pool where we rely on bluestore checksums for detecting
corruption.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
qa/tasks/rados: test sparse reads with ec overwrites
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
test/osd/osd-scrub-repair.sh: add ec overwrites test cases
Move pool and profile creation into a single function, and
add a 'allow_overwrites' parameter for it so each ec test
can be paramterized by it.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
osd,OSDMonitor: try to protect against ec overwrites with filestore
This isn't perfect, but it's better than nothing. Prevent enabling the
allow_ec_overwrites flag if any of a sample of pgs in the pool map to
osds using filestore. This mainly protects filestore-only clusters
from enabling it.

If a filestore osd is started later, warn in the cluster log when it
gets a pg with ec overwrites enabled.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
PendingReleaseNotes: note about ec overwrites
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
doc: update ec docs for overwrite support
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
qa: move ec overwrites tests to a bluestore-only subsuite
Signed-off-by: Josh Durgin <jdurgin@redhat.com>
qa: use 4k stripe_width again for test_rados_tool.sh
The stripe_unit change altered the default stripe_width to depend on k.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
qa: use bluestore for cephfs cephtool tests
This lets the tests enable ec overwrites

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
@jdurgin

This comment has been minimized.

Copy link
Member Author

jdurgin commented Apr 20, 2017

qa: use bluestore for (k)rbd ec overwrites tests
Only bluestore supports ec overwrites

Signed-off-by: Josh Durgin <jdurgin@redhat.com>

@jdurgin jdurgin force-pushed the jdurgin:wip-ec-overwrites-cleanup branch from 00758ef to cab2968 Apr 20, 2017

@jdurgin jdurgin merged commit 535b2cf into ceph:master Apr 20, 2017

1 of 3 checks passed

Signed-off-by checking if commits are signed
Details
default Build triggered. sha1 is merged.
Details
Unmodifed Submodules submodules for project are unmodified
Details

@jdurgin jdurgin deleted the jdurgin:wip-ec-overwrites-cleanup branch Apr 20, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.