Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

osd: make ec overwrites ready to use #14496

Merged
merged 11 commits into from
Apr 20, 2017
Merged

osd: make ec overwrites ready to use #14496

merged 11 commits into from
Apr 20, 2017

Commits on Apr 20, 2017

  1. OSDMonitor: get stripe_width via stripe_unit in ec profile 

    With bluestore, making the smallest write match min_alloc_size avoids
write amplification. With EC pools this is the stripe unit, or
stripe_width / num_data_chunks. Rather than requiring people to divide
by k to get the smallest ec write, allow it to be specified directly
via stripe_unit. Store it in the ec profile so changing a monitor
config option isn't necessary to set it.

This is particularly important for ec overwrites since they allow random i/o
which should match bluestore's checksum granularity (aka min_alloc_size).

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    930eae2 View commit details
    Browse the repository at this point in the history

  2. osd, OSDMonitor, qa: mark ec overwrites non-experimental 

    Keep the pool flag around so we can distinguish between a pool that
should maintain hashes for each chunk, and a missing one is a bug, vs
an overwrites pool where we rely on bluestore checksums for detecting
corruption.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    6fba80c View commit details
    Browse the repository at this point in the history

  3. qa/tasks/rados: test sparse reads with ec overwrites 

    Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    a219319 View commit details
    Browse the repository at this point in the history

  4. test/osd/osd-scrub-repair.sh: add ec overwrites test cases 

    Move pool and profile creation into a single function, and
add a 'allow_overwrites' parameter for it so each ec test
can be paramterized by it.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    3ca750d View commit details
    Browse the repository at this point in the history

  5. osd,OSDMonitor: try to protect against ec overwrites with filestore 

    This isn't perfect, but it's better than nothing. Prevent enabling the
allow_ec_overwrites flag if any of a sample of pgs in the pool map to
osds using filestore. This mainly protects filestore-only clusters
from enabling it.

If a filestore osd is started later, warn in the cluster log when it
gets a pg with ec overwrites enabled.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    4315221 View commit details
    Browse the repository at this point in the history

  6. PendingReleaseNotes: note about ec overwrites 

    Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    25e8e1a View commit details
    Browse the repository at this point in the history

  7. doc: update ec docs for overwrite support 

    Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    bad6066 View commit details
    Browse the repository at this point in the history

  8. qa: move ec overwrites tests to a bluestore-only subsuite 

    Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    a3e5cba View commit details
    Browse the repository at this point in the history

  9. qa: use 4k stripe_width again for test_rados_tool.sh 

    The stripe_unit change altered the default stripe_width to depend on k.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    4a7c058 View commit details
    Browse the repository at this point in the history

  10. qa: use bluestore for cephfs cephtool tests 

    This lets the tests enable ec overwrites

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    f52b9d1 View commit details
    Browse the repository at this point in the history

  11. qa: use bluestore for (k)rbd ec overwrites tests 

    Only bluestore supports ec overwrites

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
    @jdurgin
    jdurgin committed Apr 20, 2017
    Configuration menu
    Copy the full SHA
    cab2968 View commit details
    Browse the repository at this point in the history