New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/bluestore: partial reshard support #13162

Merged
merged 20 commits into from Feb 7, 2017

Conversation

Projects
None yet
2 participants
@liewegas
Member

liewegas commented Jan 27, 2017

Reshard only a subset of the extent map. If a single extent gets big, we will reshard only
it. If one gets small, we will combine with the smaller of our two neighbors.

@liewegas liewegas changed the title from os/bluestore: partial reshard support to DNM: os/bluestore: partial reshard support Jan 27, 2017

Show outdated Hide outdated src/os/bluestore/BlueStore.cc Outdated
Show outdated Hide outdated src/os/bluestore/BlueStore.cc Outdated
b->id = bid++;
spanning_blob_map[b->id] = b;
dout(20) << __func__ << " adding spanning " << *b << dendl;
} else {

This comment has been minimized.

@ifed01

ifed01 Jan 31, 2017

Contributor

else -> else if(e->blob->is_spanning())

@ifed01

ifed01 Jan 31, 2017

Contributor

else -> else if(e->blob->is_spanning())

Show outdated Hide outdated src/os/bluestore/BlueStore.cc Outdated
Show outdated Hide outdated src/os/bluestore/BlueStore.h Outdated
si_end < shards.size())) {
// we resharded a partial range; we must produce at least one output
// shard
new_shard_info.emplace_back(bluestore_onode_t::shard_info());

This comment has been minimized.

@ifed01

ifed01 Jan 31, 2017

Contributor

You might want to implement shard_info(offset) ctor and apply it here and throughout?

@ifed01

ifed01 Jan 31, 2017

Contributor

You might want to implement shard_info(offset) ctor and apply it here and throughout?

liewegas added some commits Jan 27, 2017

os/bluestore: replace needs_reshard flag with a range
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: ExtentMap: use reshard ranges from update()
This gives us a single point of control for reshard, and tells us
which range we care about at the same time.

Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: ExtentMap: count extents per Shard
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: ExtentMap: remove reshard 0 extent special case
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore/bluestore_types: beautify use_tracker_t operator<<
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: reshard on truncate if shards past EOF
This ensure we mop up shards past EOF instead of encoding them
empty and confusing future code (that, say, assumes no shards past
EOF).

Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: fsck: check for shards past EOF
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: cosmetic cleanup in ExtentMap::update
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: ExtentMap: put un-spanning in normal loop
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: ExtentMap: use normal iterator for reshard loop
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: ExtentMap: request reshard range from update() intellig…
…ently

Split ourselves, or merge with our immediate predecessor or
successor.

Signed-off-by: Sage Weil <sage@redhat.com>

@liewegas liewegas changed the title from DNM: os/bluestore: partial reshard support to os/bluestore: partial reshard support Feb 2, 2017

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Feb 2, 2017

Member

Updated, with several bugs fixed. And rebased now that use_tracker is merged. Please take a look!

Member

liewegas commented Feb 2, 2017

Updated, with several bugs fixed. And rebased now that use_tracker is merged. Please take a look!

liewegas added some commits Feb 2, 2017

os/memstore: return 0 from fsck
Signed-off-by: Sage Weil <sage@redhat.com>
ceph_test_objectstore: error out on fsck failure
It's much more helpful this way.

Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: fsck: dump_onode after shards are loaded
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: trivial cleanups in csum error code
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: dump num spanning blobs from dump_onode
Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: ExtentMap: reshard only requested range
Only reshard extents in the requested range.

Note that the strategy for unspanning blobs changed; we now span or
unspan specific blobs at the end based on our reshard result instead
of unspanning all at the start.  This keeps the spanning id stable,
which is important because it may be referenced from another that
we aren't even looking at.

Also note that this requires a bit of restructuring: an encode_some may
hit a spanning shard, *requiring* us to reshard, which means we should
always conduct the initial pass through update to discover other reshard
requirements, even if we already know some resharding will be needed.

Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: simplify dirty shard handling in ExtentMap::update
We can do this with a single allocation with a simple vector<>.

Signed-off-by: Sage Weil <sage@redhat.com>
os/bluestore: drop ExtentMap::Shard offset field
This is mirrored in shard_info, which we have a pointer to.

Note that shard_info is also a bit redundant as we can also look it up in
the onode vector in the same position, but it makes it more awkward to use
iterators.  Something to consider later to save memory.

Signed-off-by: Sage Weil <sage@redhat.com>
@ifed01

ifed01 approved these changes Feb 3, 2017

Show outdated Hide outdated src/os/bluestore/BlueStore.cc Outdated
Show outdated Hide outdated src/os/bluestore/BlueStore.cc Outdated
@liewegas

This comment has been minimized.

Show comment
Hide comment

@liewegas liewegas merged commit d62a494 into ceph:master Feb 7, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details

@liewegas liewegas deleted the liewegas:wip-bluestore-reshard branch Feb 7, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment