Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/bluestore: multi-tier support in BlueStore #18211

Closed
wants to merge 12 commits into from

Conversation

@kiizawa
Copy link

commented Oct 10, 2017

Allow users to add a fast tier (ex. SSD) to BlueStore so that clients can do tiering in order to improve performance.

http://tracker.ceph.com/issues/21741

@liewegas liewegas added the bluestore label Oct 10, 2017

if (!bdev_fast) {
_txc_update_store_statfs(txc);
return;
}

This comment has been minimized.

Copy link
@liewegas

liewegas Oct 10, 2017

Member

goto end of put the below in a block; otherwise we duplicate code

This comment has been minimized.

Copy link
@kiizawa

kiizawa Oct 11, 2017

Author

It turns out that simply using goto statement doesn't work because of compile errors (cross initialization of variables).
Instead, I surrounded the latter part of _txc_finalize_kv() with curly braces to avoid code duplication.

@liewegas

This comment has been minimized.

Copy link
Member

commented Oct 10, 2017

Overall this looks pretty good! All the setup and teardown looks like it's done right (yay!).

The main change I think we would do for a mergeable implementation would be to mark invididual extents with the block device (e.g., using the high bit(s) of the offset). That way if the fast or slow device fills up we can spill over to the other one.

Also, if we choose to go this path for reals, we should probably combine the block.db and block.fast so that bluestore can use one fast device for data and metadata. More complex code but easier for the user to manage--they just provide some fast, some slow, and we make the best of what we're given.

But for the purposes of this experiment, I think what you have here looks just fine!

@kiizawa kiizawa force-pushed the kiizawa:wip-bluestore-tiering branch 2 times, most recently from d601d83 to 6950f4d Oct 11, 2017

@kiizawa

This comment has been minimized.

Copy link
Author

commented Oct 12, 2017

Jenkins, test this please

@markhpc markhpc added the performance label Oct 12, 2017

@kiizawa kiizawa force-pushed the kiizawa:wip-bluestore-tiering branch 3 times, most recently from 7f826a4 to 37f5981 Oct 12, 2017

@kiizawa

This comment has been minimized.

Copy link
Author

commented Oct 12, 2017

@liewegas Thank you for your review.

I use the highest bit of offset in bluestore_pextent_t as the flag which indicates where this physical extent is stored.
I totally agree that combining block.db and block.fast would be useful, but I'd like to leave it as future work for now.

@kiizawa kiizawa force-pushed the kiizawa:wip-bluestore-tiering branch 2 times, most recently from 3543196 to b1ccd5b Oct 25, 2017

@kiizawa kiizawa force-pushed the kiizawa:wip-bluestore-tiering branch 3 times, most recently from bd1535f to 10ea487 Nov 6, 2017

@kiizawa

This comment has been minimized.

Copy link
Author

commented Nov 10, 2017

Jenkins, test this please

@kiizawa kiizawa force-pushed the kiizawa:wip-bluestore-tiering branch from 10ea487 to 38ff40d Nov 10, 2017

@kiizawa kiizawa force-pushed the kiizawa:wip-bluestore-tiering branch from 38ff40d to 52336d8 Dec 12, 2017

@kiizawa kiizawa force-pushed the kiizawa:wip-bluestore-tiering branch from 52336d8 to 94c0382 Dec 31, 2017

@shinobu-x

This comment has been minimized.

Copy link
Contributor

commented Jan 15, 2018

@kiizawa ping

@kiizawa kiizawa force-pushed the kiizawa:wip-bluestore-tiering branch from 94c0382 to 8a0df73 Jan 16, 2018

@kiizawa

This comment has been minimized.

Copy link
Author

commented Jan 16, 2018

@shinobu-x I’ve resolved some merge conflicts I got when I rebased.
However, the process of merging this PR is being suspended by @liewegas, because this might not be the best way to implement tiering in Bluestore.
For example, we could implement it in block layer instead of directly in Bluestore.

root and others added 7 commits Oct 8, 2017
Initial commit
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
Initial commit
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
Added debug message
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
disable deferred I/O when writing to fast tier
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
disable deferred I/O when writing to fast tier
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
delete unused variable
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
Avoid code duplication
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
kiizawa added 5 commits Oct 11, 2017
use the highest bit of offset in bluestore_pextent_t as the flag
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
Added asynchronous set_alloc_hint
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
Allow set_alloc_hint to return an error code
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
Reserve space on destination tier before deleting data on source tier
Signed-off-by: kiizawa <kiizawa@ucsc.edu>
Allow write to return an error code
Signed-off-by: kiizawa <kiizawa@ucsc.edu>

@kiizawa kiizawa force-pushed the kiizawa:wip-bluestore-tiering branch 3 times, most recently from 0906fd5 to d08ee9f Jan 29, 2018

@stale

This comment has been minimized.

Copy link

commented Oct 18, 2018

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you are a maintainer or core committer, please follow-up on this issue to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@stale stale bot added the stale label Oct 18, 2018

@liewegas liewegas closed this Dec 20, 2018

yanghonggang pushed a commit to yanghonggang/ceph that referenced this pull request Sep 15, 2019
Yang Honggang
os/bluestore: multi-tier support in BlueStore
port kiizawa's patch(ceph#18211)

Signed-off-by: Yang Honggang <yanghonggang@umcloud.com>
yanghonggang pushed a commit to yanghonggang/ceph that referenced this pull request Sep 15, 2019
Yang Honggang
os/bluestore: multi-tier support in BlueStore
port kiizawa's patch(ceph#18211)

Signed-off-by: Yang Honggang <yanghonggang@umcloud.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.