New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os/bluestore: tune deferred_batch_ops separately for hdd and ssd #14435

Merged
merged 1 commit into from Apr 14, 2017

Conversation

Projects
None yet
3 participants
@liewegas
Member

liewegas commented Apr 10, 2017

Signed-off-by: Sage Weil sage@redhat.com

On hdd + nvme journal with #14399 this gets me 2800 iops sequential, 400 iops random (qd 16) via
vstart (fresh rbd image, no aging, etc.).

@@ -1789,6 +1789,7 @@ class BlueStore : public ObjectStore,
uint64_t min_alloc_size = 0; ///< minimum allocation unit (power of 2)
size_t min_alloc_size_order = 0; ///< bits for min_alloc_size
uint64_t prefer_deferred_size = 0; ///< size threshold for forced deferred writes
int deferred_batch_ops = 0; ///< deferred batch size

This comment has been minimized.

@ifed01

ifed01 Apr 10, 2017

Contributor

Shouldn't we use atomic vars for dynamically loadable settings?

@ifed01

ifed01 Apr 10, 2017

Contributor

Shouldn't we use atomic vars for dynamically loadable settings?

This comment has been minimized.

@liewegas

liewegas Apr 13, 2017

Member

In practice I think as long as the value is a word size or less we will still get an atomic update -- just not immediately. Which should be okay. I'm worried that using atomics might introduce a read barrier on read?

@liewegas

liewegas Apr 13, 2017

Member

In practice I think as long as the value is a word size or less we will still get an atomic update -- just not immediately. Which should be okay. I'm worried that using atomics might introduce a read barrier on read?

@markhpc

This comment has been minimized.

Show comment
Hide comment
@markhpc

markhpc Apr 11, 2017

Member

On HDD with NVMe WAL/DB and QD1, I'm seeing improvements even up to to 512 batched ops (though in reality the writes currently don't coalesce that well at the blk layer). I'd be tempted to bump the default HDD batched ops up to 128 (or higher) depending on the side effects.

Member

markhpc commented Apr 11, 2017

On HDD with NVMe WAL/DB and QD1, I'm seeing improvements even up to to 512 batched ops (though in reality the writes currently don't coalesce that well at the blk layer). I'd be tempted to bump the default HDD batched ops up to 128 (or higher) depending on the side effects.

@liewegas

This comment has been minimized.

Show comment
Hide comment
@liewegas

liewegas Apr 11, 2017

Member
Member

liewegas commented Apr 11, 2017

@markhpc

This comment has been minimized.

Show comment
Hide comment
@markhpc

markhpc Apr 12, 2017

Member

That was my concern as well, though interestingly it didn't seem to hurt very much for random IO. I agree though, this works best if we can identify sequential IOs.

Member

markhpc commented Apr 12, 2017

That was my concern as well, though interestingly it didn't seem to hurt very much for random IO. I agree though, this works best if we can identify sequential IOs.

@ifed01

ifed01 approved these changes Apr 13, 2017

@ifed01

This comment has been minimized.

Show comment
Hide comment
@ifed01

ifed01 Apr 13, 2017

Contributor

rebase?

Contributor

ifed01 commented Apr 13, 2017

rebase?

os/bluestore: tune deferred_batch_ops separately for hdd and ssd
Signed-off-by: Sage Weil <sage@redhat.com>

@liewegas liewegas merged commit 2a4e431 into ceph:master Apr 14, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details

@liewegas liewegas deleted the liewegas:wip-bluestore-deferred branch Apr 14, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment