New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ceph_osd: remove client message cap limit #14944

Merged
merged 1 commit into from May 24, 2017

Conversation

Projects
None yet
4 participants
@yuyuyu101
Member

yuyuyu101 commented May 4, 2017

Legacy value is too small even for hdd under large cluster.
We even found osd_client_message_cap may result to live lock among different
messages.

Signed-off-by: Haomai Wang haomai@xsky.com

@yuyuyu101 yuyuyu101 added the common label May 4, 2017

@yuyuyu101

This comment has been minimized.

Member

yuyuyu101 commented May 4, 2017

Another direction is removing the osd_client_message_cap option. From my view, msgr throttle main purpose is preventing memory abusing, such as bursted messages from client under a slow osd may occur unexpected memory. But we don't expect msgr throttle to be a good disk throttle to keep reasonable io depth, and it can't make this.

Like bluestore did, we can remove ops throttle totally, and only leave bytes throttle to prevent resource usage in reasonable range

@branch-predictor

This comment has been minimized.

Member

branch-predictor commented May 4, 2017

If the reason behind message throttler was memory, then it would be better to change it, so it takes into account the messages size (that can be roughly estimated from on-wire bytestream) and not their count.

@gregsfortytwo

This comment has been minimized.

Member

gregsfortytwo commented May 4, 2017

There's a separate bytes throttler which is also applied. I confess I don't remember why we have a separate one for total message count.

@liewegas

This comment has been minimized.

Member

liewegas commented May 4, 2017

It seemed like a good idea at the time (shrug). We can remove it! If we still want to avoid too many distinct messages we can add a constant factor per message.

ceph_osd: remove client message cap limit
Leave the messenger ops throttle interface, we could keep watching on changing

Signed-off-by: Haomai Wang <haomai@xsky.com>

@yuyuyu101 yuyuyu101 changed the title from config: improve osd_client_message_cap from 100 to 500 to ceph_osd: remove client message cap limit May 5, 2017

@liewegas liewegas merged commit 035cad4 into ceph:master May 24, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details

@yuyuyu101 yuyuyu101 deleted the yuyuyu101:wip-msgr-throttle-ops branch May 25, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment