New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

osd: support dumping long ops #13019

Merged
merged 2 commits into from Apr 21, 2017

Conversation

Projects
None yet
3 participants
@wonzhq
Contributor

wonzhq commented Jan 20, 2017

No description provided.

@liewegas

This comment has been minimized.

Member

liewegas commented Jan 20, 2017

I'm a bit confused. Isn't the OpHistory already collecting the N longest ops we've ever seen? Is the goal to track the, say, 10 longest, and also the last 1000 over some specific threshold?

@liewegas liewegas self-assigned this Jan 20, 2017

@wonzhq

This comment has been minimized.

Contributor

wonzhq commented Jan 20, 2017

The current OpHistory only tracks the last 20 completed ops or the ops in the last 600s. The intent here is to track the last 20 ops whose processing time is over a threshold value (say 10s).

@gregsfortytwo

This comment has been minimized.

Member

gregsfortytwo commented Feb 1, 2017

This seems fine to me conceptually — it'd be nice if you got warnings about delayed ops if they didn't get washed out by the time you actually got to dumping historic ops.

@wonzhq

This comment has been minimized.

Contributor

wonzhq commented Feb 9, 2017

Resolved conflict. Ping @liewegas

@wonzhq

This comment has been minimized.

Contributor

wonzhq commented Feb 9, 2017

retest this please

@liewegas

This comment has been minimized.

Member

liewegas commented Feb 9, 2017

Can we call these "recent slow" ops everywhere? Otherwise, LGTM!

@@ -1825,6 +1827,11 @@ bool OSD::asok_command(string command, cmdmap_t& cmdmap, string format,
ss << "op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. \
Please enable \"osd_enable_op_tracker\", and the tracker will start to track new ops received afterwards.";
}
} else if (command == "dump_historic_long_ops") {

This comment has been minimized.

@liewegas

liewegas Feb 9, 2017

Member

this command also needs to be registered and unregsitered.. search for 'dump_history_ops' in this file to see where/how that happens

@wonzhq

This comment has been minimized.

Contributor

wonzhq commented Feb 9, 2017

Updated, @liewegas

if (changed.count("osd_op_history_slow_op_size") ||
changed.count("osd_op_history_slow_op_threshold")) {
op_tracker.set_history_size_and_duration(cct->_conf->osd_op_history_slow_op_size,
cct->_conf->osd_op_history_slow_op_threshold);

This comment has been minimized.

@liewegas

liewegas Feb 9, 2017

Member

this is calling the wrong setter

And can we do "recent_slow" instead of "slow"? the historic ops are already the slowest ops ever; the difference here is that we're also tracking the most recent N slow ops

This comment has been minimized.

@wonzhq

wonzhq Feb 9, 2017

Contributor

oops, should be set_history_slow_op_size_and_threshold.

The current historic ops are not the slowest ops, I think. The are only the most recent ops.

@liewegas

This comment has been minimized.

Member

liewegas commented Apr 19, 2017

rebase please?

Zhiqiang Wang added some commits Jan 20, 2017

Zhiqiang Wang
common: support tracking slow op
Add slow op in op tracker history.

Signed-off-by: Zhiqiang Wang <zhiqiang@xsky.com>
Zhiqiang Wang
osd: support dumping historic slow ops
ceph daemon osd.x dump_historic_slow_ops

Signed-off-by: Zhiqiang Wang <zhiqiang@xsky.com>
@wonzhq

This comment has been minimized.

Contributor

wonzhq commented Apr 20, 2017

done, @liewegas

@liewegas liewegas added the needs-qa label Apr 20, 2017

@liewegas liewegas merged commit 97478a7 into ceph:master Apr 21, 2017

3 checks passed

Signed-off-by all commits in this PR are signed
Details
Unmodifed Submodules submodules for project are unmodified
Details
default Build finished.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment