mon: fix slow op warning on mon, improve slow op warnings #21684
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jdurgin
approved these changes
Apr 26, 2018
|
@jdurgin issues like 23769 make me nervous about global whitelisting SLOW_OPS. Perhaps we should distinguish slow ops and "stuck" ops somehow. |
|
@batrick that's a good idea. a higher threshold for detecting bugs like that makes sense |
Otherwise it is very hard to identify which OSD ops are slow when we've seen a SLOW_OPS health warning in a qa run. Notably, without this, bugs like http://tracker.ceph.com/issues/23769 are very challenging to track down. Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
Signed-off-by: Sage Weil <sage@redhat.com>
If we don't note that we don't reply then we don't close out the routed mon request and the op will appear as slow on the forwarding mon. Fixes: http://tracker.ceph.com/issues/23769 Signed-off-by: Sage Weil <sage@redhat.com>
liewegas
changed the title
|
Great catch! Thanks Sage. |
|
@batrick i think with this we should revert the blanket SLOW_OPS whitelist in teuthology. IMO we should do that explicitly on runs doing thrashing or stressy/heavy workloads or whatever. |
|
Agreed |
|
batrick
added a commit
to batrick/teuthology
that referenced
this pull request
May 3, 2018
Should no longer be necessary after [1]. [1] ceph/ceph#21684 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes http://tracker.ceph.com/issues/23769