HBASE-27535: Separate slowlog thresholds for scans vs other requests #5188
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
@virajjasani if you have any feedback here then I'd definitely appreciate it! Thank you in advance 😄 |
virajjasani
reviewed
Apr 19, 2023
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
Outdated
Show resolved
Hide resolved
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
Outdated
Show resolved
Hide resolved
hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/RpcServer.java
Outdated
Show resolved
Hide resolved
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
virajjasani
approved these changes
Apr 20, 2023
There was a problem hiding this comment.
+1, could you also add the new configs descriptions to https://github.com/apache/hbase/blob/master/src/main/asciidoc/_chapters/ops_mgt.adoc ?
(after hbase.ipc.warn.response.time and hbase.ipc.warn.response.size)
| @@ -1904,14 +1904,20 @@ It is also prepended with identifying tags `(responseTooSlow)`, `(responseTooLar | |||
|
|
|||
| ==== Configuration | |||
|
|
|||
| There are two configuration knobs that can be used to adjust the thresholds for when queries are logged. | |||
| There are four configuration knobs that can be used to adjust the thresholds for when queries are logged. | |||
There was a problem hiding this comment.
Can you add 1-2 more sentences here explaining why Scan gets its own? For example "Two knobs allow controlling the size and time threshold for all queries. Since Scans can often be larger and slower than other queries, there is an additional two knobs for setting a different threshold for those."
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
virajjasani
approved these changes
Apr 21, 2023
bbeaudreault
approved these changes
Apr 22, 2023
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
Thanks @rmdmattingly and @bbeaudreault !! |
virajjasani
pushed a commit
that referenced
this pull request
Apr 22, 2023
…5188) Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org>
bbeaudreault
pushed a commit
to HubSpot/hbase
that referenced
this pull request
Jun 19, 2023
…pache#5188) Signed-off-by: Bryan Beaudreault <bbeaudreault@apache.org> Signed-off-by: Viraj Jasani <vjasani@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
HBASE-27535
Scans by their nature are able to more efficiently pull back larger response sizes than gets. They also may take longer to execute than other request types. We should make it possible to configure a separate threshold for response time and response time for scans. This will allow us to tune down the thresholds for others without adding unnecessary noise for requests which are known to be slower/bigger.
This PR introduces two new dynamically configurable keys:
hbase.ipc.warn.response.time.scanhbase.ipc.warn.response.size.scanBy default, scans will fallback to using the values of the original keys without a
.scansuffix. Because of this fallback this change is a no-op by default.I've loaded this changeset onto a test cluster and verified the intended behavior, both in the case where these values are specified and unspecified.
cc @bbeaudreault