HBASE-23083 Collect Executor status info periodically and report to metric system #664
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chenxu14
force-pushed
the
HBASE-23083
branch
from
6bbf89a
to
5ad507e
Compare
chenxu14
changed the title
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
chenxu14
force-pushed
the
HBASE-23083
branch
from
5ad507e
to
8e37736
Compare
|
🎊 +1 overall
This message was automatically generated. |
saintstack
reviewed
Sep 27, 2019
| @@ -1191,6 +1191,9 @@ | |||
| "hbase.node.health.failure.threshold"; | |||
| public static final int DEFAULT_HEALTH_FAILURE_THRESHOLD = 3; | |||
|
|
|||
| public static final String EXECUTOR_STATUS_COLLECTE_ENABLED = | |||
| "hbase.executors.status.collect.enabled"; | |||
| public static final boolean DEFAULT_EXECUTOR_STATUS_COLLECTE_ENABLED = true; | |||
| public ExecutorStatusChore(int sleepTime, Stoppable stopper, ExecutorService service, | ||
| MetricsRegionServerSource metrics) { | ||
| super("ExecutorStatusChore", stopper, sleepTime); | ||
| LOG.info("ExecutorStatusChore runs every " + StringUtils.formatTime(sleepTime)); |
There was a problem hiding this comment.
s/runs every/runs every {}/ and s/+/,/... i.e. use parameterized logging.
hbase-server/src/main/java/org/apache/hadoop/hbase/ExecutorStatusChore.java
Show resolved
Hide resolved
| HConstants.DEFAULT_EXECUTOR_STATUS_COLLECTE_ENABLED)) { | ||
| int sleepTime = this.conf.getInt(ExecutorStatusChore.WAKE_FREQ, | ||
| ExecutorStatusChore.DEFAULT_WAKE_FREQ); | ||
| executorStatusChore = new ExecutorStatusChore(sleepTime, this, this.getExecutorService(), |
There was a problem hiding this comment.
Are there other executor services running in RS? This the only one?
There was a problem hiding this comment.
Not the only one inHRegionServer#startServices()
There was a problem hiding this comment.
Hi Stack, maybe I misunderstood it. I mead there is only one ExecutorService, but it will start many Executors.
There was a problem hiding this comment.
My question is if this patch will get metrics on all executors in the regionserver or just one set? If it does not get metrics on them all, should it? Thanks.
There was a problem hiding this comment.
So, we only need one chore to report on all executors?
chenxu14
force-pushed
the
HBASE-23083
branch
from
8e37736
to
d6dc72d
Compare
|
🎊 +1 overall
This message was automatically generated. |
saintstack
reviewed
Oct 9, 2019
| * and report to metrics system | ||
| */ | ||
| @InterfaceAudience.Private | ||
| public class ExecutorStatusChore extends ScheduledChore { |
There was a problem hiding this comment.
Yeah, is it a chore per executorservice monitoried or is it one chore monitoring all executorservices in the process?
| HConstants.DEFAULT_EXECUTOR_STATUS_COLLECTE_ENABLED)) { | ||
| int sleepTime = this.conf.getInt(ExecutorStatusChore.WAKE_FREQ, | ||
| ExecutorStatusChore.DEFAULT_WAKE_FREQ); | ||
| executorStatusChore = new ExecutorStatusChore(sleepTime, this, this.getExecutorService(), |
There was a problem hiding this comment.
So, we only need one chore to report on all executors?
|
Hi @saintstack , just one Chore is enough, it will get metrics on all executors in the regionserver. metrics looks like this |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
metrics system