HDFS-16634. Dynamically adjust slow peer report size on JMX metrics #4448
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
tomscut
reviewed
Jun 17, 2022
| * the highest number of votes from peers. | ||
| */ | ||
| private final int maxNodesToReport; | ||
| private int maxNodesToReport; |
Contributor
There was a problem hiding this comment.
Please set this to volatile. Although it doesn't make a big difference here, I think it's better to be consistent with other reconfig changes. What do you think of this?
Contributor
Author
There was a problem hiding this comment.
Yes, you have made good point @tomscut, this can be done to remain in line with other reconfig changes, however it might cause bit of a performance issue for JMX metrics API overall, hence I was bit reluctant to make the change. But if you have strong preference, I can make the change, WDYT?
Contributor
There was a problem hiding this comment.
This field is almost always read-only and rarely changed, and for read-only operations, it doesn't have much impact. WDYT?
Contributor
Author
There was a problem hiding this comment.
Yeah it's fine I think, this is not a big concern either ways. So let me make this change.
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
Contributor
|
Thanks @virajjasani for your contribution! |
Contributor
|
Hi @virajjasani , could you please submit another PR for branch-3.3 since there are some conflicts when cherry-pick. Thanks. |
HarshitGupta11
pushed a commit
to HarshitGupta11/hadoop
that referenced
this pull request
Nov 28, 2022
…pache#4448) Signed-off-by: Tao Li <tomscut@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
On a busy cluster, sometimes it takes bit of time for deleted node(from the cluster)'s "slow node report" to get removed from slow peer json report on Namenode JMX metrics. In the meantime, user should be able to browse through more entries in the report by adjusting i.e. reconfiguring "dfs.datanode.max.nodes.to.report" so that the list size can be adjusted without user having to bounce active Namenode just for this purpose.
How was this patch tested?
Dev cluster and using UT.
For code changes: