HDFS-17064. Document the usage of the new Balancer "sortTopNodes" and "hotBlockTimeInterval" parameter #5794
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…otBlockTimeInterval parameter
|
💔 -1 overall
This message was automatically generated. |
| + "\tSort datanodes based on the utilization so " | ||
| + "that highly utilized datanodes get scheduled first."; | ||
| + "that highly utilized datanodes get scheduled first." |
There was a problem hiding this comment.
Code style: align at the beginning of a line.
| @@ -304,6 +306,7 @@ Usage: | |||
| | `-idleiterations` \<iterations\> | Maximum number of idle iterations before exit. This overwrites the default idleiterations(5). | | |||
| | `-runDuringUpgrade` | Whether to run the balancer during an ongoing HDFS upgrade. This is usually not desired since it will not affect used space on over-utilized machines. | | |||
| | `-asService` | Run Balancer as a long running service. | | |||
| | `-sortTopNodes` | Whether to run the balancer during will sort nodes b capacity usage percentage to prioritize top used nodes. | | |||
There was a problem hiding this comment.
This sentence doesn't seem to make sense.
|
Thanks @zhangshuyan0 help me review it. Follow your suggestions and update PR, please help review it again, Thanks. |
|
💔 -1 overall
This message was automatically generated. |
zhangshuyan0
reviewed
Jul 3, 2023
| @@ -304,6 +306,7 @@ Usage: | |||
| | `-idleiterations` \<iterations\> | Maximum number of idle iterations before exit. This overwrites the default idleiterations(5). | | |||
| | `-runDuringUpgrade` | Whether to run the balancer during an ongoing HDFS upgrade. This is usually not desired since it will not affect used space on over-utilized machines. | | |||
| | `-asService` | Run Balancer as a long running service. | | |||
| | `-sortTopNodes` | Whether to run the balancer during will sort datanodes based on the utilization so that highly utilized datanodes get scheduled first. | | |||
There was a problem hiding this comment.
What does this sentence mean? It makes me confused.
There was a problem hiding this comment.
Hi @zhangshuyan0 sir, it means setting the option "-sortTopNodes", each iteration will choose the high used capacity datanode in the overUtilized list as the source datanode of the moving block.
There was a problem hiding this comment.
Should it be "Sort datanodes based on the utilization so that highly utilized datanodes are scheduled first." ?
There was a problem hiding this comment.
Should it be "Sort datanodes based on the utilization so that highly utilized datanodes are scheduled first." ?
noted, this description is easier to understand.
I will update it later, Thanks.
There was a problem hiding this comment.
Update PR.
@zhangshuyan0 please help review it again, Thanks.
|
💔 -1 overall
This message was automatically generated. |
|
Hi,sir @Hexiaoqiao @ayushtkn Could you please help me review this minor changes when you have free time ? Thanks a lot~ |
jiajunmao
pushed a commit
to jiajunmao/hadoop-MLEC
that referenced
this pull request
Feb 6, 2024
… "hotBlockTimeInterval" parameter (apache#5794). Contributed by Haiyang Hu. Reviewed-by: Shuyan Zhang <zqingchai@gmail.com> Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org> Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
https://issues.apache.org/jira/browse/HDFS-17064
Document the usage of the new Balancer "sortTopNodes" and "hotBlockTimeInterval" parameter