New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
YARN-6648. BackPort [GPG] Add SubClusterCleaner in Global Policy Generator. #5676
Conversation
💔 -1 overall
This message was automatically generated. |
// The expiration time for a subcluster heartbeat, default is 30 minutes | ||
public static final String GPG_SUBCLUSTER_EXPIRATION_MS = | ||
FEDERATION_GPG_PREFIX + "subcluster.heartbeat.expiration-ms"; | ||
public static final long DEFAULT_GPG_SUBCLUSTER_EXPIRATION_MS = 1800000; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make it getTimeDuration() friendly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for helping to review the code, I will improve the code.
*/ | ||
public SubClusterCleaner(Configuration conf, GPGContext gpgContext) { | ||
this.heartbeatExpirationMillis = | ||
conf.getLong(YarnConfiguration.GPG_SUBCLUSTER_EXPIRATION_MS, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
getTimeDuration()
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will improve the code.
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
@goiri Can you help review this pr? Thank you very much! |
...hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/conf/YarnConfiguration.java
Show resolved
Hide resolved
@@ -87,10 +100,32 @@ protected void serviceInit(Configuration conf) throws Exception { | |||
@Override | |||
protected void serviceStart() throws Exception { | |||
super.serviceStart(); | |||
|
|||
// Scheduler SubClusterCleaner service | |||
long scCleanerIntervalMs = getConfig().getLong( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be getTimeDuration()?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will modify the code.
// The second subcluster didn't heartbeat for two seconds, should mark lost | ||
subClusterId = subClusterIds.get(1); | ||
stateStore.setSubClusterLastHeartbeat(subClusterId, | ||
System.currentTimeMillis() - 2000); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Define it as TWO_SECONDS maybe with TimeUnit.SECONDS.toMillis(2)
The expiration time for a subcluster heartbeat, default is 30 minutes. | ||
</description> | ||
<name>yarn.federation.gpg.subcluster.heartbeat.expiration-ms</name> | ||
<value>1800000</value> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Define it with time duration format.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will modify this configuration value.
🎊 +1 overall
This message was automatically generated. |
@goiri Can you help review this PR again? Thank you very much! |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
@goiri Thank you very much for your help in reviewing the code! |
Description of PR
JIRA: YARN-6648. BackPort [GPG] Add SubClusterCleaner in Global Policy Generator.
How was this patch tested?
For code changes:
LICENSE
,LICENSE-binary
,NOTICE-binary
files?