New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
YARN-11359. [Federation] Routing admin invocations transparently to multiple RMs. #5057
Conversation
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
...-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/RouterServerUtil.java
Show resolved
Hide resolved
@@ -491,4 +514,24 @@ public static SubClusterId getRandomActiveSubCluster( | |||
// Randomly choose a SubCluster | |||
return subClusterIds.get(rand.nextInt(subClusterIds.size())); | |||
} | |||
|
|||
public static UserGroupInformation setupUser(String userName) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
final String userName
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will fix it.
/** | ||
* Class to define admin method,params and arguments. | ||
*/ | ||
public class AdminMethod { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there anything really admin about this class?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class is created to be used when the RMAdmin
interface is called, so this class is called AdminMethod
.
|
||
import java.io.IOException; | ||
import java.lang.reflect.Method; | ||
import java.util.*; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Expand
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will fix it.
return new String[0]; | ||
} | ||
|
||
<R> Collection<R> invokeConcurrent(AdminMethod request, Class<R> clazz) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't we have very similar methods in other places?
Can we refactor?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you very much for your suggestion, I will refactor this part of the code.
@@ -200,7 +200,6 @@ public void shutdown() { | |||
MockNM mockNM = mockNMs.getOrDefault(subClusterId, null); | |||
try { | |||
mockNM.unRegisterNode(); | |||
mockNM = null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no need?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will fix it.
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
@goiri Thank you very much for your help in reviewing the code! |
JIRA: YARN-11359. [Federation] Routing admin invocations transparently to multiple RMs.
The YARN Router does not support calling SubCluster's RMAdmin directly.
If we want to use management commands in Federation mode, we still need to find the Yarn client of the specified SubCluster, and then execute the Yarn management command.
The JIRA(YARN-11351) plan will enhance the router's capabilities, provide a set of CLI commands, and execute SubCluster RM management commands on the Router side.
This JIRA (YARN-11359) is a Sub-Task of YARN-11351, the purpose is to create
FederationRMAdminInterceptor
, complete multi-RM access support, and complete Junit tests.FederationRMAdminInterceptor contains many methods, which will be implemented separately by submitting PR. This PR only completes the refreshQueues method.