-
Notifications
You must be signed in to change notification settings - Fork 97
[REEF-1411] Support multiple RMs and other Hadoop distributions #1020
Conversation
This addressed the issue by * Reading RM web addresses from yarn-site.xml. * Deprecating MultipleRMUrlProvider in favor of YarnConfigurationUrlProvider. JIRA: [REEF-1411](https://issues.apache.org/jira/browse/REEF-1411)
|
||
namespace Org.Apache.REEF.Utilities | ||
{ | ||
public static class Yarn |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Document.
Would it make sense to create a mirror of the |
@markusweimer There is already a |
The |
@markusweimer I see. I think it's a good idea, but it's a much larger undertaking than what this PR calls for. Should we open a JIRA item for it? |
By the way, I don't really like putting the YARN static class in Utilities since it's runtime dependent; however, creating a new project seems too aggressive. Do you think it's fine leaving it there for now? Maybe we can add an |
Agreed. I was just wondering whether we should start with the couple of configuration keys we need now here, and keep adding to them as we need more. I don't think it is wise for us to blindly create access methods for all configuration options without a clear use case. |
/// <summary> | ||
/// The Hadoop configuration directory environment variable. | ||
/// </summary> | ||
public const string HadoopConfDirEnvVariable = "HADOOP_CONF_DIR"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The const
strings are dangerous when their value ever changes, right? I believe the C# compiler is free to inline them in client code of this. Maybe make them readonly
instead?
Done with a pass. |
@markusweimer I've addressed your comments, please have another look. Thanks! |
LGTM. Will test and merge. |
@afchung I merged this but forgot the PR comment in the commit message. Please close. |
AppVeyor reports test failure on It reproduces on my local machine 100% of the runs. @afchung Please look into this failure with high priority. |
@tcNickolas I've noticed. Opened #1036. Thanks! |
Sorry, this somehow slipped past me. Will test and merge #1036 |
This addressed the issue by
JIRA:
REEF-1411