This repository was archived by the owner on Jul 10, 2024. It is now read-only.
SUBMARINE-308. Support PyTorch with TonY runtime#145
Closed
pingsutw wants to merge 3 commits intoapache:masterfrom
Closed
SUBMARINE-308. Support PyTorch with TonY runtime#145pingsutw wants to merge 3 commits intoapache:masterfrom
pingsutw wants to merge 3 commits intoapache:masterfrom
Conversation
Contributor
|
Change looks good. Test failure appears unrelated. |
pingsutw
force-pushed
the
SUBMARINE-308
branch
from
c294cc3 to
e505bd2
Compare
Member
Author
|
@jojochuang thanks for the review, I re-push the patch to trigger Travis ci |
adamantal
reviewed
Jan 9, 2020
| } | ||
|
|
||
| LOG.info("Starting Tony runtime.."); | ||
|
|
There was a problem hiding this comment.
Could you add a log entry here about the type of the framework?
| tonyConf = YarnUtils.tonyConfFromClientContext( | ||
| (ParametersHolder) parameters); | ||
| } catch (Exception e) { | ||
| throw new RuntimeException("Failed to create tony conf from client context", e); |
There was a problem hiding this comment.
Maybe throw a SubmarineException here?
| // Resources for PS & Worker | ||
| if (parameters.getPsResource() != null) { | ||
| if (parameters.getOptionValue(CliConstants.PS_RES) != null) { | ||
| Resource resource = getResource(parameters, CliConstants.PS_RES); |
There was a problem hiding this comment.
Suggested change
| Resource resource = getResource(parameters, CliConstants.PS_RES); | |
| Resource psResource = getResource(parameters, CliConstants.PS_RES); |
| } | ||
| if (parameters.getWorkerResource() != null) { | ||
| if (parameters.getOptionValue(CliConstants.WORKER_RES) != null) { | ||
| Resource resource = getResource(parameters, CliConstants.WORKER_RES); |
There was a problem hiding this comment.
Suggested change
| Resource resource = getResource(parameters, CliConstants.WORKER_RES); | |
| Resource workerResource = getResource(parameters, CliConstants.WORKER_RES); |
|
|
||
| private static Resource getResource(Parameter parametersHolder, String option) | ||
| throws ParseException, YarnException { | ||
| String ResourceStr = |
There was a problem hiding this comment.
Suggested change
| String ResourceStr = | |
| String resourceStr = |
| throws ParseException, YarnException { | ||
| String ResourceStr = | ||
| parametersHolder.getOptionValue(option); | ||
| if (ResourceStr == null) { |
There was a problem hiding this comment.
Suggested change
| if (ResourceStr == null) { | |
| if (resourceStr == null) { |
Member
Author
|
Thanks @adamantal for the review. |
xunliu
approved these changes
Jan 11, 2020
Member
Author
|
Will merge if no more comments |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this PR for?
Support PyTorch with TonY runtime
What type of PR is it?
[Bug Fix]
Todos
What is the Jira issue?
https://issues.apache.org/jira/browse/SUBMARINE-308
How should this be tested?
https://travis-ci.org/pingsutw/hadoop-submarine/builds/634462654
Screenshots (if appropriate)
Questions: