Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does angel support the outputPath without hdfs prefix? #130

Closed
facaiy opened this issue Jul 7, 2017 · 0 comments
Closed

Does angel support the outputPath without hdfs prefix? #130

facaiy opened this issue Jul 7, 2017 · 0 comments

Comments

@facaiy
Copy link
Contributor

facaiy commented Jul 7, 2017

When path is specified without hdfs prefix, like

./bin/angel-submit \
    --action.type train \
    --angel.app.submit.class com.tencent.angel.ml.classification.lr.LRRunner  \
    --angel.train.data.path /usr/foo/data \
    --angel.save.model.path /usr/foo/model \

Angel always failed with the exception, which is really ambiguous:

Exit code: 1
Stack trace: ExitCodeException exitCode=1:
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
        at org.apache.hadoop.util.Shell.run(Shell.java:456)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

So I checked all logs on yarn, and noticed that the names of output directories start with null.

17/07/03 15:02:02 INFO utils.HdfsUtil: tmp output dir is null:///tmp/application_1498535347111_0403_6bb79e70-16df-40a5-8207-1181b7bea2e9
17/07/03 15:02:02 INFO utils.HdfsUtil: tmp output dir is null:///tmp/application_1498535347111_0403_f2b40825-7a27-42bd-b983-164ff3342026
17/07/03 15:02:02 INFO client.AngelClient: angel.tmp.output.path=null:/tmp/f/application_1498535347111_0403_6bb79e70-16df-40a5-8207-1181b7bea2e9
17/07/03 15:02:02 INFO client.AngelClient: internal state file is null:/tmp/application_1498535347111_0403_f2b40825-7a27-42bd-b983-164ff3342026/state

@facaiy facaiy changed the title Does angel support that outputPath without hdfs prefix? Does angel support the outputPath without hdfs prefix? Jul 7, 2017
@facaiy facaiy closed this as completed Jul 10, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant