New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loci loading from file seems to fail when running given a `file:///` path while running on yarn #446

Closed
timodonnell opened this Issue Apr 21, 2016 · 4 comments

Comments

Projects
None yet
3 participants
@timodonnell
Member

timodonnell commented Apr 21, 2016

Getting this exception

java.lang.IllegalArgumentException: Wrong FS: file:/hpc/users/odonnt02/sinai/git/projects/guacamole-validation/joint-caller-runs/aocs/runs/aocs-034-wgs/aocs_034_called_loci.txt, expected: hdfs://demeter-nn1.demeter.hpc.mssm.edu:8020
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645)
at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:302)
at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766)
at org.hammerlab.guacamole.Common$.lociFromFile(Common.scala:244)
at org.hammerlab.guacamole.Common$.loci(Common.scala:269)
at org.hammerlab.guacamole.commands.jointcaller.SomaticJoint$Caller$.run(SomaticJointCaller.scala:86)
at org.hammerlab.guacamole.commands.jointcaller.SomaticJoint$Caller$.run(SomaticJointCaller.scala:62)
at org.hammerlab.guacamole.SparkCommand.run(Command.scala:55)
at org.hammerlab.guacamole.Command.run(Command.scala:46)
at org.hammerlab.guacamole.Guacamole$.main(Guacamole.scala:70)
at org.hammerlab.guacamole.Guacamole.main(Guacamole.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542)

https://github.com/hammerlab/guacamole/blob/master/src/main/scala/org/hammerlab/guacamole/Common.scala#L244

coming from this argument

--force-call-loci-from-file file:///hpc/users/odonnt02/sinai/git/projects/guacamole-validation/joint-caller-runs/aocs/runs/aocs-034-wgs/aocs_034_called_loci.txt

@e5c

This comment has been minimized.

Show comment
Hide comment
@e5c

e5c May 1, 2016

Member

@timodonnell This has been fixed in this branch:
https://github.com/hammerlab/guacamole/tree/loci_io

Pending an actual PR to avoid merge conflicts with #459 (it'll be an easy rebase)

Member

e5c commented May 1, 2016

@timodonnell This has been fixed in this branch:
https://github.com/hammerlab/guacamole/tree/loci_io

Pending an actual PR to avoid merge conflicts with #459 (it'll be an easy rebase)

@arahuja

This comment has been minimized.

Show comment
Hide comment
@arahuja

arahuja May 6, 2016

Contributor

Closed by #468 ?

Is the solution now that it can read from either HDFS or the local file system?

Contributor

arahuja commented May 6, 2016

Closed by #468 ?

Is the solution now that it can read from either HDFS or the local file system?

@timodonnell

This comment has been minimized.

Show comment
Hide comment
@timodonnell

timodonnell May 6, 2016

Member

That's my understanding

Member

timodonnell commented May 6, 2016

That's my understanding

@e5c

This comment has been minimized.

Show comment
Hide comment
@e5c

e5c May 6, 2016

Member

@arahuja yup should be fixed by #468

Member

e5c commented May 6, 2016

@arahuja yup should be fixed by #468

@arahuja arahuja closed this May 6, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment