Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IllegalArgumentException Wrong FS for vcf_head files on HDFS #1272

Closed
heuermh opened this issue Nov 16, 2016 · 1 comment
Closed

IllegalArgumentException Wrong FS for vcf_head files on HDFS #1272

heuermh opened this issue Nov 16, 2016 · 1 comment

Comments

@heuermh
Copy link
Member

@heuermh heuermh commented Nov 16, 2016

From adam-developers@googlegroups.com:

$ adam-submit adam2vcf \
  "hdfs://localhost:54310/vcf_pruebas/vcfpruebas.adam" \
  "hdfs://localhost:54310/vcf_pruebas/vcfpruebas2.vcf"

...
2016-11-10 16:46:49 ERROR Executor:91 - Exception in task 0.0 in stage 1.0 (TID 1)
java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:54310/vcf_pruebas/vcfpruebas2.vcf_head, expected: file:///
  at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:647)
  at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:82)
  at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:604)
  at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:822)
  at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:599)
  at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:421)
  at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:140)
  at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:341)
  at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:767)
  at org.seqdoop.hadoop_bam.util.WrapSeekable.openPath(WrapSeekable.java:60)
  at org.seqdoop.hadoop_bam.KeyIgnoringVCFOutputFormat.readHeaderFrom(KeyIgnoringVCFOutputFormat.java:82)
  at org.bdgenomics.adam.rdd.variation.ADAMVCFOutputFormat.getRecordWriter(ADAMVCFOutputFormat.scala:46)
...

When the process finishes, it leaves in the HDFS two new files:

  • An empty "vcfpruebas2.vcf" (the expected name for the final file)
  • A file "vcfpruebas2.vcf_head", that contains: a meta-information VCF structure, with just some of the FORMAT and INFO keys that were in the original VCF file (and many less than those that are in the INFO column of the original variants contents), and the header line, including all the sample columns.
@fnothaft
Copy link
Member

@fnothaft fnothaft commented Mar 3, 2017

@heuermh without more info, we can't debug this, as I suspect this is a Hadoop conf issue. We've added support for reading/writing with non-default configured FS. I am going to close this ticket, and we can reopen if this reoccurs.

@fnothaft fnothaft closed this Mar 3, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.