New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No asSingleFile parameter for saveAsFasta in NucleotideContigFragmentRDD #1438

Closed
akmorrow13 opened this Issue Mar 14, 2017 · 5 comments

Comments

Projects
3 participants
@akmorrow13
Contributor

akmorrow13 commented Mar 14, 2017

This could be solved by moving

from FeatureRDD to a more central location to allow NucleotideContigFragmentRDD to save single file fastas.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Mar 14, 2017

Member

Perhaps we could move that into FileMerger?

Member

fnothaft commented Mar 14, 2017

Perhaps we could move that into FileMerger?

@fnothaft fnothaft changed the title from No aSingleFileParameter for saveAsFasta in NucleotideContigFragmentRDD to No asSingleFile parameter for saveAsFasta in NucleotideContigFragmentRDD Mar 14, 2017

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 14, 2017

Member

Yeah the save stuff needs work in a lot of places; see e.g. #1295 (comment). I was planning to address some of it when implementing the fix for #1327, waiting for after the next bdg-utils release for ADAM version 0.23.0.

Member

heuermh commented Mar 14, 2017

Yeah the save stuff needs work in a lot of places; see e.g. #1295 (comment). I was planning to address some of it when implementing the fix for #1327, waiting for after the next bdg-utils release for ADAM version 0.23.0.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 16, 2017

Member

For reference

$ jar -tvf adam-assembly/target/adam_2.10-0.21.1-SNAPSHOT.jar | grep ".class" | grep bdgenomics | grep -v '\$' | awk '{print $8}' | sed -e 's/\//./g' -e 's/.class//g' | xargs javap -private -s -classpath adam-assembly/target/adam_2.10-0.21.1-SNAPSHOT.jar | grep save | grep -v "default"
  public <U extends org.apache.avro.specific.SpecificRecordBase> void saveAvro(java.lang.String, org.apache.spark.SparkContext, org.apache.avro.Schema, scala.collection.Seq<U>, scala.reflect.ClassTag<U>);
  public void saveRddAsParquet(org.bdgenomics.utils.cli.SaveArgs);
  public void saveRddAsParquet(java.lang.String, int, int, org.apache.parquet.hadoop.metadata.CompressionCodecName, boolean, scala.Option<org.apache.avro.Schema>);
  public void saveMetadata(java.lang.String);
  public void saveAsParquet(org.bdgenomics.utils.cli.SaveArgs);
  public void saveAsParquet(java.lang.String, int, int, org.apache.parquet.hadoop.metadata.CompressionCodecName, boolean);
  public void saveAsParquet(java.lang.String, java.lang.Integer, java.lang.Integer, org.apache.parquet.hadoop.metadata.CompressionCodecName, java.lang.Boolean);
  public void saveAsParquet(java.lang.String);
  public void saveMetadata(java.lang.String);
  public void saveAsParquet(org.bdgenomics.utils.cli.SaveArgs);
  public void saveAsParquet(java.lang.String, int, int, org.apache.parquet.hadoop.metadata.CompressionCodecName, boolean, scala.Option<org.apache.avro.Schema>);
  public void save(java.lang.String);
  public void saveAsFasta(java.lang.String, int);
  public void save(java.lang.String, java.lang.Boolean);
  public void save(java.lang.String, java.lang.Boolean);
  public void saveAsGtf(java.lang.String, boolean);
  public void saveAsGff3(java.lang.String, boolean);
  public void saveAsBed(java.lang.String, boolean);
  public void saveAsIntervalList(java.lang.String, boolean);
  public void saveAsNarrowPeak(java.lang.String, boolean);
  public void save(java.lang.String);
  public void saveMetadata(java.lang.String);
  public boolean save(org.bdgenomics.adam.rdd.ADAMSaveAnyArgs, boolean);
  public java.lang.Boolean save(java.lang.String, java.lang.Boolean);
  public java.lang.String saveAsSamString();
  public void saveAsSam(java.lang.String, scala.Option<org.seqdoop.hadoop_bam.SAMFormat>, boolean, boolean, boolean);
  public void saveAsSam(java.lang.String, org.seqdoop.hadoop_bam.SAMFormat, java.lang.Boolean, java.lang.Boolean);
  public void saveAsPairedFastq(java.lang.String, java.lang.String, boolean, htsjdk.samtools.ValidationStringency, scala.Option<org.apache.spark.storage.StorageLevel>);
  public void saveAsFastq(java.lang.String, scala.Option<java.lang.String>, boolean, boolean, htsjdk.samtools.ValidationStringency, scala.Option<org.apache.spark.storage.StorageLevel>);
  public void save(java.lang.String);
  public boolean save(org.bdgenomics.utils.cli.SaveArgs);
  public void saveAsVcf(org.bdgenomics.utils.cli.SaveArgs, boolean);
  public void saveAsVcf(java.lang.String, boolean, htsjdk.samtools.ValidationStringency);
  public void saveAsVcf(org.bdgenomics.utils.cli.SaveArgs, boolean);
  public void saveAsVcf(java.lang.String, boolean, htsjdk.samtools.ValidationStringency);
  public void saveMetadata(java.lang.String);
  public void save(java.lang.String);
  public void saveAsVcf(java.lang.String, boolean, htsjdk.samtools.ValidationStringency);
Member

heuermh commented Mar 16, 2017

For reference

$ jar -tvf adam-assembly/target/adam_2.10-0.21.1-SNAPSHOT.jar | grep ".class" | grep bdgenomics | grep -v '\$' | awk '{print $8}' | sed -e 's/\//./g' -e 's/.class//g' | xargs javap -private -s -classpath adam-assembly/target/adam_2.10-0.21.1-SNAPSHOT.jar | grep save | grep -v "default"
  public <U extends org.apache.avro.specific.SpecificRecordBase> void saveAvro(java.lang.String, org.apache.spark.SparkContext, org.apache.avro.Schema, scala.collection.Seq<U>, scala.reflect.ClassTag<U>);
  public void saveRddAsParquet(org.bdgenomics.utils.cli.SaveArgs);
  public void saveRddAsParquet(java.lang.String, int, int, org.apache.parquet.hadoop.metadata.CompressionCodecName, boolean, scala.Option<org.apache.avro.Schema>);
  public void saveMetadata(java.lang.String);
  public void saveAsParquet(org.bdgenomics.utils.cli.SaveArgs);
  public void saveAsParquet(java.lang.String, int, int, org.apache.parquet.hadoop.metadata.CompressionCodecName, boolean);
  public void saveAsParquet(java.lang.String, java.lang.Integer, java.lang.Integer, org.apache.parquet.hadoop.metadata.CompressionCodecName, java.lang.Boolean);
  public void saveAsParquet(java.lang.String);
  public void saveMetadata(java.lang.String);
  public void saveAsParquet(org.bdgenomics.utils.cli.SaveArgs);
  public void saveAsParquet(java.lang.String, int, int, org.apache.parquet.hadoop.metadata.CompressionCodecName, boolean, scala.Option<org.apache.avro.Schema>);
  public void save(java.lang.String);
  public void saveAsFasta(java.lang.String, int);
  public void save(java.lang.String, java.lang.Boolean);
  public void save(java.lang.String, java.lang.Boolean);
  public void saveAsGtf(java.lang.String, boolean);
  public void saveAsGff3(java.lang.String, boolean);
  public void saveAsBed(java.lang.String, boolean);
  public void saveAsIntervalList(java.lang.String, boolean);
  public void saveAsNarrowPeak(java.lang.String, boolean);
  public void save(java.lang.String);
  public void saveMetadata(java.lang.String);
  public boolean save(org.bdgenomics.adam.rdd.ADAMSaveAnyArgs, boolean);
  public java.lang.Boolean save(java.lang.String, java.lang.Boolean);
  public java.lang.String saveAsSamString();
  public void saveAsSam(java.lang.String, scala.Option<org.seqdoop.hadoop_bam.SAMFormat>, boolean, boolean, boolean);
  public void saveAsSam(java.lang.String, org.seqdoop.hadoop_bam.SAMFormat, java.lang.Boolean, java.lang.Boolean);
  public void saveAsPairedFastq(java.lang.String, java.lang.String, boolean, htsjdk.samtools.ValidationStringency, scala.Option<org.apache.spark.storage.StorageLevel>);
  public void saveAsFastq(java.lang.String, scala.Option<java.lang.String>, boolean, boolean, htsjdk.samtools.ValidationStringency, scala.Option<org.apache.spark.storage.StorageLevel>);
  public void save(java.lang.String);
  public boolean save(org.bdgenomics.utils.cli.SaveArgs);
  public void saveAsVcf(org.bdgenomics.utils.cli.SaveArgs, boolean);
  public void saveAsVcf(java.lang.String, boolean, htsjdk.samtools.ValidationStringency);
  public void saveAsVcf(org.bdgenomics.utils.cli.SaveArgs, boolean);
  public void saveAsVcf(java.lang.String, boolean, htsjdk.samtools.ValidationStringency);
  public void saveMetadata(java.lang.String);
  public void save(java.lang.String);
  public void saveAsVcf(java.lang.String, boolean, htsjdk.samtools.ValidationStringency);
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft
Member

fnothaft commented Mar 16, 2017

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 21, 2017

Member

Thank goodness that grep/awk/sed line above isn't fully visible since it scrolls off to the right :)

Member

heuermh commented Mar 21, 2017

Thank goodness that grep/awk/sed line above isn't fully visible since it scrolls off to the right :)

fnothaft added a commit to fnothaft/adam that referenced this issue Jun 26, 2017

[ADAM-1438] Add ability to save FASTA back as a single file.
Resolves bigdatagenomics#1438. To support saving FASTA as a single file, moved the
private `writeTextRdd` method from the `FeatureRDD` to `GenomicRDD`, and opened
protections to `protected`.

fnothaft added a commit to fnothaft/adam that referenced this issue Jul 11, 2017

[ADAM-1438] Add ability to save FASTA back as a single file.
Resolves bigdatagenomics#1438. To support saving FASTA as a single file, moved the
private `writeTextRdd` method from the `FeatureRDD` to `GenomicRDD`, and opened
protections to `protected`.

heuermh added a commit that referenced this issue Jul 11, 2017

[ADAM-1438] Add ability to save FASTA back as a single file.
Resolves #1438. To support saving FASTA as a single file, moved the
private `writeTextRdd` method from the `FeatureRDD` to `GenomicRDD`, and opened
protections to `protected`.

@heuermh heuermh modified the milestone: 0.23.0 Jul 22, 2017

@heuermh heuermh added this to Completed in Release 0.23.0 Jan 4, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment