New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Factor out *RDDFunctions classes #1011

Closed
fnothaft opened this Issue Apr 24, 2016 · 7 comments

Comments

Projects
None yet
2 participants
@fnothaft
Member

fnothaft commented Apr 24, 2016

We have many *RDDFunctions classes in org.bdgenomics.adam.rdd that should be factored out. I think that if we did this, we could eliminate most of the implicit conversions in org.bdgenomics.adam.rdd.ADAMContext too.

@fnothaft fnothaft self-assigned this Apr 24, 2016

@fnothaft fnothaft added this to the 0.20.0 milestone Apr 24, 2016

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Apr 24, 2016

Member

Please elaborate a bit more. I had some further clean up and moving stuff around in mind.

ADAMContext is a nasty bugger in terms of cyclic package dependencies and extensibility. Perhaps it needs to be that way so that users only have to import ADAMContext._ though.

Member

heuermh commented Apr 24, 2016

Please elaborate a bit more. I had some further clean up and moving stuff around in mind.

ADAMContext is a nasty bugger in terms of cyclic package dependencies and extensibility. Perhaps it needs to be that way so that users only have to import ADAMContext._ though.

fnothaft added a commit to fnothaft/adam that referenced this issue Apr 24, 2016

[ADAM-909] Refactoring variation RDDs.
Resolves #909:

* Refactors `org.bdgenomics.adam.rdd.variation` to add `GenomicRDD`s for
  `Genotype`, `Variant`, and `VariantContext`. These classes write
  sequence and sample metadata to disk.
* Refactors `ADAMRDDFunctions` to an abstract class in preparation for
  further refactoring in #1011.
* Added `AvroGenomicRDD` trait which consolidates Parquet + Avro metadata
  writing code across all Avro data models.
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Apr 24, 2016

Member

I just opened #1015, which I think will shade more light on my thoughts. Specifically, I'd like to eliminate all the *RDDFunctions classes and their implicits by migrating the code into the classes that extend GenomicRDDs.

Member

fnothaft commented Apr 24, 2016

I just opened #1015, which I think will shade more light on my thoughts. Specifically, I'd like to eliminate all the *RDDFunctions classes and their implicits by migrating the code into the classes that extend GenomicRDDs.

fnothaft added a commit to fnothaft/adam that referenced this issue May 20, 2016

[ADAM-909] Refactoring variation RDDs.
Resolves #909:

* Refactors `org.bdgenomics.adam.rdd.variation` to add `GenomicRDD`s for
  `Genotype`, `Variant`, and `VariantContext`. These classes write
  sequence and sample metadata to disk.
* Refactors `ADAMRDDFunctions` to an abstract class in preparation for
  further refactoring in #1011.
* Added `AvroGenomicRDD` trait which consolidates Parquet + Avro metadata
  writing code across all Avro data models.

fnothaft added a commit to fnothaft/adam that referenced this issue May 25, 2016

[ADAM-909] Refactoring variation RDDs.
Resolves #909:

* Refactors `org.bdgenomics.adam.rdd.variation` to add `GenomicRDD`s for
  `Genotype`, `Variant`, and `VariantContext`. These classes write
  sequence and sample metadata to disk.
* Refactors `ADAMRDDFunctions` to an abstract class in preparation for
  further refactoring in #1011.
* Added `AvroGenomicRDD` trait which consolidates Parquet + Avro metadata
  writing code across all Avro data models.

fnothaft added a commit to fnothaft/adam that referenced this issue May 26, 2016

[ADAM-909] Refactoring variation RDDs.
Resolves #909:

* Refactors `org.bdgenomics.adam.rdd.variation` to add `GenomicRDD`s for
  `Genotype`, `Variant`, and `VariantContext`. These classes write
  sequence and sample metadata to disk.
* Refactors `ADAMRDDFunctions` to an abstract class in preparation for
  further refactoring in #1011.
* Added `AvroGenomicRDD` trait which consolidates Parquet + Avro metadata
  writing code across all Avro data models.

fnothaft added a commit to fnothaft/adam that referenced this issue May 26, 2016

[ADAM-909] Refactoring variation RDDs.
Resolves #909:

* Refactors `org.bdgenomics.adam.rdd.variation` to add `GenomicRDD`s for
  `Genotype`, `Variant`, and `VariantContext`. These classes write
  sequence and sample metadata to disk.
* Refactors `ADAMRDDFunctions` to an abstract class in preparation for
  further refactoring in #1011.
* Added `AvroGenomicRDD` trait which consolidates Parquet + Avro metadata
  writing code across all Avro data models.

fnothaft added a commit to fnothaft/adam that referenced this issue May 26, 2016

[ADAM-909] Refactoring variation RDDs.
Resolves #909:

* Refactors `org.bdgenomics.adam.rdd.variation` to add `GenomicRDD`s for
  `Genotype`, `Variant`, and `VariantContext`. These classes write
  sequence and sample metadata to disk.
* Refactors `ADAMRDDFunctions` to an abstract class in preparation for
  further refactoring in #1011.
* Added `AvroGenomicRDD` trait which consolidates Parquet + Avro metadata
  writing code across all Avro data models.

fnothaft added a commit to fnothaft/adam that referenced this issue Jun 1, 2016

[ADAM-909] Refactoring variation RDDs.
Resolves #909:

* Refactors `org.bdgenomics.adam.rdd.variation` to add `GenomicRDD`s for
  `Genotype`, `Variant`, and `VariantContext`. These classes write
  sequence and sample metadata to disk.
* Refactors `ADAMRDDFunctions` to an abstract class in preparation for
  further refactoring in #1011.
* Added `AvroGenomicRDD` trait which consolidates Parquet + Avro metadata
  writing code across all Avro data models.

@fnothaft fnothaft modified the milestones: 0.21.0, 0.20.0 Jun 3, 2016

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jun 3, 2016

Member

Punting to 0.21.0.

Member

fnothaft commented Jun 3, 2016

Punting to 0.21.0.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jun 3, 2016

Member

Not sure I agree. I don't see the utility in cutting a release halfway through a major refactoring.

Member

heuermh commented Jun 3, 2016

Not sure I agree. I don't see the utility in cutting a release halfway through a major refactoring.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jun 3, 2016

Member

That's a fair point. Then maybe we push 0.20.0 out to end of June?

Member

fnothaft commented Jun 3, 2016

That's a fair point. Then maybe we push 0.20.0 out to end of June?

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jun 3, 2016

Member

Yeah, let's try to do a round of triage and estimation while we're all together in person.

Member

heuermh commented Jun 3, 2016

Yeah, let's try to do a round of triage and estimation while we're all together in person.

@fnothaft fnothaft modified the milestones: 0.20.0, 0.21.0 Jun 3, 2016

@heuermh heuermh referenced this issue Jun 7, 2016

Closed

Release ADAM version 0.20.0 #1048

47 of 61 tasks complete

fnothaft added a commit to fnothaft/adam that referenced this issue Jul 9, 2016

Removed VariationRDDFunctions.
* Removed `VariationRDDFunctions` and moved remaining methods to
  `VariantContextRDD`. Moved test suite code as well.
* Removed abstract class `ADAMSequenceDictionaryRDDAggregator`.
* Added Java helper methods to various variation RDDs and accompanying loader
  methods in `JavaADAMContext`.
* Miscellaneous documentation cleanup in `org.bdgenomics.adam.rdd.variation`.

Resolves #1011.
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jul 18, 2016

Member

This will have big conflicts with #1081 and #964. I would like to merge this in first, since it is the vastly more complex PR, and to then rebase the other two on top.

Member

fnothaft commented Jul 18, 2016

This will have big conflicts with #1081 and #964. I would like to merge this in first, since it is the vastly more complex PR, and to then rebase the other two on top.

fnothaft added a commit to fnothaft/adam that referenced this issue Jul 18, 2016

Removed VariationRDDFunctions.
* Removed `VariationRDDFunctions` and moved remaining methods to
  `VariantContextRDD`. Moved test suite code as well.
* Removed abstract class `ADAMSequenceDictionaryRDDAggregator`.
* Added Java helper methods to various variation RDDs and accompanying loader
  methods in `JavaADAMContext`.
* Miscellaneous documentation cleanup in `org.bdgenomics.adam.rdd.variation`.

Resolves #1011.

fnothaft added a commit to fnothaft/adam that referenced this issue Jul 18, 2016

Removed VariationRDDFunctions.
* Removed `VariationRDDFunctions` and moved remaining methods to
  `VariantContextRDD`. Moved test suite code as well.
* Removed abstract class `ADAMSequenceDictionaryRDDAggregator`.
* Added Java helper methods to various variation RDDs and accompanying loader
  methods in `JavaADAMContext`.
* Miscellaneous documentation cleanup in `org.bdgenomics.adam.rdd.variation`.

Resolves #1011.

@heuermh heuermh closed this in da2a142 Jul 18, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment