New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAM-1996] Load and save VariantContexts as partitioned Parquet. #2001

Merged
merged 2 commits into from Jun 27, 2018

Conversation

Projects
None yet
4 participants
@heuermh
Copy link
Member

heuermh commented Jun 21, 2018

Fixes #1996.

override val isPartitioned: Boolean = true,
override val optPartitionBinSize: Option[Int] = Some(1000000),
override val optLookbackPartitions: Option[Int] = Some(1)) extends VariantContextRDD
with DatasetBoundGenomicDataset[VariantContext, VariantContextProduct, VariantContextRDD] {

This comment has been minimized.

@heuermh

heuermh Jun 21, 2018

Member

I'm not fully aware of the implications of adding this trait, thought it preferable to copying the referenceRegionsToDatasetQueryString and filterByOverlappingRegions methods

This comment has been minimized.

@fnothaft

fnothaft Jun 27, 2018

Member

This trait should be mixed in, so LGTM.

@coveralls

This comment has been minimized.

Copy link

coveralls commented Jun 21, 2018

Coverage Status

Coverage decreased (-0.1%) to 79.049% when pulling f544f01 on heuermh:partitioned-variant-contexts into f05aa98 on bigdatagenomics:master.

2 similar comments
@coveralls

This comment has been minimized.

Copy link

coveralls commented Jun 21, 2018

Coverage Status

Coverage decreased (-0.1%) to 79.049% when pulling f544f01 on heuermh:partitioned-variant-contexts into f05aa98 on bigdatagenomics:master.

@coveralls

This comment has been minimized.

Copy link

coveralls commented Jun 21, 2018

Coverage Status

Coverage decreased (-0.1%) to 79.049% when pulling f544f01 on heuermh:partitioned-variant-contexts into f05aa98 on bigdatagenomics:master.

@coveralls

This comment has been minimized.

Copy link

coveralls commented Jun 21, 2018

Coverage Status

Coverage increased (+0.2%) to 79.39% when pulling b42e69b on heuermh:partitioned-variant-contexts into f05aa98 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Copy link

AmplabJenkins commented Jun 21, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2773/
Test PASSed.

@heuermh heuermh requested review from fnothaft and jpdna Jun 22, 2018

@AmplabJenkins

This comment has been minimized.

Copy link

AmplabJenkins commented Jun 22, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2774/
Test PASSed.

@heuermh heuermh added this to the 0.24.1 milestone Jun 23, 2018

@fnothaft fnothaft merged commit 28bcd56 into bigdatagenomics:master Jun 27, 2018

2 checks passed

Codacy/PR Quality Review Up to standards. A positive pull request.
Details
default Merged build finished.
Details
@fnothaft

This comment has been minimized.

Copy link
Member

fnothaft commented Jun 27, 2018

Merged! Thanks @heuermh!

@heuermh heuermh deleted the heuermh:partitioned-variant-contexts branch Jun 28, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment