Run HaplotypeCallerSpark on WGS in strict mode #5721
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #5721 +/- ##
===============================================
- Coverage 86.985% 86.984% -0.001%
- Complexity 31863 31865 +2
===============================================
Files 1943 1943
Lines 146768 146775 +7
Branches 16223 16225 +2
===============================================
+ Hits 127666 127671 +5
Misses 13189 13189
- Partials 5913 5915 +2
|
jamesemery
approved these changes
Mar 4, 2019
| // 5. Convert shards to assembly regions. | ||
| JavaRDD<AssemblyRegion> assemblyRegions = assemblyRegionShardedReads.map((Function<Shard<GATKRead>, AssemblyRegion>) shard -> toAssemblyRegion(shard, header)); | ||
| // 5. Convert shards to assembly regions. Reads downsampling is done again here, and is assumed to be consistent | ||
| // with the downsampling done in step 1, since it is deterministic by locus. |
There was a problem hiding this comment.
Since it sounds like you want to get this branch in first, i would instead make this a comment referencing the other branch and noting the issue.
| @@ -159,12 +159,20 @@ | |||
| // at which points the reads can be filled in. (See next step.) | |||
| JavaRDD<ReadlessAssemblyRegion> readlessAssemblyRegions = contigToGroupedStates | |||
| .flatMap(getReadlessAssemblyRegionsFunction(header, assemblyRegionArgs)); | |||
| // repartition to distribute the data evenly across the cluster again | |||
| readlessAssemblyRegions = readlessAssemblyRegions.repartition(readlessAssemblyRegions.getNumPartitions()); | |||
There was a problem hiding this comment.
Before this repartitioning it looks like the reads may have been partitioned by contig? Flatmap I assume doesn't repartition automatically.
…ing assembly regions
tomwhite
force-pushed
the
tw_hcs_strict_genome
branch
from
9034302
to
955d034
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These are the changes needed to run on a whole genome in strict mode. We get out of memory errors without these changes.
Reads downsampling was missing for the part where
AssemblyRegions are filled with reads - this PR adds it in. Downsampling is not deterministic yet, since that depends on #5437, but that's an orthogonal issue so it's OK to merge this change and add #5437 later.