Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
[ADAM-1164] Add parallel file merger. #1441
Just following up with runtime numbers. With this, saving the NA12878 234GB BAM back to a single BAM from Parquet runs in 4.4 minutes on 833 cores (2.4 minutes to go ADAM->BAM, 2.0 minutes to do the merge). Without this, it takes 44 minutes (2.3 minutes to go ADAM->BAM, the remainder to merge).