Skip to content

combine bqsr processes into one#877

Merged
gongyixiao merged 17 commits intodevelopfrom
enhancement/combineBQSR
Aug 11, 2021
Merged

combine bqsr processes into one#877
gongyixiao merged 17 commits intodevelopfrom
enhancement/combineBQSR

Conversation

@anoronh4
Copy link
Copy Markdown
Collaborator

@anoronh4 anoronh4 commented Feb 6, 2021

Adding BQSRPipelineSpark in one process as an alternative to BaseRecalibrator and ApplyBQSR in two separate processes.

If the process fails twice it switches from BQSRPipelineSpark to BaseRecalibrator and ApplyBQSR in the same process for the remaining attempt(s)

@anoronh4 anoronh4 requested a review from gongyixiao February 6, 2021 22:11
@anoronh4 anoronh4 changed the title adjusted containers combine bqsr processes into one Feb 7, 2021
Comment thread pipeline.nf Outdated
@anoronh4 anoronh4 force-pushed the enhancement/combine_merge_markduplicates branch from 8426eed to 8714be0 Compare February 7, 2021 15:17
@gongyixiao gongyixiao added the enhancement New feature or request label Feb 8, 2021
@gongyixiao gongyixiao added this to the 1.4 milestone Feb 8, 2021
@anoronh4 anoronh4 marked this pull request as ready for review February 9, 2021 03:01
@anoronh4 anoronh4 changed the base branch from enhancement/combine_merge_markduplicates to develop February 9, 2021 03:01
Comment thread pipeline.nf Outdated
task.time = task.exitStatus.toString() in wallTimeExitCode ? { params.maxWallTime } : { params.medWallTime }
}
if (workflow.profile == "juno") {
if(bam.size() > 60.GB) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are 60.GB and 50.GB are optimized value based on our test run?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the logic behind this choice may be a little outdated, from when i was combining MarkDup and BQSR together. I cut the numbers from MarkDup in half to account for the fact that we were processing/writing the bam twice. I'll revert this back to what the parameters were for RecalibrateBam and then revert based on testing.

@gongyixiao
Copy link
Copy Markdown
Collaborator

containers/gatk-samtools/Dockerfile and containers/gatk-samtools/environment.yml are not needed I believe.

@gongyixiao
Copy link
Copy Markdown
Collaborator

containers/gatk-samtools/Dockerfile and containers/gatk-samtools/environment.yml are not needed I believe.

Please remove these two files

@gongyixiao gongyixiao merged commit ec6d10d into develop Aug 11, 2021
@gongyixiao gongyixiao deleted the enhancement/combineBQSR branch August 11, 2021 03:06
@gongyixiao gongyixiao mentioned this pull request Aug 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants