Skip to content

Deepvariant#149

Merged
robert-a-forsyth merged 39 commits into
devfrom
deepvariant
Mar 31, 2026
Merged

Deepvariant#149
robert-a-forsyth merged 39 commits into
devfrom
deepvariant

Conversation

@robert-a-forsyth
Copy link
Copy Markdown
Collaborator

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

Copilot AI review requested due to automatic review settings March 24, 2026 11:58
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR integrates DeepVariant (and DeepSomatic) into the lrsomatic pipeline and refactors small-variant calling / phasing logic by splitting former happhase subworkflows into dedicated tumor-only, paired, consensus, and phasing components. It also adds multiple nf-core module implementations (bcftools helpers, DeepVariant processes) plus nf-test coverage for the new modules/subworkflow.

Changes:

  • Refactor LRSOMATIC small-variant calling into new subworkflows (tumor-only, paired germline/somatic, consensus, phasing/haplotyping) and wire outputs into downstream steps (VEP/Severus).
  • Add nf-core DeepVariant subworkflow and its component processes (makeexamples/callvariants/postprocessvariants) with tests.
  • Introduce bcftools helper modules (norm/isec/query/annotate) and config wiring (including GPU toggle via params.use_gpu).

Reviewed changes

Copilot reviewed 93 out of 94 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
workflows/lrsomatic.nf Rewires pipeline to new small-variant subworkflows and PoN handling.
subworkflows/nf-core/deepvariant/main.nf Adds DeepVariant subworkflow wrapper around makeexamples/callvariants/postprocessvariants.
subworkflows/nf-core/deepvariant/meta.yml Defines DeepVariant subworkflow I/O contract.
subworkflows/nf-core/deepvariant/README.md Documents DeepVariant subworkflow usage and metadata-uniqueness requirement.
subworkflows/nf-core/deepvariant/deepvariant.diff Patch tracking for nf-core DeepVariant subworkflow.
subworkflows/nf-core/deepvariant/tests/main.nf.test Adds nf-test coverage for DeepVariant subworkflow.
subworkflows/nf-core/deepvariant/tests/main.nf.test.snap Snapshot outputs for DeepVariant subworkflow tests.
subworkflows/nf-core/deepvariant/tests/equality.nf.test Adds equality test comparing subworkflow vs DEEPVARIANT_RUNDEEPVARIANT process.
subworkflows/nf-core/deepvariant/tests/deepvariant-workflow-and-process-equality-tester.nf Helper workflow to run both implementations and emit comparable outputs.
subworkflows/nf-core/deepvariant/tests/nextflow.config Test-specific DeepVariant args configuration.
subworkflows/nf-core/deepvariant/tests/disable-small-model.conf Test config to disable small-model generation.
subworkflows/local/tumor_only_happhase.nf Removes prior tumor-only happhase implementation.
subworkflows/local/tumor_normal_happhase.nf Removes prior paired (tumor-normal) happhase implementation.
subworkflows/local/tumor_only/tumoronly_smallvar.nf New tumor-only small-variant calling (ClairS-TO / DeepVariant / DeepSomatic + consensus selection).
subworkflows/local/paired/paired_smallvar_germline.nf New paired germline calling (Clair3 / DeepVariant + consensus selection).
subworkflows/local/paired/paired_smallvar_somatic.nf New paired somatic calling (ClairS / DeepSomatic + consensus selection).
subworkflows/local/small_variant_consensus.nf Adds consensus/all-variants merging logic using bcftools modules.
subworkflows/local/phasing_haplotyping.nf Adds unified phasing/haplotyping + (optional) modcall flow for tumor-only and paired samples.
subworkflows/local/deepsomatic.nf Adds DeepSomatic subworkflow wrapper (makeexamples/callvariants/postprocessvariants).
nextflow.config Adds new pipeline params (germline_var_keep, somatic_var_keep, trust_caller, use_gpu, skip_modcall, PoN params).
nextflow_schema.json Adds schema block for small-variant calling options.
conf/modules.config Adds process-level config for new modules/subworkflows (DeepVariant, DeepSomatic, bcftools helpers, LongPhase split).
conf/test.config Adds test-profile overrides to constrain regions for DeepVariant/DeepSomatic.
conf/base.config Updates GPU configuration to be driven by params.use_gpu.
modules.json Registers newly added nf-core modules/subworkflow and patch files.
.gitignore Ignores out/ directory.
modules/nf-core/deepvariant/makeexamples/main.nf Adds/patches DeepVariant makeexamples (incl. --sample_name).
modules/nf-core/deepvariant/makeexamples/meta.yml Adds DeepVariant makeexamples module metadata.
modules/nf-core/deepvariant/makeexamples/deepvariant-makeexamples.diff Patch tracking for DeepVariant makeexamples module.
modules/nf-core/deepvariant/makeexamples/tests/nextflow.config Test configuration for DeepVariant makeexamples.
modules/nf-core/deepvariant/makeexamples/tests/main.nf.test nf-test coverage for DeepVariant makeexamples.
modules/nf-core/deepvariant/makeexamples/tests/main.nf.test.snap Snapshots for DeepVariant makeexamples tests.
modules/nf-core/deepvariant/callvariants/main.nf Adds/patches DeepVariant callvariants (GPU/noaccel label selection).
modules/nf-core/deepvariant/callvariants/meta.yml Adds DeepVariant callvariants module metadata.
modules/nf-core/deepvariant/callvariants/deepvariant-callvariants.diff Patch tracking for DeepVariant callvariants module.
modules/nf-core/deepvariant/callvariants/tests/nextflow.config Test configuration for DeepVariant callvariants.
modules/nf-core/deepvariant/callvariants/tests/main.nf.test nf-test coverage for DeepVariant callvariants.
modules/nf-core/deepvariant/callvariants/tests/main.nf.test.snap Snapshots for DeepVariant callvariants tests.
modules/nf-core/deepvariant/postprocessvariants/main.nf Adds/patches DeepVariant postprocessvariants (incl. --sample_name).
modules/nf-core/deepvariant/postprocessvariants/meta.yml Adds DeepVariant postprocessvariants module metadata.
modules/nf-core/deepvariant/postprocessvariants/deepvariant-postprocessvariants.diff Patch tracking for DeepVariant postprocessvariants module.
modules/nf-core/deepvariant/postprocessvariants/tests/nextflow.config Test configuration for DeepVariant postprocessvariants.
modules/nf-core/deepvariant/postprocessvariants/tests/main.nf.test nf-test coverage for DeepVariant postprocessvariants.
modules/nf-core/deepvariant/postprocessvariants/tests/main.nf.test.snap Snapshots for DeepVariant postprocessvariants tests.
modules/nf-core/bcftools/norm/main.nf Adds bcftools norm module used for normalization and indexing logic.
modules/nf-core/bcftools/norm/meta.yml Metadata for bcftools norm module.
modules/nf-core/bcftools/norm/environment.yml Conda env for bcftools norm module.
modules/nf-core/bcftools/norm/tests/nextflow.config Test defaults for bcftools norm.
modules/nf-core/bcftools/norm/tests/nextflow.vcf.config Test config for VCF output mode.
modules/nf-core/bcftools/norm/tests/nextflow.vcf_gz.config Test config for VCF.GZ output mode.
modules/nf-core/bcftools/norm/tests/nextflow.bcf.config Test config for BCF output mode.
modules/nf-core/bcftools/norm/tests/nextflow.bcf_gz.config Test config for BCF.GZ output mode.
modules/nf-core/bcftools/norm/tests/vcf_gz_index.config Test config for default index writing.
modules/nf-core/bcftools/norm/tests/vcf_gz_index_tbi.config Test config for TBI index writing.
modules/nf-core/bcftools/norm/tests/vcf_gz_index_csi.config Test config for CSI index writing.
modules/nf-core/bcftools/isec/main.nf Adds bcftools isec module with extra emitted outputs for consensus/private sets.
modules/nf-core/bcftools/isec/meta.yml Metadata for bcftools isec module.
modules/nf-core/bcftools/isec/environment.yml Conda env for bcftools isec module.
modules/nf-core/bcftools/isec/bcftools-isec.diff Patch tracking for bcftools isec module.
modules/nf-core/bcftools/isec/tests/nextflow.config Test args for bcftools isec module.
modules/nf-core/bcftools/isec/tests/main.nf.test nf-test coverage for bcftools isec module.
modules/nf-core/bcftools/isec/tests/main.nf.test.snap Snapshots for bcftools isec tests.
modules/nf-core/bcftools/query/main.nf Adds bcftools query module that bgzips + tabix-indexes output.
modules/nf-core/bcftools/query/meta.yml Metadata for bcftools query module.
modules/nf-core/bcftools/query/environment.yml Conda env for bcftools query module.
modules/nf-core/bcftools/query/bcftools-query.diff Patch tracking for bcftools query module.
modules/nf-core/bcftools/query/tests/nextflow.config Test args for bcftools query module.
modules/nf-core/bcftools/query/tests/main.nf.test nf-test coverage for bcftools query module.
modules/nf-core/bcftools/query/tests/main.nf.test.snap Snapshots for bcftools query tests.
modules/nf-core/bcftools/annotate/main.nf Adds bcftools annotate module (annotate + output type/index handling).
modules/nf-core/bcftools/annotate/meta.yml Metadata for bcftools annotate module.
modules/nf-core/bcftools/annotate/environment.yml Conda env for bcftools annotate module.
modules/nf-core/bcftools/annotate/tests/nextflow.config Test config for bcftools annotate module.
modules/nf-core/bcftools/annotate/tests/main.nf.test.snap Snapshots for bcftools annotate tests.
modules/nf-core/bcftools/merge/main.nf Adjusts bcftools merge input signature for fasta/fai separation.
modules/nf-core/bcftools/merge/bcftools-merge.diff Patch tracking for bcftools merge input change.
modules/nf-core/longphase/phase/main.nf Adjusts tabix indexing to handle multiple LongPhase outputs.
modules/nf-core/severus/main.nf Fixes output filename patterns (breakpoint(s) naming).
modules/local/clairsto/main.nf Updates ClairS-TO module to accept PoN VCFs/flags and set --sample_name.
modules/local/clairs/main.nf Adds --sample_name support via prefix.
modules/local/clair3/main.nf Adds GPU/noaccel label and passes --sample_name.
modules/local/vcfsplit/main.nf Updates non-somatic filtering expression before constructing germline VCF.
modules/local/longphase/modcall/main.nf Adds LongPhase modcall module.
modules/local/fibertoolsrs/predictm6a/main.nf Adds GPU/noaccel label selection.
modules/local/fibertoolsrs/nucleosomes/main.nf Adds GPU/noaccel label selection.
modules/local/fibertoolsrs/fire/main.nf Adds GPU/noaccel label selection.
modules/local/deepsomatic/makeexamples/main.nf Adds DeepSomatic makeexamples module.
modules/local/deepsomatic/callvariants/main.nf Adds DeepSomatic callvariants module (GPU/noaccel label selection).
modules/local/deepsomatic/postprocessvariants/main.nf Adds DeepSomatic postprocessvariants module.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread workflows/lrsomatic.nf Outdated
Comment thread subworkflows/local/tumor_only/tumoronly_smallvar.nf
Comment thread subworkflows/local/tumor_only/tumoronly_smallvar.nf
Comment thread subworkflows/local/phasing_haplotyping.nf
Comment thread modules/nf-core/bcftools/isec/main.nf
Comment thread subworkflows/local/phasing_haplotyping.nf Outdated
Comment thread subworkflows/local/phasing_haplotyping.nf Outdated
Comment thread modules/nf-core/bcftools/query/main.nf
Comment thread modules/local/vcfsplit/main.nf Outdated
robert-a-forsyth and others added 7 commits March 24, 2026 13:17
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 24, 2026

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit deefcf9

+| ✅ 208 tests passed       |+
#| ❔  21 tests were ignored |#
!| ❗  34 tests had warnings |!
Details

❗ Test warnings:

  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in README.md: Include a figure that guides the user through the major workflow steps. Many nf-core
  • pipeline_todos - TODO string in nextflow.config: Specify your pipeline's command line flags
  • pipeline_todos - TODO string in nextflow.config: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
  • pipeline_todos - TODO string in meta.yml: #Add a description of the module and list keywords
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
  • schema_description - No description provided in schema for parameter: autocorrelation
  • schema_description - No description provided in schema for parameter: vep_custom
  • schema_description - No description provided in schema for parameter: vep_custom_tbi
  • schema_description - No description provided in schema for parameter: severus_minsupport
  • schema_description - No description provided in schema for parameter: wakhan_chroms
  • local_component_structure - prepare_reference_files.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - deepsomatic.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - phasing_haplotyping.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_annotation.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - small_variant_consensus.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

  • files_exist - File is ignored: CODE_OF_CONDUCT.md
  • files_exist - File is ignored: assets/nf-core-lrsomatic_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-lrsomatic_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-lrsomatic_logo_dark.png
  • files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
  • files_exist - File is ignored: .github/workflows/awstest.yml
  • files_exist - File is ignored: .github/workflows/awsfulltest.yml
  • nextflow_config - Config variable ignored: manifest.name
  • nextflow_config - Config variable ignored: manifest.homePage
  • files_unchanged - File ignored due to lint config: CODE_OF_CONDUCT.md
  • files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
  • files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
  • files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/email_template.txt
  • files_unchanged - File ignored due to lint config: assets/nf-core-lrsomatic_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-lrsomatic_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-lrsomatic_logo_dark.png
  • files_unchanged - File ignored due to lint config: docs/README.md
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/lrsomatic/lrsomatic/.github/workflows/awstest.yml
  • schema_params - schema_params

✅ Tests passed:

Run details

  • nf-core/tools version 3.5.2
  • Run at 2026-03-31 17:02:14

Comment thread modules/local/deepsomatic/callvariants/main.nf
Comment thread modules/local/deepsomatic/makeexamples/main.nf
Comment thread modules/nf-core/bcftools/isec/main.nf
Comment thread modules/nf-core/longphase/haplotag/longphase-haplotag.diff Outdated
Comment thread modules/nf-core/longphase/phase/longphase-phase.diff
Comment thread workflows/lrsomatic.nf
Comment thread workflows/lrsomatic.nf Outdated
Comment thread subworkflows/local/paired/paired_smallvar_germline.nf Outdated
Comment thread subworkflows/local/paired/paired_smallvar_somatic.nf Outdated
Comment thread modules/local/deepsomatic/postprocessvariants/main.nf
@robert-a-forsyth robert-a-forsyth merged commit d2cc09b into dev Mar 31, 2026
7 checks passed
@ljwharbers ljwharbers deleted the deepvariant branch April 22, 2026 14:11
Copilot AI mentioned this pull request May 19, 2026
10 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants