Skip to content

Refactor iGenomes handling to expose params object pipeline-wide #483

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Aug 1, 2025
Merged

Refactor iGenomes handling to expose params object pipeline-wide #483

merged 2 commits into from
Aug 1, 2025

Conversation

@LeonHafner
Copy link
Contributor

@LeonHafner LeonHafner commented Jul 26, 2025 •
edited
Loading

This PR builds on: #469

It moves the getGenomeAttribute function to main.nf, making igenomes-derived parameters available pipeline-wide rather than just within main.nf as described by @nictru in the linked PR. This eliminates the need to manually pass parameters to subworkflows, simplifying the addition of new iGenomes attributes. Currently, only fasta, gtf, and star_index are supported.

Additionally I had to move the imports below the igenomes fetching, as the getGenomeAttribute is otherwise not available.

The changes align with implementations in nf-core/tfactivity (nf-core/tfactivity#30)
and nf-core/rnaseq (nf-core/rnaseq/main.nf#L184).

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/scrnaseq branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@LeonHafner LeonHafner self-assigned this Jul 26, 2025
@nf-core-bot
Copy link
Member

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.2.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

@github-actions
Copy link

github-actions bot commented Jul 26, 2025 •
edited
Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit bb9001d

+| ✅ 215 tests passed       |+
#| ❔   4 tests were ignored |#
!| ❗  21 tests had warnings |!

❗ Test warnings:

  • pipeline_todos - TODO string in ro-crate-metadata.json: "description": "

    \n \n <source media="(prefers-color-scheme: dark)" srcset="docs/images/nf-core-scrnaseq_logo_dark.png">\n <img alt="nf-core/scrnaseq" src="docs/images/nf-core-scrnaseq_logo_light.png">\n \n

    \n\nGitHub Actions CI Status\nGitHub Actions Linting StatusAWS CICite with Zenodo\nnf-test\n\nNextflow\nrun with conda\nrun with docker\nrun with singularity\nLaunch on Seqera Platform\n\nGet help on SlackFollow on TwitterFollow on MastodonWatch on YouTube\n\n## Introduction\n\nnf-core/scrnaseq is a bioinformatics pipeline that ...\n\n TODO nf-core:\n Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the\n major pipeline sections and the types of output it produces. You're giving an overview to someone new\n to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction\n\n\n Include a figure that guides the user through the major workflow steps. Many nf-core\n workflows use the "tube map" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples. \n Fill in short bullet-pointed list of the default steps in the pipeline 1. Read QC (FastQC)2. Present QC for raw reads (MultiQC)\n\n## Usage\n\n> [!NOTE]\n> If you are new to Nextflow and nf-core, please refer to this page on how to set-up Nextflow. Make sure to test your setup with -profile test before running the workflow on actual data.\n\n Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.\n Explain what rows and columns represent. For instance (please edit as appropriate):\n\nFirst, prepare a samplesheet with your input data that looks as follows:\n\nsamplesheet.csv:\n\ncsv\nsample,fastq_1,fastq_2\nCONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz\n\n\nEach row represents a fastq file (single-end) or a pair of fastq files (paired end).\n\n\n\nNow, you can run the pipeline using:\n\n update the following command to include all required parameters for a minimal example \n\nbash\nnextflow run nf-core/scrnaseq \\\n -profile <docker/singularity/.../institute> \\\n --input samplesheet.csv \\\n --outdir <OUTDIR>\n\n\n> [!WARNING]\n> Please provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.\n\nFor more details and further functionality, please refer to the usage documentation and the parameter documentation.\n\n## Pipeline output\n\nTo see the results of an example test run with a full size dataset refer to the results tab on the nf-core website pipeline page.\nFor more details about the output files and reports, please refer to the\noutput documentation.\n\n## Credits\n\nnf-core/scrnaseq was originally written by Bailey PJ, Botvinnik O, Marques de Almeida F, Peltzer A, Sturm G.\n\nWe thank the following people for their extensive assistance in the development of this pipeline:\n\n If applicable, make list of people who have also contributed \n\n## Contributions and Support\n\nIf you would like to contribute to this pipeline, please see the contributing guidelines.\n\nFor further information or help, don't hesitate to get in touch on the Slack #scrnaseq channel (you can join with this invite).\n\n## Citations\n\n Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. \n If you use nf-core/scrnaseq for your analysis, please cite it using the following doi: 10.5281/zenodo.XXXXXX \n\n Add bibliography of tools and data used in your pipeline \n\nAn extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.\n\nYou can cite the nf-core publication as follows:\n\n> The nf-core framework for community-curated bioinformatics pipelines.\n>\n> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.\n>\n> Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.\n",
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • schema_description - No description provided in schema for parameter: skip_emptydrops
  • local_component_structure - gffread_transcriptome.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - star_align.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - alevinqc.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - gtf_gene_filter.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - concat_h5ad.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - anndatar_convert.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - mtx_to_h5ad.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - parse_cellrangermulti_samplesheet.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
  • local_component_structure - h5ad_conversion.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - align_cellrangerarc.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - kallisto_bustools.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - starsolo.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - simpleaf.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - fastqc.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - align_cellrangermulti.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - align_cellranger.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 3.2.1
  • Run at 2025-07-26 13:38:14

@LeonHafner LeonHafner requested review from fmalmeida and nictru July 27, 2025 20:15
@LeonHafner LeonHafner marked this pull request as ready for review July 27, 2025 20:15
@grst
Copy link
Member

grst commented Jul 28, 2025

Hi @LeonHafner, thanks for the PR. Implementationwise this looks good.

I'm not sure though if it is desirable to not pass parameters to the subworkflow separately. If all params are explicitly passed to the subworkflow (I'm aware this is currently anyway not the case), it would make it easier to import that subworkflow as-is into another pipeline.

Maybe @nictru is aware if there's any official nf-core recommendations about parameter handling at the workflow level?

@LeonHafner
Copy link
Contributor Author

I kicked off a discussion in the nf-core slack. As of writing this comment the opinion seems to be that it is still okay to directly use the params object in workflows/scrnaseq.nf as most other pipelines do as well. See here and here for nf-core/sarek and nf-core/rnaseq, they use it about 100 times in workflows/<PIPELINE_NAME>/main.nf.

However there will be a new nextflow guideline which recommends using params only in the entry workflow. This is not enforced by nf-core yet but might be in the future. If we want to future proof we have to pass every parameter from main.nf to workflows/scrnaseq.nf and avoid using params in the latter.

I think it’s up to the pipeline maintainer to decide.

grst
grst approved these changes Aug 1, 2025
View reviewed changes
Copy link
Member

@grst grst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, then let's get this in.
If we want to isolate the workflow better, we anyway need to refactor a couple of other things.

@grst grst merged commit c1aefc4 into dev Aug 1, 2025
17 checks passed
@grst grst deleted the fix_igenomes_params branch August 1, 2025 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@grst grst grst approved these changes

@fmalmeida fmalmeida Awaiting requested review from fmalmeida

@nictru nictru Awaiting requested review from nictru

Assignees

@LeonHafner LeonHafner

Labels

Ready for review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@LeonHafner @nf-core-bot @grst