Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DSL2 conversion #73

Merged
merged 65 commits into from Dec 15, 2021
Merged

DSL2 conversion #73

merged 65 commits into from Dec 15, 2021

Conversation

ggabernet
Copy link
Member

@ggabernet ggabernet commented Oct 28, 2021

  • Pipeline DSL2 conversion
  • Updated template

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
    • If you've added a new tool - add to the software_versions process and a regex to scrape_software_versions.py
    • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
    • If necessary, also make a PR on the nf-core/scrnaseq branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint .).
  • Ensure the test suite passes (nextflow run . -profile test,docker).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@ggabernet ggabernet marked this pull request as ready for review October 29, 2021 14:04
@github-actions
Copy link

github-actions bot commented Oct 29, 2021

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 51d4ff4

+| ✅ 137 tests passed       |+
!| ❗  14 tests had warnings |!

❗ Test warnings:

  • readme - README did not have a Nextflow minimum version mentioned in Quick Start section.
  • pipeline_todos - TODO string in README.md: Write a 1-2 sentence summary of what data the pipeline is for and what it does
  • pipeline_todos - TODO string in README.md: Add full-sized test dataset and amend the paragraph below if applicable
  • pipeline_todos - TODO string in WorkflowMain.groovy: Add Zenodo DOI for pipeline after first release
  • pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
  • pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
  • pipeline_todos - TODO string in scrnaseq.nf: Add all file path parameters for the pipeline to the list below
  • pipeline_todos - TODO string in check_samplesheet.py: Update the check_samplesheet function
  • pipeline_todos - TODO string in check_samplesheet.py: Update the column names for the input samplesheet
  • pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
  • schema_description - Ungrouped param in schema: skip_multiqc
  • schema_description - Ungrouped param in schema: seq_center
  • schema_description - No description provided in schema for parameter: protocol
  • schema_description - No description provided in schema for parameter: star_ignore_sjdbgtf

✅ Tests passed:

Run details

  • nf-core/tools version 2.1
  • Run at 2021-11-02 08:09:32

Copy link
Contributor

@edmundmiller edmundmiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few things, otherwise looks good!

bin/check_samplesheet.py Show resolved Hide resolved
conf/test_full.config Show resolved Hide resolved
modules/local/kallistobustools_count.nf Outdated Show resolved Hide resolved
Co-authored-by: Edmund Miller <edmund.a.miller@protonmail.com>
Copy link
Member

@grst grst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a few comments, but I don't see any major issues!

README.md Outdated Show resolved Hide resolved
The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker containers making installation trivial and results highly reproducible.
The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from [nf-core/modules](https://github.com/nf-core/modules) in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!

<!-- TODO nf-core: Add full-sized test dataset and amend the paragraph below if applicable -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this todo be removed?

mode: 'copy',
saveAs: { filename -> saveFiles(filename:filename, options:params.options, publish_dir:getSoftwareName(task.process), publish_id:'') }

container "streitlab/custom-nf-modules-cellranger:latest"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we had that in the nf core namespace now?

Comment on lines +29 to +39
if("${gtf}".endsWith('.gz')){
name = "${gtf.baseName}"
unzip = "gunzip -f ${gtf}"
} else {
unzip = ""
name = "${gtf}"
}
"""
$unzip
cat $name | t2g.py --use_version > transcripts_to_genes.txt
"""
Copy link
Member

@grst grst Oct 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if("${gtf}".endsWith('.gz')){
name = "${gtf.baseName}"
unzip = "gunzip -f ${gtf}"
} else {
unzip = ""
name = "${gtf}"
}
"""
$unzip
cat $name | t2g.py --use_version > transcripts_to_genes.txt
"""
"""
gzip -cdf $gtf | t2g.py --use_version > transcripts_to_genes.txt
"""

Comment on lines +40 to +43
if (meta.single_end) {
array = [ meta, [ file(row.fastq_1, checkIfExists: true) ] ]
} else {
array = [ meta, [ file(row.fastq_1, checkIfExists: true), file(row.fastq_2, checkIfExists: true) ] ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have single end in this pipeline, do we?

Copy link
Member

@apeltzer apeltzer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing really found on my side - looks good.

Co-authored-by: Gregor Sturm <mail@gregor-sturm.de>
@grst
Copy link
Member

grst commented Dec 14, 2021

Seems some tests have been renamed - Can someone with write access please update the required tests for this repo?

@ggabernet
Copy link
Member Author

I'll be merging this and work on the next template update in a separate PR

@ggabernet ggabernet merged commit 672864b into dev Dec 15, 2021
@grst grst deleted the dsl2 branch December 29, 2021 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

None yet

7 participants