Clean up differential expression workflow #147

nictru · 2024-06-11T14:00:25Z

Implements the changes proposed in this slack message
Moves most scripts from the bin directory to the corresponding modules using the template syntax
Deletes some artifact files
Simplifies the circtest implementation

github-actions · 2024-06-11T14:02:42Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 4974eaf

+| ✅ 218 tests passed       |+
!| ❗   1 tests had warnings |!

❗ Test warnings:

readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-circrna_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-circrna_logo_light.png
files_exist - File found: docs/images/nf-core-circrna_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-circrna_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowCircrna.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.validationShowHiddenParams
nextflow_config - Config variable found: params.validationSchemaIgnoreParams
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: dev
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.tool= circexplorer2
nextflow_config - Config default value correct: params.module= circrna_discovery
nextflow_config - Config default value correct: params.bsj_reads= 0
nextflow_config - Config default value correct: params.tool_filter= 0
nextflow_config - Config default value correct: params.duplicates_fun= mean
nextflow_config - Config default value correct: params.save_intermediates= false
nextflow_config - Config default value correct: params.exon_boundary= 200
nextflow_config - Config default value correct: params.sjdboverhang= 100
nextflow_config - Config default value correct: params.chimJunctionOverhangMin= 10
nextflow_config - Config default value correct: params.alignSJDBoverhangMin= 10
nextflow_config - Config default value correct: params.limitSjdbInsertNsj= 1000000
nextflow_config - Config default value correct: params.chimSegmentMin= 10
nextflow_config - Config default value correct: params.seglen= 25
nextflow_config - Config default value correct: params.min_intron= 20
nextflow_config - Config default value correct: params.max_intron= 1000000
nextflow_config - Config default value correct: params.min_map_len= 40
nextflow_config - Config default value correct: params.min_fusion_distance= 200
nextflow_config - Config default value correct: params.bootstrap_samples= 30
nextflow_config - Config default value correct: params.save_unaligned= false
nextflow_config - Config default value correct: params.save_reference= true
nextflow_config - Config default value correct: params.hisat2_build_memory= 200.GB
nextflow_config - Config default value correct: params.skip_trimming= false
nextflow_config - Config default value correct: params.save_trimmed= false
nextflow_config - Config default value correct: params.skip_fastqc= false
nextflow_config - Config default value correct: params.min_trimmed_reads= 10000
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.max_cpus= 50
nextflow_config - Config default value correct: params.max_memory= 300.GB
nextflow_config - Config default value correct: params.max_time= 240.h
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-circrna_logo_light.png matches the template
files_unchanged - docs/images/nf-core-circrna_logo_light.png matches the template
files_unchanged - docs/images/nf-core-circrna_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
pipeline_todos - No TODO strings found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (378 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
modules_config - conf/modules.config found and not ignored.
modules_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/modules.config and Nextflow scripts.
modules_config - CLEAN_FASTA found in conf/modules.config and Nextflow scripts.
modules_config - SEQKIT_SPLIT found in conf/modules.config and Nextflow scripts.
modules_config - BOWTIE_BUILD found in conf/modules.config and Nextflow scripts.
modules_config - BOWTIE2_BUILD found in conf/modules.config and Nextflow scripts.
modules_config - BWA_INDEX found in conf/modules.config and Nextflow scripts.
modules_config - HISAT2_EXTRACTSPLICESITES found in conf/modules.config and Nextflow scripts.
modules_config - HISAT2_BUILD found in conf/modules.config and Nextflow scripts.
modules_config - STAR_GENOMEGENERATE found in conf/modules.config and Nextflow scripts.
modules_config - FILTER_BSJS found in conf/modules.config and Nextflow scripts.
modules_config - ANNOTATION found in conf/modules.config and Nextflow scripts.
modules_config - UPSET_SAMPLES found in conf/modules.config and Nextflow scripts.
modules_config - UPSET_ALL found in conf/modules.config and Nextflow scripts.
modules_config - ADD_BACKSPLICE found in conf/modules.config and Nextflow scripts.
modules_config - COMBINE_TRANSCRIPTOME_GTFS found in conf/modules.config and Nextflow scripts.
modules_config - EXCLUDE_OVERLONG_TRANSCRIPTS found in conf/modules.config and Nextflow scripts.
modules_config - MARK_CIRCULAR found in conf/modules.config and Nextflow scripts.
modules_config - TARGETSCAN_DATABASE found in conf/modules.config and Nextflow scripts.
modules_config - TARGETSCAN found in conf/modules.config and Nextflow scripts.
modules_config - MIRANDA found in conf/modules.config and Nextflow scripts.
modules_config - MIRNA_TARGETS found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 2.14.1

Run details

nf-core/tools version 2.14.1
Run at 2024-06-14 10:04:46

mahesh-panchal

LGTM generally.

README.md

docs/usage.md

modules/local/annotation/main.nf

modules/local/circtest/circtest/main.nf

Co-authored-by: Mahesh Binzer-Panchal <mahesh.binzer-panchal@nbis.se>

nictru added 15 commits June 10, 2024 12:29

Add new STATISTICAL_TESTS subworkflow

8150c43

Implement CIRCTEST_PREPARE

7ef6b03

Implement circTest

5893d6d

Fix circtest problems

9c5906d

Re-enable circtest plots

21236c8

Fix circtest plot problem

69f82c0

Remove artifacts of old differential expression analysis

d2839a3

Simplify annotation module

b67e980

Switch annotation to template syntax

6fdc23d

Remove unused combine_quantification script

a87569b

Switch merge_samples to template syntax

fa7cc09

Switch to template syntax in merge_tools

f3e63fd

Remove custom DESeq2 module

4a49a48

Remove DIFFERENTIAL_EXPRESSION traces

ec9198b

Remove unwanted biotypes

3441048

nictru linked an issue Jun 11, 2024 that may be closed by this pull request

Decide how to proceed with the differential expression subworkflow #144

Closed

nictru added 3 commits June 11, 2024 16:04

Fix linting problems

abd9dcd

Prettier

ffbada6

Remove traces of differential_expression in docs

2d259c3

mahesh-panchal approved these changes Jun 14, 2024

View reviewed changes

README.md Outdated Show resolved Hide resolved

docs/usage.md Outdated Show resolved Hide resolved

modules/local/annotation/main.nf Show resolved Hide resolved

modules/local/circtest/circtest/main.nf Outdated Show resolved Hide resolved

nictru and others added 3 commits June 14, 2024 10:21

Update modules/local/circtest/circtest/main.nf

00d5f0a

Co-authored-by: Mahesh Binzer-Panchal <mahesh.binzer-panchal@nbis.se>

Clarify docs

4f37a03

Update gnu/sort

4974eaf

nictru merged commit ad02b05 into dev Jun 14, 2024
4 checks passed

nictru deleted the 144-differential-expression branch June 14, 2024 10:31

nictru mentioned this pull request Jun 14, 2024

Deprecate module parameter #148

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up differential expression workflow #147

Clean up differential expression workflow #147

nictru commented Jun 11, 2024 •

edited

Loading

github-actions bot commented Jun 11, 2024 •

edited

Loading

❗ Test warnings:

✅ Tests passed:

Run details

mahesh-panchal left a comment

Clean up differential expression workflow #147

Clean up differential expression workflow #147

Conversation

nictru commented Jun 11, 2024 • edited Loading

github-actions bot commented Jun 11, 2024 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

✅ Tests passed:

Run details

mahesh-panchal left a comment

Choose a reason for hiding this comment

nictru commented Jun 11, 2024 •

edited

Loading

github-actions bot commented Jun 11, 2024 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️