Fix some bugs and add upset plots

[nictru]

Hey, this PR addresses 4 things:

1. Exclude overlong circRNAs from quantification as psirc-quant throws an error otherwise
2. Use only the circRNAs which pass the detection majority vote in quantification
3. Add upset plot functionality for comparing detection tools and add to multiqc
4. Add TPM to the RDS output of the quantification subworkflow

[github-actions]

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 86af7b2

+| ✅ 260 tests passed       |+
!| ❗   1 tests had warnings |!

❗ Test warnings:

• readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).

✅ Tests passed:

• files_exist - File found: .gitattributes
• files_exist - File found: .gitignore
• files_exist - File found: .nf-core.yml
• files_exist - File found: .editorconfig
• files_exist - File found: .prettierignore
• files_exist - File found: .prettierrc.yml
• files_exist - File found: CHANGELOG.md
• files_exist - File found: CITATIONS.md
• files_exist - File found: CODE_OF_CONDUCT.md
• files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
• files_exist - File found: nextflow_schema.json
• files_exist - File found: nextflow.config
• files_exist - File found: README.md
• files_exist - File found: .github/.dockstore.yml
• files_exist - File found: .github/CONTRIBUTING.md
• files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
• files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
• files_exist - File found: .github/workflows/branch.yml
• files_exist - File found: .github/workflows/ci.yml
• files_exist - File found: .github/workflows/linting_comment.yml
• files_exist - File found: .github/workflows/linting.yml
• files_exist - File found: assets/email_template.html
• files_exist - File found: assets/email_template.txt
• files_exist - File found: assets/sendmail_template.txt
• files_exist - File found: assets/nf-core-circrna_logo_light.png
• files_exist - File found: conf/modules.config
• files_exist - File found: conf/test.config
• files_exist - File found: conf/test_full.config
• files_exist - File found: docs/images/nf-core-circrna_logo_light.png
• files_exist - File found: docs/images/nf-core-circrna_logo_dark.png
• files_exist - File found: docs/output.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/usage.md
• files_exist - File found: main.nf
• files_exist - File found: assets/multiqc_config.yml
• files_exist - File found: conf/base.config
• files_exist - File found: conf/igenomes.config
• files_exist - File found: .github/workflows/awstest.yml
• files_exist - File found: .github/workflows/awsfulltest.yml
• files_exist - File found: modules.json
• files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
• files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
• files_exist - File not found check: .github/workflows/push_dockerhub.yml
• files_exist - File not found check: .markdownlint.yml
• files_exist - File not found check: .nf-core.yaml
• files_exist - File not found check: .yamllint.yml
• files_exist - File not found check: bin/markdown_to_html.r
• files_exist - File not found check: conf/aws.config
• files_exist - File not found check: docs/images/nf-core-circrna_logo.png
• files_exist - File not found check: lib/Checks.groovy
• files_exist - File not found check: lib/Completion.groovy
• files_exist - File not found check: lib/NfcoreTemplate.groovy
• files_exist - File not found check: lib/Utils.groovy
• files_exist - File not found check: lib/Workflow.groovy
• files_exist - File not found check: lib/WorkflowMain.groovy
• files_exist - File not found check: lib/WorkflowCircrna.groovy
• files_exist - File not found check: parameters.settings.json
• files_exist - File not found check: pipeline_template.yml
• files_exist - File not found check: Singularity
• files_exist - File not found check: lib/nfcore_external_java_deps.jar
• files_exist - File not found check: .travis.yml
• nextflow_config - Config variable found: manifest.name
• nextflow_config - Config variable found: manifest.nextflowVersion
• nextflow_config - Config variable found: manifest.description
• nextflow_config - Config variable found: manifest.version
• nextflow_config - Config variable found: manifest.homePage
• nextflow_config - Config variable found: timeline.enabled
• nextflow_config - Config variable found: trace.enabled
• nextflow_config - Config variable found: report.enabled
• nextflow_config - Config variable found: dag.enabled
• nextflow_config - Config variable found: process.cpus
• nextflow_config - Config variable found: process.memory
• nextflow_config - Config variable found: process.time
• nextflow_config - Config variable found: params.outdir
• nextflow_config - Config variable found: params.input
• nextflow_config - Config variable found: params.validationShowHiddenParams
• nextflow_config - Config variable found: params.validationSchemaIgnoreParams
• nextflow_config - Config variable found: manifest.mainScript
• nextflow_config - Config variable found: timeline.file
• nextflow_config - Config variable found: trace.file
• nextflow_config - Config variable found: report.file
• nextflow_config - Config variable found: dag.file
• nextflow_config - Config variable (correctly) not found: params.nf_required_version
• nextflow_config - Config variable (correctly) not found: params.container
• nextflow_config - Config variable (correctly) not found: params.singleEnd
• nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
• nextflow_config - Config variable (correctly) not found: params.name
• nextflow_config - Config variable (correctly) not found: params.enable_conda
• nextflow_config - Config timeline.enabled had correct value: true
• nextflow_config - Config report.enabled had correct value: true
• nextflow_config - Config trace.enabled had correct value: true
• nextflow_config - Config dag.enabled had correct value: true
• nextflow_config - Config manifest.name began with nf-core/
• nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
• nextflow_config - Config dag.file ended with .html
• nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
• nextflow_config - Config manifest.version ends in dev: dev
• nextflow_config - Config params.custom_config_version is set to master
• nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
• nextflow_config - Lines for loading custom profiles found
• nextflow_config - nextflow.config contains configuration profile test
• nextflow_config - Config default value correct: params.tool= circexplorer2
• nextflow_config - Config default value correct: params.module= circrna_discovery
• nextflow_config - Config default value correct: params.bsj_reads= 0
• nextflow_config - Config default value correct: params.tool_filter= 0
• nextflow_config - Config default value correct: params.duplicates_fun= mean
• nextflow_config - Config default value correct: params.save_intermediates= false
• nextflow_config - Config default value correct: params.exon_boundary= 200
• nextflow_config - Config default value correct: params.sjdboverhang= 100
• nextflow_config - Config default value correct: params.chimJunctionOverhangMin= 10
• nextflow_config - Config default value correct: params.alignSJDBoverhangMin= 10
• nextflow_config - Config default value correct: params.limitSjdbInsertNsj= 1000000
• nextflow_config - Config default value correct: params.chimSegmentMin= 10
• nextflow_config - Config default value correct: params.seglen= 25
• nextflow_config - Config default value correct: params.min_intron= 20
• nextflow_config - Config default value correct: params.max_intron= 1000000
• nextflow_config - Config default value correct: params.min_map_len= 40
• nextflow_config - Config default value correct: params.min_fusion_distance= 200
• nextflow_config - Config default value correct: params.bootstrap_samples= 30
• nextflow_config - Config default value correct: params.save_unaligned= false
• nextflow_config - Config default value correct: params.save_reference= true
• nextflow_config - Config default value correct: params.hisat2_build_memory= 200.GB
• nextflow_config - Config default value correct: params.skip_trimming= false
• nextflow_config - Config default value correct: params.save_trimmed= false
• nextflow_config - Config default value correct: params.skip_fastqc= false
• nextflow_config - Config default value correct: params.min_trimmed_reads= 10000
• nextflow_config - Config default value correct: params.custom_config_version= master
• nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
• nextflow_config - Config default value correct: params.max_cpus= 50
• nextflow_config - Config default value correct: params.max_memory= 300.GB
• nextflow_config - Config default value correct: params.max_time= 240.h
• nextflow_config - Config default value correct: params.publish_dir_mode= copy
• nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
• nextflow_config - Config default value correct: params.validate_params= true
• files_unchanged - .gitattributes matches the template
• files_unchanged - .prettierrc.yml matches the template
• files_unchanged - CODE_OF_CONDUCT.md matches the template
• files_unchanged - LICENSE matches the template
• files_unchanged - .github/.dockstore.yml matches the template
• files_unchanged - .github/CONTRIBUTING.md matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
• files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
• files_unchanged - .github/workflows/branch.yml matches the template
• files_unchanged - .github/workflows/linting_comment.yml matches the template
• files_unchanged - .github/workflows/linting.yml matches the template
• files_unchanged - assets/email_template.html matches the template
• files_unchanged - assets/email_template.txt matches the template
• files_unchanged - assets/sendmail_template.txt matches the template
• files_unchanged - assets/nf-core-circrna_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-circrna_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-circrna_logo_dark.png matches the template
• files_unchanged - docs/README.md matches the template
• files_unchanged - .gitignore matches the template
• files_unchanged - .prettierignore matches the template
• actions_ci - '.github/workflows/ci.yml' is triggered on expected events
• actions_ci - '.github/workflows/ci.yml' checks minimum NF version
• actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
• readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
• pipeline_todos - No TODO strings found
• pipeline_name_conventions - Name adheres to nf-core convention
• template_strings - Did not find any Jinja template strings (370 files)
• schema_lint - Schema lint passed
• schema_lint - Schema title + description lint passed
• schema_lint - Input mimetype lint passed: 'text/csv'
• schema_params - Schema matched params returned from nextflow config
• system_exit - No System.exit calls found
• actions_schema_validation - Workflow validation passed: awstest.yml
• actions_schema_validation - Workflow validation passed: branch.yml
• actions_schema_validation - Workflow validation passed: ci.yml
• actions_schema_validation - Workflow validation passed: clean-up.yml
• actions_schema_validation - Workflow validation passed: fix-linting.yml
• actions_schema_validation - Workflow validation passed: release-announcements.yml
• actions_schema_validation - Workflow validation passed: linting.yml
• actions_schema_validation - Workflow validation passed: awsfulltest.yml
• actions_schema_validation - Workflow validation passed: linting_comment.yml
• actions_schema_validation - Workflow validation passed: download_pipeline.yml
• merge_markers - No merge markers found in pipeline files
• modules_json - Only installed modules found in modules.json
• multiqc_config - assets/multiqc_config.yml found and not ignored.
• multiqc_config - assets/multiqc_config.yml contains report_section_order
• multiqc_config - assets/multiqc_config.yml contains export_plots
• multiqc_config - assets/multiqc_config.yml contains report_comment
• multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
• multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
• multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
• modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
• base_config - conf/base.config found and not ignored.
• modules_config - conf/modules.config found and not ignored.
• modules_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/modules.config and Nextflow scripts.
• modules_config - CLEAN_FASTA found in conf/modules.config and Nextflow scripts.
• modules_config - SEQKIT_SPLIT found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE_BUILD found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE2_BUILD found in conf/modules.config and Nextflow scripts.
• modules_config - BWA_INDEX found in conf/modules.config and Nextflow scripts.
• modules_config - HISAT2_EXTRACTSPLICESITES found in conf/modules.config and Nextflow scripts.
• modules_config - HISAT2_BUILD found in conf/modules.config and Nextflow scripts.
• modules_config - STAR_GENOMEGENERATE found in conf/modules.config and Nextflow scripts.
• modules_config - SEGEMEHL_INDEX found in conf/modules.config and Nextflow scripts.
• modules_config - SEGEMEHL_ALIGN found in conf/modules.config and Nextflow scripts.
• modules_config - SEGEMEHL_FILTER found in conf/modules.config and Nextflow scripts.
• modules_config - STAR_1ST_PASS found in conf/modules.config and Nextflow scripts.
• modules_config - STAR_SJDB found in conf/modules.config and Nextflow scripts.
• modules_config - STAR_2ND_PASS found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCEXPLORER2_REF found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCEXPLORER2_PAR found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCEXPLORER2_ANN found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCEXPLORER2_FLT found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCRNA_FINDER_FILTER found in conf/modules.config and Nextflow scripts.
• modules_config - FIND_CIRC_ALIGN found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_VIEW found in conf/modules.config and Nextflow scripts.
• modules_config - FIND_CIRC_ANCHORS found in conf/modules.config and Nextflow scripts.
• modules_config - FIND_CIRC found in conf/modules.config and Nextflow scripts.
• modules_config - FIND_CIRC_FILTER found in conf/modules.config and Nextflow scripts.
• modules_config - CIRIQUANT found in conf/modules.config and Nextflow scripts.
• modules_config - CIRIQUANT_FILTER found in conf/modules.config and Nextflow scripts.
• modules_config - DCC_MATE1_1ST_PASS found in conf/modules.config and Nextflow scripts.
• modules_config - DCC_MATE1_SJDB found in conf/modules.config and Nextflow scripts.
• modules_config - DCC_MATE1_2ND_PASS found in conf/modules.config and Nextflow scripts.
• modules_config - DCC_MATE2_1ST_PASS found in conf/modules.config and Nextflow scripts.
• modules_config - DCC_MATE2_SJDB found in conf/modules.config and Nextflow scripts.
• modules_config - DCC_MATE2_2ND_PASS found in conf/modules.config and Nextflow scripts.
• modules_config - DCC found in conf/modules.config and Nextflow scripts.
• modules_config - MAPSPLICE_REFERENCE found in conf/modules.config and Nextflow scripts.
• modules_config - MAPSPLICE_ALIGN found in conf/modules.config and Nextflow scripts.
• modules_config - MAPSPLICE_PARSE found in conf/modules.config and Nextflow scripts.
• modules_config - MAPSPLICE_ANNOTATE found in conf/modules.config and Nextflow scripts.
• modules_config - MAPSPLICE_FILTER found in conf/modules.config and Nextflow scripts.
• modules_config - ANNOTATION found in conf/modules.config and Nextflow scripts.
• modules_config - UPSET_SAMPLES found in conf/modules.config and Nextflow scripts.
• modules_config - UPSET_ALL found in conf/modules.config and Nextflow scripts.
• modules_config - INTERSECT_ANNOTATION found in conf/modules.config and Nextflow scripts.
• modules_config - COMBINE_ANNOTATION_BEDS found in conf/modules.config and Nextflow scripts.
• modules_config - COMBINE_ANNOTATION_GTFS found in conf/modules.config and Nextflow scripts.
• modules_config - REMOVE_SCORE_STRAND found in conf/modules.config and Nextflow scripts.
• modules_config - FASTA found in conf/modules.config and Nextflow scripts.
• modules_config - COUNTS_COMBINED found in conf/modules.config and Nextflow scripts.
• modules_config - COMBINE_TRANSCRIPTOME_GTFS found in conf/modules.config and Nextflow scripts.
• modules_config - EXCLUDE_OVERLONG_TRANSCRIPTS found in conf/modules.config and Nextflow scripts.
• modules_config - MARK_CIRCULAR found in conf/modules.config and Nextflow scripts.
• modules_config - TARGETSCAN_DATABASE found in conf/modules.config and Nextflow scripts.
• modules_config - TARGETSCAN found in conf/modules.config and Nextflow scripts.
• modules_config - MIRANDA found in conf/modules.config and Nextflow scripts.
• modules_config - MIRNA_TARGETS found in conf/modules.config and Nextflow scripts.
• modules_config - HISAT2_ALIGN found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_SORT found in conf/modules.config and Nextflow scripts.
• modules_config - STRINGTIE_STRINGTIE found in conf/modules.config and Nextflow scripts.
• modules_config - STRINGTIE_PREPDE found in conf/modules.config and Nextflow scripts.
• modules_config - DESEQ2_DIFFERENTIAL_EXPRESSION found in conf/modules.config and Nextflow scripts.
• modules_config - PARENT_GENE found in conf/modules.config and Nextflow scripts.
• modules_config - PREPARE_CLR_TEST found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCTEST found in conf/modules.config and Nextflow scripts.
• nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
• nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 2.14.1

Run details

• nf-core/tools version 2.14.1
• Run at 2024-05-10 16:47:02

[erinyoung]

I have no changes to recommend.