Added logic for transformation method #206

WackerO · 2023-06-14T10:42:32Z

This PR adds a threshold for size factor variance (0.05) above which the pipeline will automatically use rlog transformation even if the user requests vst (can be overridden by user).
Additionally, heatmaps are now saved inside a trycatch (especially png() often throws an error), meanSdPlot has now correct plot titles depending on rlog/vst and < or > comparisons (is pval < 0.05) are now always <= or >= (is pval <= 0.05) as this is also what the gost() function does.

@qbicStefanC

PR checklist

github-actions · 2023-06-14T10:44:04Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 646ffa4

+| ✅ 125 tests passed       |+
#| ❔  27 tests were ignored |#
!| ❗  10 tests had warnings |!

❗ Test warnings:

readme - README did not have a Nextflow minimum version badge.
pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your prefered methods description, e.g. add publication citation for this pipeline
pipeline_todos - TODO string in rnadeseq.nf: Add all file path parameters for the pipeline to the list below
pipeline_todos - TODO string in WorkflowMain.groovy: Add Zenodo DOI for pipeline after first release
pipeline_todos - TODO string in base.config: Check the defaults for all processes
pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
schema_lint - Parameter input is not defined in the correct subschema (input_output_options)
system_exit - System.exit in WorkflowRnadeseq.groovy: System.exit(1) [line 67]
system_exit - System.exit in WorkflowMain.groovy: System.exit(1) [line 85]

❔ Tests ignored:

files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
files_exist - File is ignored: .github/workflows/awsfulltest.yml
files_exist - File is ignored: .github/workflows/awstest.yml
files_exist - File is ignored: .github/workflows/push_dockerhub.yml
files_exist - File is ignored: assets/multiqc_config.yaml
files_exist - File is ignored: assets/nf-core-qbic-pipelines/rnadeseq_logo_light.png
files_exist - File is ignored: bin/markdown_to_html.r
files_exist - File is ignored: conf/test_full.config
files_exist - File is ignored: docs/images/nf-core-qbic-pipelines/rnadeseq_logo_dark.png
files_exist - File is ignored: docs/images/nf-core-qbic-pipelines/rnadeseq_logo_light.png
files_exist - File is ignored: lib/WorkflowQbic-pipelines/rnadeseq.groovy
nextflow_config - Config variable ignored: manifest.name
nextflow_config - Config variable ignored: manifest.homePage
nextflow_config - Config variable ignored: params.input
files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/config.yml
files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/feature_request.yml
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
files_unchanged - File ignored due to lint config: assets/email_template.html
files_unchanged - File ignored due to lint config: assets/email_template.txt
files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
files_unchanged - File ignored due to lint config: docs/README.md
files_unchanged - File ignored due to lint config: .gitignore or .prettierignore or pyproject.toml

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-rnadeseq_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: docs/images/nf-core-rnadeseq_logo_light.png
files_exist - File found: docs/images/nf-core-rnadeseq_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: lib/WorkflowRnadeseq.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-rnadeseq_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: '2.2dev'
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - lib/NfcoreTemplate.groovy matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Zenodo placeholder was replaced with DOI.
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (132 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_params - Schema matched params returned from nextflow config
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: push_github_container_reg.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: branch.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.8
Run at 2023-06-20 11:38:05

…o <= and => for logFC and pval

SusiJo · 2023-06-20T06:30:44Z

assets/RNAseq_report.Rmd


 # Write cds assay table to file
-write.table(counts(cds, normalized=T), paste("differential_gene_expression/gene_counts_tables/deseq2_table.tsv", sep=""), append=F, quote = F, sep = "\t", eol = "\n", na = "NA", dec = ".", row.names = T, col.names = T, qmethod = c("escape", "double"))
+write.table(counts(cds, normalized=T), paste0("differential_gene_expression/gene_counts_tables/deseq2_table.tsv"), append=F, quote = F, sep = "\t", eol = "\n", na = "NA", dec = ".", row.names = T, col.names = T, qmethod = c("escape", "double"))


Could the output be named more explicitly? Like deseq2_normalized_gene_counts.tsv ?

Renaming to deseq2_library_scaled_gene_counts.tsv to not produce confusion with the rlog/vst normalization stuff that is done later. I'll also add this file to the file explanation in the expand/collapse box

assets/RNAseq_report.Rmd

SusiJo · 2023-06-20T07:12:53Z

assets/RNAseq_report.Rmd

@@ -1808,18 +1835,41 @@ for (file in contrast_files){
                    mat$gene_name <- NULL
                    mat <- data.matrix(mat)

-                    # Skip the heatmaps for CI tests
+                    # Skip the heatmaps for CI tests because especially png() works only very unreliably for the heatmaps for some reason


Unreliably in terms of md5sums?

No, unfortunately sometimes it also just does not work. I.e. Unable to start png device is thrown and the pipeline stops.

SusiJo · 2023-06-20T07:22:14Z

assets/RNAseq_report.Rmd

@@ -2140,10 +2187,12 @@ as.character(params$revision),
 "` pipeline [^5], which was written using the nf-core template [@ewels2020nf]. For differential expression analysis, the read quantification data resulting from `",


Is this [^5] the citation number?

If so, it would be missing for the other citations like @love2014...

yep, it adds a superscript number and also links it to the citation

ah sorry, no, some citations are just added without the reference thing. The reference is only done for links, i.e. [^5] moves the report down to this link: https://github.com/qbic-pipelines/rnadeseq. The other citations like love2014 are just listed without being clickable

Is there a reason why this is not consistent with the clickable references?

assets/RNAseq_report.Rmd

SusiJo · 2023-06-20T07:25:07Z

assets/RNAseq_report.Rmd

+" `, `AnnotationDbi v", version_annotation,
+"` and `", name_species, " v", version_annotation,
+"` were employed. ", database_string, ".\n",
+"Pathways were classified as enriched for those genes with padj <= ", pval_text, "."


Suggested change

"Pathways were classified as enriched for those genes with padj <= ", pval_text, "."

"Pathways were classified as enriched for genes with an adjusted p-value below ", pval_text, "."

counter proposal: an adjusted p-value <= pval_test? I prefer having <= (or alternatively "less than or equal to") in the text so people don't wonder, even if that is an edge case

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

docs/usage.md

modules/local/report.nf

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

SusiJo · 2023-06-20T08:11:25Z

If you are changing the parameter name of pval_threshold please also update the docs.

WackerO · 2023-06-20T11:24:51Z

If you are changing the parameter name of pval_threshold please also update the docs.

Param is now called adj_pval_threshold and this is reflected in usage.md, I assume that is what you meant? I'll ask @qbicStefanC about the renaming and if necessary do a follow-up PR to rename it back to pval_threshold

bin/Execute_report.R

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

SusiJo

LGTM! 🚀

Added logic for transformation method

7788bfa

Oskar Wacker and others added 5 commits June 14, 2023 15:01

updated changelog, changed text in report, updated docs

2cbb218

Added separate sizefactor computation for tximport, changed < and > t…

d3ac03d

…o <= and => for logFC and pval

fixed some incorrect < and layout issues

aaa7d0f

minor change to string

7625737

added trycatch around heatmaps

6e217c8

SusiJo reviewed Jun 20, 2023

View reviewed changes

assets/RNAseq_report.Rmd Outdated Show resolved Hide resolved

SusiJo reviewed Jun 20, 2023

View reviewed changes

assets/RNAseq_report.Rmd Outdated Show resolved Hide resolved

SusiJo reviewed Jun 20, 2023

View reviewed changes

assets/RNAseq_report.Rmd Outdated Show resolved Hide resolved

SusiJo reviewed Jun 20, 2023

View reviewed changes

assets/RNAseq_report.Rmd Outdated Show resolved Hide resolved

SusiJo reviewed Jun 20, 2023

View reviewed changes

assets/RNAseq_report.Rmd Show resolved Hide resolved

removed pipe bind

df0fb8e

SusiJo reviewed Jun 20, 2023

View reviewed changes

assets/RNAseq_report.Rmd Outdated Show resolved Hide resolved

SusiJo reviewed Jun 20, 2023

View reviewed changes

assets/RNAseq_report.Rmd Outdated Show resolved Hide resolved

SusiJo reviewed Jun 20, 2023

View reviewed changes

WackerO and others added 2 commits June 20, 2023 09:30

Update assets/RNAseq_report.Rmd

3610ddf

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

Update assets/RNAseq_report.Rmd

3b1e4ca

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

SusiJo reviewed Jun 20, 2023

View reviewed changes

docs/usage.md Outdated Show resolved Hide resolved

SusiJo reviewed Jun 20, 2023

View reviewed changes

docs/usage.md Outdated Show resolved Hide resolved

SusiJo reviewed Jun 20, 2023

View reviewed changes

modules/local/report.nf Outdated Show resolved Hide resolved

WackerO and others added 3 commits June 20, 2023 10:05

Update assets/RNAseq_report.Rmd

ce4f349

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

Update docs/usage.md

c326b6b

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

Update docs/usage.md

e2abe68

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

WackerO added 2 commits June 20, 2023 10:32

changed some things for the review

4470d2a

Merge branch 'rlogic' of https://github.com/WackerO/rnadeseq into rlogic

ad3cc86

WackerO added 3 commits June 20, 2023 10:53

prettier

06382e0

renamed trafo to norm_method, updated test configs

9e0666a

updated docs

5fbb30f

SusiJo reviewed Jun 20, 2023

View reviewed changes

bin/Execute_report.R Outdated Show resolved Hide resolved

SusiJo reviewed Jun 20, 2023

View reviewed changes

bin/Execute_report.R Outdated Show resolved Hide resolved

WackerO and others added 2 commits June 20, 2023 13:36

Update bin/Execute_report.R

adef6aa

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

Update bin/Execute_report.R

646ffa4

Co-authored-by: SusiJo <43847534+SusiJo@users.noreply.github.com>

SusiJo approved these changes Jun 20, 2023

View reviewed changes

WackerO merged commit 96a8583 into qbic-pipelines:dev Jun 21, 2023

WackerO deleted the rlogic branch June 21, 2023 07:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added logic for transformation method #206

Added logic for transformation method #206

WackerO commented Jun 14, 2023 •

edited

Loading

github-actions bot commented Jun 14, 2023 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

SusiJo Jun 20, 2023

WackerO Jun 20, 2023 •

edited

Loading

SusiJo Jun 20, 2023

WackerO Jun 20, 2023

SusiJo Jun 20, 2023

SusiJo Jun 20, 2023

WackerO Jun 20, 2023

WackerO Jun 20, 2023

SusiJo Jun 20, 2023

SusiJo Jun 20, 2023 •

edited

Loading

WackerO Jun 20, 2023

SusiJo commented Jun 20, 2023

WackerO commented Jun 20, 2023

SusiJo left a comment

		@@ -2140,10 +2187,12 @@ as.character(params$revision),
		"` pipeline [^5], which was written using the nf-core template [@ewels2020nf]. For differential expression analysis, the read quantification data resulting from `",

	"Pathways were classified as enriched for those genes with padj <= ", pval_text, "."
	"Pathways were classified as enriched for genes with an adjusted p-value below ", pval_text, "."

Added logic for transformation method #206

Added logic for transformation method #206

Conversation

WackerO commented Jun 14, 2023 • edited Loading

PR checklist

github-actions bot commented Jun 14, 2023 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

Choose a reason for hiding this comment

WackerO Jun 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SusiJo Jun 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SusiJo commented Jun 20, 2023

WackerO commented Jun 20, 2023

SusiJo left a comment

Choose a reason for hiding this comment

WackerO commented Jun 14, 2023 •

edited

Loading

github-actions bot commented Jun 14, 2023 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️

WackerO Jun 20, 2023 •

edited

Loading

SusiJo Jun 20, 2023 •

edited

Loading