Adding md5-sums to the test-yml-files #696

asp8200 · 2022-07-28T14:27:33Z

Adding checks for md5-sums in CI-tests - where possible. For text-based output-files with changing md5-sums, I've tried adding some tests of certain strings in the output-files.

The binary-output-files with changing md5-sums could perhaps be tested with some custom tests (https://pytest-workflow.readthedocs.io/en/stable/#writing-custom-tests). I haven't tried doing that here. It would be another tedious and lengthy task to do custom tests for all the binary-output-files with changing md5-sums.

If this looks okay to you guys, then I'll update the changelog accordingly.

PR checklist

github-actions · 2022-07-28T14:38:07Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 7749eb7

+| ✅ 145 tests passed       |+
#| ❔   4 tests were ignored |#
!| ❗   1 tests had warnings |!

❗ Test warnings:

readme - README did not have a Nextflow minimum version badge.

❔ Tests ignored:

files_unchanged - File ignored due to lint config: assets/nf-core-sarek_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo_dark.png
files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-sarek_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-sarek_logo_light.png
files_exist - File found: docs/images/nf-core-sarek_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: lib/WorkflowSarek.groovy
files_exist - File found: modules.json
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-sarek_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.version
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: '3.1dev'
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - .gitignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version in Quick Start section matched config. README: 21.10.3, Config: 21.10.3
pipeline_todos - No TODO strings found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (375 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_params - Schema matched params returned from nextflow config
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: awsfulltest_somatic.yml
actions_schema_validation - Workflow validation passed: ci.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains a matching 'report_comment'.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.

Run details

nf-core/tools version 2.4.1
Run at 2022-08-23 07:32:02

FriederikeHanssen

This is fantastic ❤️ Thank you so much for going through everything and adding the md5sums.

There are some file for which you are testing for content when the md45sums change and for some you aren't. Is there a reason for it?

tests/test_aligner.yml

FriederikeHanssen · 2022-08-16T06:56:20Z

tests/test_gatk_spark.yml

    - path: results/reports/fastqc/test-test_L1
    - path: results/reports/markduplicates/test/test.md.metrics
+    # The text-based output-file test.md.metrics only contains comments and no metrics, which is a bit odd.


I forgot about this, we should check it out, to make sure it works fine

tests/test_umi.yml

asp8200 · 2022-08-16T09:08:42Z

There are some file for which you are testing for content when the md45sums change and for some you aren't. Is there a reason for it?

If the md5sum is changing, then I have (or should have) tested the file using contains if
1 the file is text-based (as we can't test binary files with contains) and
2 the file contains some string suitable for testing.

There are some text-based files which only contain only comments (didn't seem relevant for testing) and some html-files with a structure that didn't lend itself very well to testing with contains (the problem was including tabs and newlines in the tested strings. I didn't try handling that with contains_regex. I just got fed up with running the tests.)

Is there any particular file or files that you feel should be tested but isn't?

Dev

tests/test_aligner.yml

tests/test_annotation.yml

tests/test_only_paired_VC.yml

maxulysse · 2022-08-22T09:02:54Z

That is some tremendous work.
You went way beyond what I had in mind.
That's wonderful.
I love this so much.
Small comments about code cohesion when having contains.
I think the VEP checks can be improved, but apart from that, it's close to perfection

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

maxulysse · 2022-08-22T09:37:21Z

Could you update CHANGELOG as well?

tests/test_annotation.yml

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

maxulysse

<3

Starting to add md5-sums to test-yml-files

a7b4ebe

asp8200 added 28 commits July 29, 2022 08:01

Adding some more md5-sums for the CI-tests

96afab5

Adding more md5-sums for CI-tests

bff777d

Adding and correcting more md5-sums for CI-tests

0aebd4c

Adding and correcting more md5-sums for CI-tests

7ced3a1

Adjusting some test-yml-files. Using contains instead of md5sum.

7a3b100

Fixing some CI-tests

2900361

prettier

0171256

Adding more md5-sums for CI-tests

5f81ce3

Adding and correcting more md5-sums for CI-tests

b9aa526

Removing md5-sums for some CI-tests.

3920119

Removing md5-sums for some CI-tests

407ba73

Removing an md5-sums from a CI-test

6a92373

Fixing some CI-tests

e67d211

Adding more md5-sums for CI-tests

d7e4a88

Adding more md5-sums for CI-tests

9de0968

Adding more md5-sums for CI-tests

39b7e4e

Updating some md5-sums for CI-tests

caddabc

Removing some md5sums from some CI-tests

e2a6725

Adding more md5-sums for CI-tests

af5776a

Adding check of strings in test.md.metrics

5007222

Adding some more md5sums for CI-tests

d11b8d5

Adding and correcting some md5-checks and string-checks in CI-tests

be6726e

Testing test.strelka.variants.TsTv.qual for strings

91e377a

Adding and correcting some md5-checks and string-checks in CI-tests

7c99029

Adding and correcting some md5-checks and string-checks in CI-tests

c7ea3e6

Adding some more md5sums for CI-tests

7123236

Adding and correcting some md5-checks and string-checks in CI-tests

a717cbf

Adding and correcting some md5-checks and string-checks in CI-tests

07230ab

FriederikeHanssen reviewed Aug 16, 2022

View reviewed changes

asp8200 and others added 3 commits August 21, 2022 22:26

Merge pull request #2 from asp8200/dev

8228710

Dev

Updating some md5sums since mpileup-issue was fixed for Sarek v3.0.1

67d7488

Updating md5sum for mpileup-file

24f1ca7

maxulysse reviewed Aug 22, 2022

View reviewed changes

tests/test_aligner.yml Show resolved Hide resolved

maxulysse reviewed Aug 22, 2022

View reviewed changes

tests/test_aligner.yml Show resolved Hide resolved

maxulysse reviewed Aug 22, 2022

View reviewed changes

tests/test_annotation.yml Show resolved Hide resolved

maxulysse reviewed Aug 22, 2022

View reviewed changes

tests/test_annotation.yml Outdated Show resolved Hide resolved

maxulysse reviewed Aug 22, 2022

View reviewed changes

tests/test_only_paired_VC.yml Show resolved Hide resolved

asp8200 and others added 5 commits August 22, 2022 11:23

Update tests/test_annotation.yml

357091d

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

Update tests/test_only_paired_VC.yml

ff582a0

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

Update tests/test_aligner.yml

d444337

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

Update tests/test_aligner.yml

e620035

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

Fixing md5-check

95c2070

asp8200 added 6 commits August 22, 2022 12:36

Fixing md5-check

9ce58a9

Updating changelog

728efa9

Fixing md5-check

c5adc58

prettier

e8582f0

Fixing md5-checks

933e3d5

Improving test of test_snpEff_VEP.ann.vcf

4f98833

maxulysse reviewed Aug 22, 2022

View reviewed changes

tests/test_annotation.yml Outdated Show resolved Hide resolved

asp8200 and others added 4 commits August 22, 2022 17:00

Update tests/test_annotation.yml

24b7789

Co-authored-by: Maxime U. Garcia <maxime.garcia@scilifelab.se>

Trying to fix the md5 checks

7a0da04

Removing check of untable md5sum

609bfc1

Removing md5 checks for unstable files

7749eb7

maxulysse approved these changes Aug 23, 2022

View reviewed changes

asp8200 merged commit f84cae4 into nf-core:dev Aug 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding md5-sums to the test-yml-files #696

Adding md5-sums to the test-yml-files #696

asp8200 commented Jul 28, 2022 •

edited

github-actions bot commented Jul 28, 2022 •

edited

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

FriederikeHanssen left a comment

FriederikeHanssen Aug 16, 2022

asp8200 commented Aug 16, 2022

maxulysse commented Aug 22, 2022

maxulysse commented Aug 22, 2022

maxulysse left a comment

Adding md5-sums to the test-yml-files #696

Adding md5-sums to the test-yml-files #696

Conversation

asp8200 commented Jul 28, 2022 • edited

PR checklist

github-actions bot commented Jul 28, 2022 • edited

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

FriederikeHanssen left a comment

Choose a reason for hiding this comment

FriederikeHanssen Aug 16, 2022

Choose a reason for hiding this comment

asp8200 commented Aug 16, 2022

maxulysse commented Aug 22, 2022

maxulysse commented Aug 22, 2022

maxulysse left a comment

Choose a reason for hiding this comment

asp8200 commented Jul 28, 2022 •

edited

github-actions bot commented Jul 28, 2022 •

edited

`nf-core lint` overall result: Passed ✅ ⚠️