Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add nf-test review guidelines #2414

Closed
wants to merge 11 commits into from
31 changes: 26 additions & 5 deletions src/content/docs/contributing/modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -1082,12 +1082,33 @@ A PR review is the process of examining a new modules' submission or the changes
- Ensure that temporary unzipped files are removed to avoid mitigating benefits and worsening problems.
- Ensure that large outputs are compressed with the correct tool (follow guidelines for gzip vs bzip2 vs other options).

#### In `../tests/modules/nf-core/modulename/main.nf` and `../tests/modules/nf-core/modulename/meta.yml`:
#### In `modules/nf-core/modulename/meta.yml`:

- Check that there are tests for all outputs, including optional ones.
- Check that the `meta.yml` file has correct documentation links and patterns of files.
- Run the tool help and check that important input (usually optional) has not been missed.
- Check that all outputs are captured by running pytest (e.g. on Gitpod).
- Check that the file has correct documentation links and patterns of files.
- Check that all inputs and outputs are documented in this file.

#### In `modules/nf-core/modulename/tests/main.nf.test`:

- Check that there are tests for all inputs and outputs, including optional ones. Also check for all possible parameter variations that could lead to different outputs.
- In general, the complete `process.out` should be snapshotted, i.e. `{ assert process.out.match() }{:groovy}`,
- [Different assertion types](https://nf-co.re/docs/contributing/tutorials/nf-test_assertions) should be used when a straightforward `process.out` snapshot is not feasible.
- Check that a stub test exists for the module, and that it's test names ends in `- stub`.
- Check that the `assertAll()` function is used and that there is at minimum a success assertion and versions in the snapshot.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should lint for this. could somebody open an issue please?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done #2415

- Check that the test names describe the test dataset and configuration used, some examples below:
- `test("homo_sapiens - [fastq1, fastq2] - bam")`
- `test("sarscov2 - [ cram, crai ] - fasta - fai")`
- `test("Should search for zipped protein hits against a DIAMOND db and return a tab separated output file of hits")`
- Check that all input data is being referenced using the `modules_testdata_base_path` parameter:
- e.g. `file(params.modules_testdata_base_path + 'genomics/sarscov2/illumina/bam/test.paired_end.sorted.bam', checkIfExists: true){:groovy}`

#### In `modules/nf-core/modulename/tests/main.nf.test.snap`:

- Check that all output files are present in the nf-test snapshot file, or at a minimum, check that the files exist.

#### Migrating pytest to nf-test:

- Ensure all previous pytest assertions are converted to nf-test.
- Confirm that all pytest files are deleted, including the entry in `tests/tags.yaml`.

## What is the `meta` map?

Expand Down
10 changes: 5 additions & 5 deletions src/content/tools/docs/dev/pipeline_lint_tests/nfcore_yml.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# nfcore\_yml
# nfcore_yml

#### `PipelineLint.nfcore_yml(){:python}`

Repository `.nf-core.yml` tests

The `.nf-core.yml` contains metadata for nf-core tools to correctly apply its features.

* repository type:
> * Check that the repository type is set.
* nf core version:
> * Check if the nf-core version is set to the latest version.
- repository type:
> - Check that the repository type is set.
- nf core version:
> - Check if the nf-core version is set to the latest version.
Loading