Fix sbdi export #537

jtangrot · 2023-02-08T09:20:24Z

This PR updates output from ampliseq option sbdi-export to match changes in the SBDI portal, and addresses issue #436
Changes include:

Change lib/WorkflowAmpliseq.groovy to look for sbdi_compatible_databases, instead of incompatible, to reduce risk of forgetting this when adding a new, potentially incompatible, database to ampliseq
Make sure e.g. UNITE can be used with sbdi-export
Reorder fields in conf/ref_databases.config so that newest version is at the top of the list for all databases, for consistency
Add urls to databases in dbversion field in conf/ref_databases.config
Update contents and names of the files produced to match updated requirements from SBDI

PR checklist

This comment contains a description of changes (with reason).
If you've fixed a bug or added code that should be tested, add tests!
If you've added a new tool - have you followed the pipeline conventions in the contribution docs- [ ] If necessary, also make a PR on the nf-core/ampliseq branch on the nf-core/test-datasets repository.
Make sure your code lints (nf-core lint).
Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
Usage Documentation in docs/usage.md is updated.
Output Documentation in docs/output.md is updated.
CHANGELOG.md is updated.
README.md is updated (including new tool citations and authors/contributors).

github-actions · 2023-02-08T09:22:26Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 2fd3038

+| ✅ 155 tests passed       |+
#| ❔   1 tests were ignored |#
!| ❗   2 tests had warnings |!

❗ Test warnings:

readme - README did not have a Nextflow minimum version badge.
schema_lint - Parameter input is not defined in the correct subschema (input_output_options)

❔ Tests ignored:

files_exist - File is ignored: conf/igenomes.config

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-ampliseq_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-ampliseq_logo_light.png
files_exist - File found: docs/images/nf-core-ampliseq_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: lib/WorkflowAmpliseq.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-ampliseq_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: '2.5.0dev'
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-ampliseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-ampliseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-ampliseq_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - lib/NfcoreTemplate.groovy matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version in Quick Start section matched config. README: 22.10.1, Config: 22.10.1
pipeline_todos - No TODO strings found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (170 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_params - Schema matched params returned from nextflow config
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains a matching 'report_comment'.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.7.2
Run at 2023-02-09 15:33:55

erikrikarddaniel

LGTM, except a comment regarding long dbversion strings that I'd like @d4straub 's input on.

conf/ref_databases.config

d4straub

As soon as all tests work I am fine with it. I approve now already.

CHANGELOG.md

workflows/ampliseq.nf

conf/ref_databases.config

Co-authored-by: Daniel Straub <42973691+d4straub@users.noreply.github.com>

jtangrot added 30 commits December 1, 2022 13:33

Add links to ENA samples/runs

e1d90ad

Update columns in annotation.tsv

c35f22d

Fix annotation_confidence, scientficName, taxonRank

faa1204

Fix annotation_confidence, scientficName, taxonRank

ed505c3

Change format of dbversio, add links

a7ac6ee

Remove sp from specificEpithet

693ccb5

Add ampliseq version etc to annotation algorithm

cfe73e8

Merge branch 'dev' of https://github.com/nf-core/ampliseq into fix_sbdi

0f31f61

Add domain to taxon_remarks

83900c1

Reorder database entries

1ecf8d5

Change sbdi_incompatible_databases to sbdi_compatible_databases

8eb0fe0

Remove sp and replace underscore with spaces

6b2f5d7

Add support for unite to sdbiexport

51ea6e0

Merge branch 'dev' of https://github.com/nf-core/ampliseq into fix_sbdi

7ee285b

Fix spaces

53eb969

Remove sampleSizeValue, fix specificEpithet and annotation_algorithm

d68b804

Add missing ranks to SBDI output

b622e47

Add database links

dc466ee

Summarize barrnap results and add to annotation file

1cc4beb

Updates done by nf-core lint

c073f20

nf-core lint --fix files_unchanged

4bf6ca6

Update format with prettier

a11a923

Add infraspecificEpithet

b429968

Fix/remove spp for species

ad6bc4c

Change speces to _

6d8791b

Update from dev

47bcb02

Merge branch 'dev' of https://github.com/nf-core/ampliseq into fix_sbdi

cbf5b39

fix spaces and other formatting

a7bf511

Incorporate changes from dev

6bc4fe9

Update CHANGELOG

d3da95e

Update CHANGELOG

b3cf2c0

jtangrot requested a review from erikrikarddaniel February 8, 2023 09:22

jtangrot added 2 commits February 8, 2023 10:35

Fix linting

8b3e89c

fix changes introduced in linting

9cd1530

erikrikarddaniel approved these changes Feb 8, 2023

View reviewed changes

conf/ref_databases.config Show resolved Hide resolved

d4straub approved these changes Feb 8, 2023

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

workflows/ampliseq.nf Outdated Show resolved Hide resolved

conf/ref_databases.config Show resolved Hide resolved

jtangrot and others added 2 commits February 9, 2023 16:31

Remove non-existing channel

1fbbf16

Co-authored-by: Daniel Straub <42973691+d4straub@users.noreply.github.com>

Update CHANGELOG.md

2fd3038

Co-authored-by: Daniel Straub <42973691+d4straub@users.noreply.github.com>

jtangrot merged commit 939f568 into nf-core:dev Feb 10, 2023

jtangrot deleted the fix_sbdi branch February 10, 2023 07:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix sbdi export #537

Fix sbdi export #537

jtangrot commented Feb 8, 2023

github-actions bot commented Feb 8, 2023 •

edited

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

erikrikarddaniel left a comment

d4straub left a comment

Fix sbdi export #537

Fix sbdi export #537

Conversation

jtangrot commented Feb 8, 2023

PR checklist

github-actions bot commented Feb 8, 2023 • edited

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

erikrikarddaniel left a comment

Choose a reason for hiding this comment

d4straub left a comment

Choose a reason for hiding this comment

github-actions bot commented Feb 8, 2023 •

edited

`nf-core lint` overall result: Passed ✅ ⚠️