Update to include tmpdir #200

DLBPointon · 2025-09-19T12:54:51Z

Addition of /tmpdir to Diamond blast modules.

sanger-tolsoft · 2025-09-19T12:55:27Z

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.2.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

github-actions · 2025-09-19T12:56:26Z

`nf-core pipelines lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit a468982

+| ✅ 187 tests passed       |+
#| ❔  28 tests were ignored |#
!| ❗  19 tests had warnings |!

❗ Test warnings:

nextflow_config - Config manifest.version should end in dev: 0.9.0
local_component_structure - windowstats_input.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - nohit_list.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - create_bed.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - compressblobdir.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - restructurebuscodir.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - jsonify_taxdump.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - generate_config.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
local_component_structure - coverage_stats.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - minimap_alignment.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - busco_diamond_blastp.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - run_blastn.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - run_blastx.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - collate_stats.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - prepare_genome.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - blobtools.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - input_check.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - view.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
local_component_structure - finalise_blobdir.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

files_exist - File is ignored: CODE_OF_CONDUCT.md
files_exist - File is ignored: assets/nf-core-blobtoolkit_logo_light.png
files_exist - File is ignored: docs/images/nf-core-blobtoolkit_logo_light.png
files_exist - File is ignored: docs/images/nf-core-blobtoolkit_logo_dark.png
files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
files_exist - File is ignored: .github/workflows/awstest.yml
files_exist - File is ignored: .github/workflows/awsfulltest.yml
files_exist - File is ignored: conf/igenomes.config
nextflow_config - Config variable ignored: manifest.name
nextflow_config - Config variable ignored: manifest.homePage
files_unchanged - File ignored due to lint config: CODE_OF_CONDUCT.md
files_unchanged - File ignored due to lint config: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_unchanged - File ignored due to lint config: .github/CONTRIBUTING.md
files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
files_unchanged - File does not exist: .github/ISSUE_TEMPLATE/config.yml
files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting_comment.yml
files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
files_unchanged - File ignored due to lint config: assets/email_template.html
files_unchanged - File ignored due to lint config: assets/email_template.txt
files_unchanged - File ignored due to lint config: assets/sendmail_template.txt
files_unchanged - File ignored due to lint config: assets/nf-core-blobtoolkit_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-blobtoolkit_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-blobtoolkit_logo_dark.png
actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/blobtoolkit/blobtoolkit/.github/workflows/awstest.yml
template_strings - template_strings
merge_markers - merge_markers

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes_ignored.config
files_exist - File found: modules.json
files_exist - File found: ro-crate-metadata.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-blobtoolkit_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowBlobtoolkit.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: lib/nfcore_external_java_deps.jar
files_exist - File not found check: .travis.yml
nextflow_config - Found nf-schema plugin
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: validation.help.enabled
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable found: validation.help.beforeText
nextflow_config - Config variable found: validation.help.afterText
nextflow_config - Config variable found: validation.help.command
nextflow_config - Config variable found: validation.summary.beforeText
nextflow_config - Config variable found: validation.summary.afterText
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config variable (correctly) not found: params.max_cpus
nextflow_config - Config variable (correctly) not found: params.max_memory
nextflow_config - Config variable (correctly) not found: params.max_time
nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
nextflow_config - Config variable (correctly) not found: params.validationLenientMode
nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.image_format= png
nextflow_config - Config default value correct: params.outdir= results
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
readme - README Nextflow minimum version badge matched config. Badge: 24.04.2, Config: 24.04.2
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
plugin_includes - No wrong validation plugin imports have been found
pipeline_name_conventions - Name adheres to nf-core convention
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: sanger_test_full.yml
actions_schema_validation - Workflow validation passed: template_version_comment.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: sanger_test.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: ci.yml
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
base_config - WINDOWSTATS_INPUT found in conf/base.config and Nextflow scripts.
base_config - BLOBTOOLKIT_WINDOWSTATS found in conf/base.config and Nextflow scripts.
base_config - FASTAWINDOWS found in conf/base.config and Nextflow scripts.
base_config - BUSCO_BUSCO found in conf/base.config and Nextflow scripts.
base_config - BLAST_BLASTN found in conf/base.config and Nextflow scripts.
base_config - NOHIT_LIST found in conf/base.config and Nextflow scripts.
modules_config - conf/modules.config found and not ignored.
modules_config - WINDOWMASKER_MKCOUNTS found in conf/modules.config and Nextflow scripts.
modules_config - WINDOWMASKER_USTAT found in conf/modules.config and Nextflow scripts.
modules_config - MINIMAP2_HIC found in conf/modules.config and Nextflow scripts.
modules_config - MINIMAP2_ILMN found in conf/modules.config and Nextflow scripts.
modules_config - MINIMAP2_CCS found in conf/modules.config and Nextflow scripts.
modules_config - MINIMAP2_CLR found in conf/modules.config and Nextflow scripts.
modules_config - MINIMAP2_ONT found in conf/modules.config and Nextflow scripts.
modules_config - MINIMAP2_ found in conf/modules.config and Nextflow scripts.
modules_config - SAMTOOLS_VIEW found in conf/modules.config and Nextflow scripts.
modules_config - SAMTOOLS_INDEX found in conf/modules.config and Nextflow scripts.
modules_config - PIGZ_COMPRESS found in conf/modules.config and Nextflow scripts.
modules_config - BLOBTK_DEPTH found in conf/modules.config and Nextflow scripts.
modules_config - BUSCO_BUSCO found in conf/modules.config and Nextflow scripts.
modules_config - RESTRUCTUREBUSCODIR found in conf/modules.config and Nextflow scripts.
modules_config - DIAMOND_BLASTP found in conf/modules.config and Nextflow scripts.
modules_config - DIAMOND_BLASTX found in conf/modules.config and Nextflow scripts.
modules_config - BLOBTK_DEPTH found in conf/modules.config and Nextflow scripts.
modules_config - BLOBTOOLKIT_WINDOWSTATS found in conf/modules.config and Nextflow scripts.
modules_config - BLOBTOOLKIT_CREATEBLOBDIR found in conf/modules.config and Nextflow scripts.
modules_config - BLOBTOOLKIT_UPDATEBLOBDIR found in conf/modules.config and Nextflow scripts.
modules_config - BLOBTOOLKIT_CHUNK found in conf/modules.config and Nextflow scripts.
modules_config - BLOBTOOLKIT_UNCHUNK found in conf/modules.config and Nextflow scripts.
modules_config - NOHIT_LIST found in conf/modules.config and Nextflow scripts.
modules_config - BLAST_BLASTN found in conf/modules.config and Nextflow scripts.
modules_config - COMPRESSBLOBDIR found in conf/modules.config and Nextflow scripts.
modules_config - BLOBTK_IMAGES found in conf/modules.config and Nextflow scripts.
modules_config - JSONIFY_TAXDUMP found in conf/modules.config and Nextflow scripts.
modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.2.1

Run details

nf-core/tools version 3.2.1
Run at 2025-09-22 11:44:45

muffato

Our convention is to control this via a parameter named use_work_dir_as_temp. We have that in readmapping and genomenote.

DLBPointon · 2025-09-19T14:55:12Z

Ok, no problem. How's this instead?

muffato

I think you can simplify the two main.nf by always doing mkdir blastp_tmp. It's the only tmpdir value that the pipeline allows anyway.

And please register use_work_dir_as_temp as the other params: nextflow.config, nextflow_schema.json, CHANGELOG.md

DLBPointon · 2025-09-22T08:54:21Z

I think you can simplify the two main.nf by always doing mkdir blastp_tmp. It's the only tmpdir value that the pipeline allows anyway.

That's how I have it for production at the minute, but didn't think it was enough for this. mkdir is in there now. Does it want to just be hardcoded for --tmpdir in that case then or should I keep modules.config as is?

And please register use_work_dir_as_temp as the other params: nextflow.config, nextflow_schema.json, CHANGELOG.md

What do you mean by this bit? use_work_dir_as_temp as no longer a hidden arg?

muffato · 2025-09-22T09:20:28Z

I think you can simplify the two main.nf by always doing mkdir blastp_tmp. It's the only tmpdir value that the pipeline allows anyway.

That's how I have it for production at the minute, but didn't think it was enough for this. mkdir is in there now. Does it want to just be hardcoded for --tmpdir in that case then or should I keep modules.config as is?

I mean hardcode mkdir blast_tmp in the main.nf, but still control --tmpdir in a closure like you have.
In the worst case, it's an unneeded mkdir but that's fine

And please register use_work_dir_as_temp as the other params: nextflow.config, nextflow_schema.json, CHANGELOG.md

What do you mean by this bit? use_work_dir_as_temp as no longer a hidden arg?

Exactly. Make it an official pipeline arg

DLBPointon · 2025-09-22T10:14:58Z

Cool,
Removed hidden: true
Added note in CHANGELOG that it is no longer hidden
tempdir control is simplified.

muffato · 2025-09-22T11:11:06Z

Ah there's a typo ! blastx_temp in main.nf, blastx_tmp in conf/modules.config

…nto add_tmp

DLBPointon added 2 commits September 19, 2025 13:53

Update to include tmpdir

49fe8a1

Update changelog

29d3bc7

muffato reviewed Sep 19, 2025

View reviewed changes

DLBPointon added 2 commits September 19, 2025 15:51

Correction based on comments from @muffato

cba18cc

linter!

c2dd92e

muffato reviewed Sep 22, 2025

View reviewed changes

DLBPointon and others added 3 commits September 22, 2025 11:09

Updates based on comments

82b9890

Updates based on comments

f134a96

Merge branch 'dev' into add_tmp

d7d45bd

muffato approved these changes Sep 22, 2025

View reviewed changes

DLBPointon added 3 commits September 22, 2025 12:22

Darn spelling mistakes!

cc150bd

Merge branch 'add_tmp' of https://github.com/sanger-tol/blobtoolkit i…

a40c7fd

…nto add_tmp

Darn spelling mistakes!

a468982

DLBPointon merged commit 54a4b37 into dev Sep 22, 2025
5 checks passed

DLBPointon deleted the add_tmp branch September 22, 2025 12:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update to include tmpdir #200

Update to include tmpdir #200

Uh oh!

DLBPointon commented Sep 19, 2025

Uh oh!

sanger-tolsoft commented Sep 19, 2025

Uh oh!

github-actions bot commented Sep 19, 2025 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

Uh oh!

muffato left a comment

Uh oh!

DLBPointon commented Sep 19, 2025

Uh oh!

muffato left a comment

Uh oh!

DLBPointon commented Sep 22, 2025

Uh oh!

muffato commented Sep 22, 2025

Uh oh!

DLBPointon commented Sep 22, 2025

Uh oh!

muffato commented Sep 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Update to include tmpdir #200

Update to include tmpdir #200

Uh oh!

Conversation

DLBPointon commented Sep 19, 2025

Uh oh!

sanger-tolsoft commented Sep 19, 2025

Uh oh!

github-actions bot commented Sep 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

nf-core pipelines lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

Uh oh!

muffato left a comment

Choose a reason for hiding this comment

Uh oh!

DLBPointon commented Sep 19, 2025

Uh oh!

muffato left a comment

Choose a reason for hiding this comment

Uh oh!

DLBPointon commented Sep 22, 2025

Uh oh!

muffato commented Sep 22, 2025

Uh oh!

DLBPointon commented Sep 22, 2025

Uh oh!

muffato commented Sep 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

github-actions bot commented Sep 19, 2025 •

edited

Loading

`nf-core pipelines lint` overall result: Passed ✅ ⚠️