We included a new linting test to ensure that nf-test snapshots contain the versions.yml file.
Components (modules and subworkflows) can now handle more possible git URLs (ssh:// and ftp://).
The nf-core download command has a new argument --tag, which allows adding additional tags to select particular revisions in the Seqera Platform interface. For example, --tag "3.10=validated"{:bash} would allow you to quickly select the validated version of the pipeline.

Pipeline template

We updated the GitHub Action which tests that the pipeline can be downloaded correctly (download_pipeline.yml)
We removed pyproject.toml from the template.
Pipeline-specific institutional configs support is now activated for all pipelines by default.
The .nf-core.yml file contains now the version of the pipeline template.

Changelog

Template

Remove fasta default from nextflow.config (#2828)
Update templates to use nf-core/setup-nextflow v2 (#2818)
Link to troubleshooting docs when pipeline fails (#2845)
Add fallback to download_pipeline.yml in case the pipeline does not support stub runs (#2846)
Set topic variable correctly in the mastodon announcement (#2848)
Add a cleanup action to download_pipeline.yml to fix failures caused by inadequate storage space on the runner (#2849)
Update python to 3.12 (#2805)
Remove pyproject.toml from template root
Shorten lines in pipeline template (#2908)
Add a new hidden --pipelines_testdata_base_path parameter to more easily switch locations of test data in test configs (#2931)[https://github.com//pull/2931]
Permanently activated pipeline-specific institutional configs support for all pipelines without need for manual intervention (#2936)
Template config: conda.channels, not channels (#2950)
Handles multiple DOIs + doi.org resolver from manifest.doi (#2946)
Update included components (#2949)
Update .editorconfig (#2953)

Linting

Only match assignments of params in main.nf and not references like params.aligner == <something> (#2833)
Include test for presence of versions in snapshot (#2888)
Components: set correct sha before running component lint tests (#2952)
Less strict logo comparison (#2956)
Handle request errors more gracefully for actions validation (#2959)

Download

Replace --tower with --platform. The former will remain for backwards compatability for now but will be removed in a future release. (#2853)
Better error message when GITHUB_TOKEN exists but is wrong/outdated
New --tag argument to add custom tags during a pipeline download (#2938)

Components

Handle more complete list of possible git URL forms (ssh:// and ftp:// prefixes specifically) (#2945)
Fix path in component update script (#2823)

General

Update CI to use nf-core/setup-nextflow v2 (#2819)
Changelog bot: handle also patch version before dev suffix (#2820)
Add force_pr flag to sync, to force a PR even though there are no changes committed (#2822)
Update prettier to 3.2.5 (#2830)
Update GitHub Actions (#2827), (#2902), (#2927), (#2939)
Switch to setup-nf-test (#2834)
Add tests for assignment and referencing of params in main.nf (#2841)
Optimize layers in dockerfile (#2842)
Update python:3.11-slim Docker digest to a2eb07f (#2847)
Strip out mention of "Nextflow Tower" and replace with "Seqera Platform" wherever possible
Fix issue with config resolution that was causing nested configs to behave unexpectedly (#2862)
Fix schema docs console output truncating (#2880)
Ensure path object converted to string before stripping quotes (#2878)
Fix incorrect assertions for called_with on mocks (#2891)
Make cli-provided module/subworkflow names case insensitive (#2869)
Get immediate parent path name for schema creation (#2886)
Remove old references to CUSTOMDUMPSOFTWAREVERSIONS and add linting checks (#2897)
Update pre-commit hook pre-commit/mirrors-mypy to v1.10.0 (#2933)
Update codecov/codecov-action digest to 5ecb98a (#2948)
Update gitpod/workspace-base Docker digest to 124f2b8 (#2943)
fix(collectfile): sort true for methods_description_mqc.yaml (#2947)
chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.4.3 (#2951)
Restructure CHANGELOG.md (#2954)

Assets 2

29 Feb 16:08

mashehu

2.13.1

9eac5e9

v2.13.1 - Tin Puppy Patc

v2.13.1 - Tin Puppy Patch - [2024-02-29]

Patch release for failing AWS actions.

Template

Remove obsolete editor settings in devcontainer.json and gitpod.yml (#2795)
Add nf-test test instructions to contributing and PR template (#2807)
Fix topic extraction step for hashtags in toots (#2810)
Update modules and subworkflows in the template (#2811)
Unpin setup-nextflow and action-tower-launch (#2806)

Download

Improved offline container image resolution by introducing symlinks, fixes issues #2751, #2644 and demultiplex#164: (#2768)

General

chore(deps): update codecov/codecov-action digest to 0cfda1d (#2794)
chore(deps): update gitpod/workspace-base docker digest to c15ee2f (#2799)

Assets 2

20 Feb 15:26

mirpedrol

2.13

b51c7d8

v2.13 - Tin Puppy

v2.13 - Tin Puppy - [2024-02-20]

Highlights 🌟

This new version of nf-core/tools includes a refactoring of the pipelines template.

The lib directory is removed: 👋 Groovy code
We now use nf-core subworkflows for pipeline initialisation (replacing the code from the lib directory)
The nf-validation plugin is now used to create an input channel from a sample sheet file.

Check the nf-core blog post for more information!

Template

Add empty line in README.md to fix badges. (#2729)
Replace automatic branch detection in nf-core download CI test with hardcoded dev and input. (#2727)
Add Github Action to automatically cleanup ubuntu-latest runners to fix runner running out of diskspace errors(#2755)
Fix GitHub Actions CI and Linting badges links (#2757)
Add hashtags to release announcement on mastodon (#2761)
update fastqc and multiqc in template (#2776)
template refactoring: remove the lib directory and use nf-core subworkflows (#2736)
use nf-validation to create an input channel from a sample sheet (#2736)

Linting

Make creat-lint-wf composable (#2733)
Add looser comparison when pipeline logos (#2744)
Handle multiple aliases in module imports correctly during linting (#2762)
Switch to markdown based API and error docs (#2758)

Modules

Handle dirty local module repos by force checkout of commits and branches if needed (#2734)
Patch: handle file not found when it is an added file to a module (#2771)
Handle symlinks when migrating pytest (#2770)
Add --profile parameter to nf-test command (#2767)

General

fix ignoring changes in partially templated files (e.g. .gitignore) (#2722)
update ruff to 0.2.0 and add it to pre-commit step (#2725)
Update codecov/codecov-action digest to e0b68c6 (#2728)
Update pre-commit hook astral-sh/ruff-pre-commit to v0.2.1 (#2730)
Update python:3.11-slim Docker digest to 2a746e2 (#2743)
Update actions/setup-python action to v5 (#2739)
Update gitpod/workspace-base Docker digest to 45e7617 (#2747)
chore(deps): pin jlumbroso/free-disk-space action to 54081f1 (#2756)
chore(deps): update actions/github-script action to v7 (#2766)
chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.2.2 (#2769)
Update gitpod/workspace-base Docker digest to 728e1fa (#2780)

Assets 2

01 Feb 10:48

mashehu

2.12.1

776089a

v2.12.1 - Aluminium Wolf - Patch

v2.12.1 - Aluminium Wolf - Patch - [2024-02-01]

Linting

Handle default values of type number from nextflow schema (#2703)
fix ignoring files_unchanged (#2707)

General

Update pre-commit hook astral-sh/ruff-pre-commit to v0.1.15 (#2705)
use types for default value comparison (#2712)
fix changelog titles (#2708)
Print relative path not absolute path in logo cmd log output (#2709)
Update codecov/codecov-action action to v4 (#2713)
Ignore nf-core-bot in renovate PRs (#2716)

Assets 2

29 Jan 14:42

mashehu

2.12

d1038fb

2.12 - Aluminium Wolf

v2.12 - Aluminium Wolf - [2024-01-29]

Template

Add a Github Action Workflow to the pipeline template that tests a successful download with 'nf-core download' (#2618)
Use pre-commit to lint files in GitHub CI (#2635)
Use pdiff also on gitpod for nf-test (#2640)
switch to new image syntax in readme (#2645)
Add conda channel order to nextflow.config (#2094)
Fix tyop in pipeline nextflow.config (#2664)
Remove nfcore_external_java_deps.jar from lib directory in pipeline template (#2675)
Add function to check -profile is well formatted (#2678)
Add new pipeline error message pointing to docs when 'requirement exceeds available memory' error message (#2680)
add 👀👍🏻🎉😕 reactions to fix-linting-bot action (#2692)

Linting

Fix linting of a pipeline with patched custom module (#2669)
linting a pipeline also lints the installed subworkflows (#2677)
environment.yml name must be lowercase (#2676)
allow ignoring specific files when template_strings (#2686)
lint nextflow.config default values match the ones specified in nextflow_schema.json (#2684)

Modules

Fix empty json output for nf-core list local (#2668)

General

Run CI-pytests for nf-core tools on self-hosted runners (#2550)
Add Ruff linter and formatter replacing Black, isort and pyupgrade (#2620)
Set pdiff as nf-test differ in Docker image for Gitpod (#2642)
Fix Renovate Dockerfile updating issues (#2648 and #2651)
Add new subcommand nf-core tui, which launches a TUI (terminal user interface) to intuitively explore the command line flags, built using Trogon (#2655)
Add new subcommand: nf-core logo-create to output an nf-core logo for a pipeline (instead of going through the website) (#2662)
Handle api redirects from the old site (#2672)
Remove redundanct v in pipeline version for emails (#2667)
add function to check -profile is well formatted (#2678)
Update pre-commit hook astral-sh/ruff-pre-commit to v0.1.14 (#2674)
Update pre-commit hook pre-commit/mirrors-mypy to v1.8.0 (#2630)
Update mshick/add-pr-comment action to v2 (#2632)
update python image version in docker file (#2636)
Update actions/cache action to v4 (#2666)
Update peter-evans/create-or-update-comment action to v4 (#2683) and (#2695)

Assets 2

20 Dec 15:50

mashehu

2.11.1

50d56f6

Magnesium Dragon Patch

v2.11.1 - Magnesium Dragon Patch - [2023-12-20]

Template

Rename release-announcments.yml to release-announcements.yml (#2610)
Fix nextflow.config docker.runOptions (#2607)

General

Only dump modules.json when it is modified (#2609)

Assets 2

19 Dec 18:29

mashehu

2.11

a12d77a

2.11

v2.11 - Magnesium Dragon + [2023-12-19]

Highlights:

nf-test is now used for testing modules and subworkflows. 🚀 This comes with updated templates and a changed commands (good-bye nf-core modules/subworkflows create-test-yml). For more information, you can refer to the revised documentation on modules and subworkflows .
- Every module and subworkflow now contains a tests/ directory, which includes a main.nf.test file. By running nf-core modules test or nf-core subworkflows test, you can perform the tests and generate the main.nf.test.snap snapshot file. As a general guideline, we recommend testing all outputs of modules/subworkflows. For complex cases, please refer to the nf-test assertion tutorial.
- To assist with the conversion of modules and subworkflows from pytest to nf-test, we've added the new --migrate-pytest option for nf-core modules test and nf-core subworkflows test, which does most of the work for you.
- The nf-core modules lint and nf-core subworkflows lint commands now also check the nf-test configuration and warn you if you are using pytest (it's really not that hard to switch 🙂 ).
We pinned the version of nf-validation, which should make offline usage of nf-core pipelines easier.
In nf-core download, we now prefix all absolute container URIs with docker://, which should make more pipelines work out of the box.
We started using type hinting in the nf-core/tools codebase. This will help us to find bugs earlier and make the code more readable.

The complete list of changes can be found below.

Template

Fix writing files to a remote outdir in the NfcoreTemplate helper functions (#2465)
Fancier syntax highlighting for example samplesheets in the usage.md template (#2503)
Use closure for multiqc ext.args (#2509)
Fix how the modules template references the conda environment file (#2540)
Unset env variable JAVA_TOOL_OPTIONS in gitpod (#2569)
Pin the version of nf-validation (#2579)
Disable process selector warnings by default (#2161)
Remove docker.userEmulation from nextflow.config in pipeline template (#2580)

Download

Add docker:// prefix for absolute container URIs as well (#2576).
Bugfix for AttributeError: ContainerError object has no attribute absoluteURI (#2543).

Linting

Fix incorrectly failing linting if 'modules' was not found in meta.yml (#2447)
Correctly pass subworkflow linting test if COMPONENT.out.versions is used in the script (#2448)
Add pyupgrade to pre-commit config and dev requirements as mentioned in #2200
Check for spaces in modules container URLs (#2452)
Correctly ignore timeline.enabled, report.enabled, trace.enabled, dag.enabled variables when linting a pipeline. (#2507)
Lint nf-test main.nf.test tags include all used components in chained tests (#2572)
Don't fail linting if md5sum for empty files are found in a stub test (#2571)
Check for existence of test profile (#2478)

Modules

Added stub test creation to create_test_yml (#2476)
Replace ModulePatch by ComponentPatch (#2482)
Fixed nf-core modules lint to work with new module structure for nf-test (#2494)
Add option --migrate-pytest to create a module with nf-test taking into account an existing module (#2549)
When installing modules and subworkflows, automatically create the ./modules directory if it doesn't exist (#2563)
When .nf-core.yml is not found create it in the current directory instead of the root filesystem (#2237)
Modules --migrate-pytest copies template scripts (#2568)

Subworkflows

Added stub test creation to create_test_yml (#2476)
Fixed nf-core subworkflows lint to work with new module structure for nf-test (#2494)
Add option --migrate-pytest to create a subworkflow with nf-test taking into account an existing subworkflow (#2549)

General

Update schema build functionality to automatically update defaults which have changed in the nextflow.config(#2479)
Change testing framework for modules and subworkflows from pytest to nf-test (#2490)
bump_version keeps now the indentation level of the updated version entries (#2514)
Add mypy to pre-commit config for the tools repo (#2545)
Use Path objects for ComponentCreate and update the structure of components templates (#2551).
GitPod base image: swap tool installation back to conda from mamba (#2566).
Sort the installed_by list in modules.json (#2570).
Unset env variable JAVA_TOOL_OPTIONS in gitpod (#2569)

New Contributors

@fa2k made their first contribution in #2472
@NovakApis made their first contribution in #2223
@rernst made their first contribution in #2507
@kedhammar made their first contribution in #2483

Contributors

fa2k, rernst, and 2 other contributors

Assets 2

25 Sep 14:44

mirpedrol

2.10

9ab896c

v2.10 - Nickel Ostrich

v2.10 - Nickel Ostrich + [2023-09-25]

This new release includes new commands, template updates, and several bug fixes. Thanks to all contributors for making this possible!

🌟 Highlights

We are now linting subworkflows nf-core subworkflows lint.
Use nf-core create-params-file params-file with documentation and default values for all parameters in a pipeline.
A new GitHub Action will automatically post new pipeline release messages to Mastodon, Bluesky and Twitter
Pipelines in sync with the new template, will dump all pipeline parameters to a file called params.json after every run

✖️ Deprecations

Deprecation of the —no-git argument for nf-core create command
Deprecation of the —template-yaml argument for the nf-core sync command - The same information will now be automatically stored in .nf-core.yml

Template

Fix links in multiqc_config.yml (#2372 and #2412)
Remove default false from nextflow_schema.json (#2376)
Add module MULTIQC to modules.config (#2377)
Add GitHub workflow for automated release announcements (#2382)
Update the Code of Conduct (#2381)
Save template information to .nf-core.yml and deprecate argument --template-yaml for nf-core sync (#2388 and #2389)
(#2397) Remove fixed Ubuntu test and added to standard testing matrix
(#2396) Reduce container finding error to warning since the registries are not consistent.
(#2415) Add autoMounts for apptainer.
Remove igenomes_base from the schema, so that nf-validation doesn't create a file path and throw errors offline for s3 objects.
Modified devcontainer permissions so that singularity can be run in Codespaces/VS Code devcontainers (Commit a103f44)
Update Gitpod profile resources to reflect base environment settings.
(#747) Add to the template the code to dump the selected pipeline parameters into a json file.

Download

Improved container image resolution and prioritization of http downloads over Docker URIs (#2364).
Registries provided with -l/--container-library will be ignored for modules with explicit container registry specifications (#2403).
Fix unintentional downloading of containers in test for the Tower download functionality. Bug reported by @adamrtalbot and @awgymer (#2434).

Linting

Add new command nf-core subworkflows lint (#2379)

Modules

Subworkflows

Fix bug: missing subworkflow name when using nf-core subworkflows create (#2435)

General

Initialise docker_image_name to fix UnboundLocalError error (#2374)
Fix prompt pipeline revision during launch (#2375)
Add a create-params-file command to create a YAML parameter file for a pipeline containing parameter documentation and defaults. (#2362)
Update the Code of Conduct (#2381)
Remove --no-git option from nf-core create (#2394)
Throw warning when custom workflow name contains special characters (#2401)
Bump version of nf-test snapshot files with nf-core bump-version (#2410)

Contributors

adamrtalbot and awgymer

Assets 2

30 Jun 16:10

mirpedrol

2.9

e5ce6ce

v2.9 - Chromium Falcon

v2.9 - Chromium Falcon + [2023-06-30]

A new big release with new features, template updates and bug fixes. Thanks to all contributors!

🌟 Highlights

A new nf-core downloadcommand compatible with Nextflow Tower.
Implementation of the plugin nf-validation for parameter validation in pipeline templates.
Nextflow version bump to 23.04.0.

Template

params.max_multiqc_email_size is no longer required (#2273)
Remove cleanup = true from test_full.config in pipeline template (#2279)
Fix usage docs for specifying params.yaml (#2279)
Added stub in modules template ([#2277])(#2277) [Contributed by @nvnieuwk]
Move registry definitions out of profile scope ([#2286])(#2286)
Remove aws_tower profile ([#2287])(#2287)
Fixed the Slack report to include the pipeline name (#2291)
Fix link in the MultiQC report to point to exact version of output docs (#2298)
Updates seqeralabs/action-tower-launch to v2.0.0 (#2301)
Remove schema validation from lib folder and use Nextflow nf-validation plugin instead (#1771)
Fix parsing of container directive when it is not typical nf-core format (#2306)
Add ability to specify custom registry for linting modules, defaults to quay.io (#2313)
Add singularity.registry = 'quay.io' in pipeline template (#2305)
Add apptainer.registry = 'quay.io' in pipeline template (#2352)
Bump minimum required NF version in pipeline template from 22.10.1 -> 23.04.0 (#2305)
Add ability to interpret docker.registry from nextflow.config file. If not found defaults to quay.io. (#2318)
Add functions to dynamically include pipeline tool citations in MultiQC methods description section for better reporting. (#2326)
Remove --tracedir parameter (#2290)
Incorrect config parameter warnings when customising pipeline template (#2333)
Use markdown syntax in the description for the meta map channels (#2358)

Download

Introduce a --tower flag for nf-core download to obtain pipelines in an offline format suited for seqeralabs® Nextflow Tower (#2247).
Refactored the CLI for --singularity-cache in nf-core download from a flag to an argument. The prior options were renamed to amend (container images are only saved in the $NXF_SINGULARITY_CACHEDIR) and copy (a copy of the image is saved with the download). remote was newly introduced and allows to provide a table of contents of a remote cache via an additional argument --singularity-cache-index (#2247).
Refactored the CLI parameters related to container images. Although downloading other images than those of the Singularity/Apptainer container system is not supported for the time being, a generic name for the parameters seemed preferable. So the new parameter --singularity-cache-index introduced in #2247 has been renamed to --container-cache-index prior to release (#2336).
To address issue #2311, a new parameter --container-library was created allowing to specify the container library (registry) from which container images in OCI format (Docker) should be pulled (#2336).
Container detection in configs was improved. This allows for DSL2-like container definitions inside the container parameter value provided to process scopes #2346.
Add apptainer to the list of false positve container strings (#2353).

Updated CLI parameters

Old parameter	New parameter
new parameter	`-d` / `--download-configuration`
new parameter	`-t` / `--tower`
`-c`/ `--container`	`-s` / `--container-system <VALUE>`
new parameter	`-l` / `--container-library <VALUE>`
`--singularity-cache`	`-u` / `--container-cache-utilisation <VALUE>`
new parameter	`-i` / `--container-cache-index <VALUE>`

In addition, -r / --revision has been changed to a parameter that can be provided multiple times so several revisions can be downloaded at once.

Linting

Warn if container access is denied (#2270)
Error if module container specification has quay.io as prefix when it shouldn't have ([#2278])(https://github.com/nf-core/tools/pull/2278/files)
Detect if container is 'simple name' and try to contact quay.io server by default (#2281)
Warn about null/None/empty default values in nextflow_schema.json (#3328)
Fix linting when creating a pipeline skipping some parts of the template and add CI test (#2330)

Modules

Don't update modules_json object if a module is not updated (#2323)

Subworkflows

General

GitPod base image: Always self-update to the latest version of Nextflow. Add pre-commit dependency.
GitPod configs: Update Nextflow as an init task, init pre-commit in pipeline config.
Refgenie: Create nxf_home/nf-core/refgenie_genomes.config path if it doesn't exist (#2312)
Add CI tests to test running a pipeline when it's created from a template skipping different areas

Contributors

nvnieuwk

Assets 2

Releases: nf-core/tools

v2.14.1 - Tantalum Toad - Patch

Template

General

v2.14.0 - Tantalum Toad

🌟 Highlights 🌟

nf-core/tools functionalities

Pipeline template

Changelog

Template

Linting

Download

Components

General

v2.13.1 - Tin Puppy Patc

v2.13.1 - Tin Puppy Patch - [2024-02-29]

Template

Download

General

v2.13 - Tin Puppy

v2.13 - Tin Puppy - [2024-02-20]

Highlights 🌟

Template

Linting

Modules

General

v2.12.1 - Aluminium Wolf - Patch

v2.12.1 - Aluminium Wolf - Patch - [2024-02-01]

Linting

General

2.12 - Aluminium Wolf

v2.12 - Aluminium Wolf - [2024-01-29]

Template

Linting

Modules

General

Magnesium Dragon Patch

v2.11.1 - Magnesium Dragon Patch - [2023-12-20]

Template

General

2.11

v2.11 - Magnesium Dragon + [2023-12-19]

Template

Download

Linting

Modules

Subworkflows

General

New Contributors

Contributors

v2.10 - Nickel Ostrich

v2.10 - Nickel Ostrich + [2023-09-25]

Template

Download

Linting

Modules

Subworkflows

General

Contributors

v2.9 - Chromium Falcon

v2.9 - Chromium Falcon + [2023-06-30]

Template

Download

Updated CLI parameters

Linting

Modules

Subworkflows

General

Contributors