nf-claw 1.0.0

nf-claw is a self-maintaining, token-minimal library of nf-core pipelines for AI agents and bioinformaticians. Each pipeline is kept unmodified as a git submodule pinned to its release, alongside a generated skill.md (inputs, parameters, run command) and a full parameter reference.md. One runner, nfclaw, launches any of them with preflight checks, recorded provenance, replay and chaining.

The pipelines are the work of the nf-core community: cite the pipeline you run, together with Nextflow, nf-core and nf-claw (CITATION.cff).

Getting 1.0.0

git clone --branch v1.0.0 --recurse-submodules https://github.com/bmds-tue/nf-claw.git
cd nf-claw && pip install -e .
nfclaw --version        # nfclaw 1.0.0 (commit 5d14ed4fb910)

The tag fixes the runner and all 95 pins. main moves on daily as the maintenance workflows bump pins; nfclaw --version and every provenance bundle record the commit that ran.

What's in 1.0.0

• 95 nf-core pipelines, each pinned to a release that passed the updater's gate or was validated by hand with recorded evidence. Find one with nfclaw list <keywords> (or grep catalog.json), then read pipelines/<name>/skill.md. The portal browses them.
• nfclaw run validates inputs and parameters against the pinned schemas before launching. --check prints the exact command without running it; --demo runs the release's bundled test; --nxf-ver pins the engine a release needs. Samplesheets can be generated from data files. Optional resource caps (--limit-*) and an opt-in --timeout are available; a stopped run removes the Docker/Podman task containers it started. nfclaw status reports how any run stands, including caveats the pipeline reported and gaps in what it recorded for replay, also as JSON (--json).
• Provenance and replay: every run writes a bundle (unless --no-provenance): the pipeline source, configs, parameters, SHA-256 of inputs and outputs, eligible container images pinned by digest, and the recorded execution environment. It does not embed input data, images or plugins, so those must stay available for a replay. provenance/commands.sh replays the run and checks that environment; nfclaw verify compares the replay's outputs with the original's (--require-environment for a stricter environment check), and nfclaw assess checks declared scientific outputs against an explicit contract. A run killed outright (SIGKILL, out of memory, a host restart) resumes with its finished tasks cached.
• Chaining: nfclaw chain run connects pipelines through 19 curated handoff rules (for example fetchngs → rnaseq, rnaseq → differentialabundance, demultiplex → sarek). It verifies every handoff by checksum, and nfclaw chain status reconstructs the lineage.
• MCP server: nfclaw mcp serves the library and runner to Claude Code, Codex, Antigravity, Hermes and other MCP clients. Long runs and chains are detached jobs that survive restarts; nfclaw jobs manages them from a shell.
• Self-maintenance: a daily job bumps pipelines to new releases and a weekly job adds new nf-core pipelines. Each update is published to main only after Nextflow acceptance, the full test suite, a schema drift gate and a real bundled analysis of every changed pipeline pass. CI covers Python 3.11–3.15 on Ubuntu and macOS, weekly Nextflow previews of every pipeline, native pipeline executions, CodeQL and the site build.

Validation

Validation establishes execution and reproducibility, not biological accuracy: validate a pipeline for your own data before relying on it. Full record, settings and limits: docs/validation.md and the audit records in docs/audit/.

Every pin on the final code. From 2026-11-07 to 2026-11-08, every pin except oncoanalyser ran its bundled test (nfclaw run <name> --demo) on a Linux host (de.NBI cloud, Docker), on the engine its skill.md names, at 8d8c8b8. Each run was replayed from its bundle (provenance/commands.sh) and compared with nfclaw verify. That tree is this release's runner apart from #259 (the wording of one error) and #260 (packaging and tests), neither of which changes what a run does; the other changes since are documentation and drugresponseeval's pin. 95 jobs, including marsseq's two:

• Runs: 93 of 95 succeeded. seqsubmit needs ENA Webin credentials; reportho failed on a reply from Ensembl that was not JSON, in its run and in its resume.
• Replays: 91 of 93 succeeded. sopa's was refused because its remote input changed (documented); funcscan's timed out downloading a database.
• Execution environment: identical in 87, different in 1 (sopa), unverified in 4 (mcmicro, circdna, raredisease, genomeqc), each with the gap recorded in its bundle.
• nfclaw verify: exit 0 for 82 replays and 1 for 9 (hicar, hlatyping, longraredisease, mag, nanoseq, nanostring, rnafusion, rnasplice, spatialaxe), each with a documented cause in docs/known-issues.md.
• 73 jobs were clean at the first attempt, 2 of them with caveats the pipeline reported (cutandrun; proteinfold, whose test runs stubs). A disk guard stopped 5, which were rerun alone. No job left a task container behind. tumourevo's and hgtseq's output contracts passed. marsseq ran without --velocity (references, then an analysis with an empty --star-index): replay identical, nfclaw verify 0.

Afterwards, on 8289263: the newest pin, drugresponseeval 1.4.0, ran, replayed with an identical environment and verified with exit 0. funcscan again verified with exit 0. reportho passed on --resume and replayed with an identical environment; nfclaw verify reported 25 extra files, one query's results that the original run had silently dropped (documented).

The release commit itself: the tagged head passed CI (ruff, the drift gate and the full suite on Ubuntu and macOS for Python 3.11–3.15, the Nextflow integration lane, the website and CodeQL). Before its last change, which only widened one test's wait, the full suite (4,198 tests) also passed on macOS (Python 3.13) and on the validation host (Linux, Python 3.12), and the website passed npm audit, the type check, the build and its 166 browser tests. On the validation host, at the same runner, the end-to-end battery ran:

• a demo with resource limits, its replay (environment identical) and nfclaw verify (exit 0), and refused limit values;
• a run without task records, reported as a provenance gap;
• stops as tasks start (no container left);
• an MCP job whose supervisor was killed, resumed with its finished tasks cached;
• MCP jobs, cancellation and refusals through the official SDK;
• a chain killed outright in stage 2, resumed with that stage's finished tasks cached, and one stopped there with SIGTERM, which left no container and resumed to success, both with their lineage verified by SHA-256;
• 115 odd argument and environment cases, all ending cleanly except one documented stall;
• nfclaw chain run --check on all 19 handoffs (15 pass on the default Nextflow 26.04.3; 4 are refused, naming their remedy);
• every pin's smoke check (95 of 95 pass on the engine their skill.md advises).

Earlier rounds: the pins of the time ran, replayed and verified on the same host in October (91 pins, 2026-10-14 to 16) and in the final pin sweep (90 pins, 2026-11-05/06); releases bumped or added in between were run and compared with their own nf-test snapshots. The audits from 2026-10-28 on re-ran the suite, the website checks, and real pipelines, chains, replays and MCP jobs.

Added on the release day, after publishing:

• oncoanalyser 3.0.0 ran its bundled test through the deployed nfclaw 1.0.0 (5d14ed4) on the validation host, on Nextflow 25.10.4 with the references upstream's CI caches (tests/cached_refs.config). It succeeded with no caveats in 40 minutes, with STAR_ALIGN peaking at 29.7 GB. Its replay had an identical environment, and nfclaw verify exited 0 (253 identical, 177 changed, 1 renamed, 0 missing, 0 extra). --require-environment refuses it only because its samplesheet is a remote validation input that can only be fetched again.
• marsseq's VELOCITY path ran, replayed and verified on the same host, on Nextflow 23.04.0 with --demo --velocity true:
  • on the pipeline's default, full STAR index (--build-references true --max-memory 48.GB --max-cpus 16), at 453d68e, whose runner is this release's: the reference build (9 tasks, 85 min; replay identical, nfclaw verify 0) and then the analysis (38 tasks, 13 min; replay identical, nfclaw verify 0, 112 identical, 85 changed, 0 missing, 0 extra);
  • and before that with a sparse STAR index (--genomeSAsparseD 3), at dbd598d: with executor.memory = 28.GB, the same outcome; without it, the build's replay was killed in a host-wide out-of-memory event when Nextflow ran STAR beside BOWTIE2_BUILD.

Not established: seqsubmit's analysis (it needs an ENA Webin account; a stub run checks the rest of its workflow); real-scale data, schedulers such as SLURM, and GPU analyses.

Requirements

Linux or macOS (on Windows, WSL 2), Python 3.11+, git, Nextflow (Java 17+), and Docker, Podman, Singularity or Apptainer. Install with pip install -e . from a clone (pip install -e ".[yaml]" for YAML parameter files, ".[scientific]" for nfclaw assess on BAM, VCF and H5AD). Use a path without spaces, and on macOS outside iCloud folders.

Known limitations

• seqsubmit's --demo needs ENA Webin credentials (set as Nextflow secrets) for its ENA query and submission; a stub run (--config with stubRun = true) checks the rest of its workflow without them.
• Memory and references: oncoanalyser's STAR_ALIGN needs about 30 GB of memory, and its references take 75 GB. marsseq's defaul...