Repository navigation
Releases: QuietFlare/clew
Release list
0.7.0
Added
- Two ways to try Clew with nothing installed. A Codespace opens with
Nextflow, Docker and two recorded nf-core/demo runs already in the
image, anddocker run -p 8770:8770 ghcr.io/quietflare/clewserves the
page with the same runs. docs/try.md walks through both.
In a Codespace, triage and the agent go through a proxy QuietFlare runs
unless you set your own keys. - Drift and Reclaim tabs on the page. Drift compares two runs of the
picked record, roots with their causes and the rest by cause. Reclaim
weighs the run's work directories from the record and the disk, with
the proof and the causes that withheld the rest, and deletes nothing.
"Explain in words" has a model read the plan on the page and say what
it reports, never whether it matters. - Browse on the page uploads a run from your own machine: a launch
folder, a.lineagefolder, or a zip or tgz of either, in every
browser. - The Nextflow lineage extractor reads the copies
publishDirmade, with
their checksums, into the graph's published files. On a store recorded
withNXF_CACHE_MODE=DEEP,clew reclaimproves a work directory
redundant from the record alone, with noclew digestpass. - Each command ends with the next one:
--list-runswith extract,
extract with impact, impact with seal, seal with verify. - The agreed design lives in
spec/: a module diagram, the rules and a
module map, reviewed on every pull request. examples/weblog_receiver.pycollects the events thenf-weblog
plugin posts into the JSONL fileclew extract nextflow-workreads, so
a run made without the lineage store still gets a graph from its work
tree. docs/try.md walks through it: receiver up, then
the run, then the extractor.- The plan table on the page shows each task's hash beside its process
and tag, so three rows of one process read apart, as they do in the
terminal.
Changed
clew demois gone. The recorded runs anddocs/try.mdreplace it.
Fixed
-
from clew.provider.nextflow import NextflowAdapter, SheetKind, the
import the providers guide shows, now works; the package exported only
NextflowAdapter. -
A
.historyentry Nextflow did not write, such as a macOS._*file
left by a tarball, is skipped with a warning instead of crashing the
extractor. -
A content-mode checksum label in a Nextflow lineage store is believed
only when the run record names Nextflow 26.09.0-edge or later. Earlier
versions wrote a standard-mode value under whatever label
NXF_CACHE_MODEgave (nextflow-io/nextflow#7579), and the extractor
took the label at its word. The coverage note names the version that
wrote the store and what to do. -
The store's
sha256mode is Nextflow's own 128-bit hash over the
content, not a SHA-256 of the file. Its digests are now labelled
nextflow-sha256:besidenextflow-deep:, so they never read as
comparable with aclew digestvalue. -
The sources doc said
process.cache = 'deep'makes the store record
content checksums. It does not; the environment variable does, on the
fixed versions.
0.6.1
0.4.0
Added
clew drift: where two runs of the same workflow part ways, and why.
Tasks paired by name, outputs compared by content digest, the cause of
each root read from the record.--runson digest, reclaim and drift: read a run straight from a
.lineagestore, a horus-lineage root, or a directory of graphs. The
digestsclew digestcomputes go to a sidecar under<runs>/.clew/.
0.3.0
Changed
- Modules are grouped by layer:
clew.graph,clew.ledger,
clew.extract,clew.questions,clew.views.clew.coreis gone. The public API is
clew.load_graph,blast_radius,classify,parse_triggerand
resolve_trigger. Commands are unchanged. clew stitchjoins runs by content digest.--resultsis gone.
Added
clew reclaim: which work directories are safe to delete, with the
proof for each. Nothing is removed without--applyand a receipt.- Content digests in the graph,
digestfields of the form
<algorithm>:<value>, recorded by the Horus, Snakemake and Nextflow
store extractors where the engine has them. clew digest: hash a run's files once, for runs the engine recorded
without content digests.- DNAnexus support:
clew extract-dnanexusbuilds a graph from an
analysis, over the API or from saved describe output. Edges join on
file ID. Optionalpriceandduration_sper task. - Latch support:
clew extract-latchbuilds a graph from an execution,
over the API or from saved records. Edges join onlatch://path. - Cromwell support:
clew extract-cromwellbuilds a graph from workflow
metadata, from a file or a server. Scatters become one node per shard,
subworkflows are flattened, edges join on path. Verified on Cromwell 92. - Snakemake support:
clew extract-snakemakereads the.snakemake
metadata store, file or SQLite backend, and carries Snakemake's own
input checksums onto the edges. Verified on Snakemake 9.26. - A graph contract in core,
contract_violations, run by the tests over
every shipped and fixture graph.
Fixed
- The runtime version now matches the package version. 0.2.0 reported
itself as 0.1.1.
v0.2.0
Added
- Horus support:
clew extract-horusreads a horus-lineage run directory
and joins edges by content digest, so a graph closes across machines. - A trigger registry and the generic
--trigger kind:valueform. An
unknown kind is read as a label key, so engines that record labels answer
label queries with no adapter. --htmlonclew impactwrites a self-contained impact report.- The lineage store extractor follows resumed runs: tasks cached from an
earlier session in the same chain stay in the graph, replaced tasks are
marked rather than dropped, and an unknown store version is refused. - A comparison of what each lineage source can prove, in the docs.
- Output sizes in the work-directory extractor.
Changed
--donoris now--subject. The old flag still works.- Generic graph queries moved from the domain adapters into core.
- Shipped demo graphs are anonymised.
- The README is a front page. The design documentation moved to
docs/.
Fixed
- External inputs that nf-prov crates record as URLs are kept rather than
dropped. - The work-directory extractor documents that forwarded files lose their
producer, and the lineage store is preferred for that reason.
v0.1.1
v0.1.0
First release on PyPI as clew-lineage.
- Lineage extraction from Nextflow's native lineage store, nf-prov RO-Crates
and work-directory symlinks, all emitting one graph format. - Blast radius for subject, container and external-input triggers, with
remove and distrust modes. - Contribution classes with fail-closed defaults and a versioned,
content-hashed remediation policy. - Storage checked against a work root, never assumed.
- Cross-run stitching at published files.
- An append-only, hash-chained event log on Postgres with two clocks.
- Sealed evidence bundles that replay offline, witnessed back into the log
and countersigned with OpenSSH keys. - A CI gate that fails closed on unknown subjects.
- A self-contained dashboard and a read-only MCP server for auditors.