v1.5.0

Canary v1.5.0

A reliability and evidence-integrity release. v1.5 closes the seven findings of an independent audit and
hardens the separation between what this project claims and what it has actually measured. It adds no
new feature area.

What changed

Seven audited product defects are fixed with adversarial regressions:

• A rewritten existing check no longer inherits independent authority — provenance is decided by git
  (addedChecks vs new modifiedChecks), and both are worker-authored evidence.
• Token accounting refuses mixed ledgers — a cell counts only on a completed provider-native ledger, and
  a run counts only if all its cells share one instrument digest. (This is why the previous token
  headline was withdrawn — see below.)
• --from-saved is HISTORICAL, not a current PASS — exit 2, every row labelled, and it prints its own
  disproof; a failed live battery exits 4.
• One attempt per output directory, with refusal on existing evidence.
• LOCAL is stated as same-user / operator-selected, not as worker-independent proof; HARDENED remains
  the worker-independent path.
• The sealed step no longer loses python / java / git and blame the project — an
  operator-authorized toolchain directory is appended after the trusted dirs (never in front), and an
  unresolvable program is attributed with SEALED ENV CANNOT RESOLVE instead of PROJECT CHECK FAILURE.
• A legitimate sealed-script check outside a test path is no longer a false red — the discrimination
  surface is anchored to a sealed plan script's text instead of a path heuristic. Less false red, no more
  false green.

Evidence

Gate	Result
CI 36168541720 (head d3f6a9c)	all six legs SUCCESS
Windows core leg	1,211 tests — 1,206 pass, 0 fail, 5 skipped, 0 cancelled, 0 todo, zero not ok; terminal, not timed out
HARDENED boundary, live	6/6 controls measured — every control re-derived from native return values and broker transcripts
Unit suite (e51b6fc)	1,284 pass, 0 fail, 4 skipped
Productization battery (e51b6fc)	104 PASS / 0 FAIL / 3 SKIP
Unit suite (released head, after the version bump)	1,284 pass, 0 fail, 4 skipped
Productization battery (released head)	106 PASS / 0 FAIL / 3 SKIP

HARDENED means one thing only: a boundary measurement in which every control was observed available. A
host that cannot confine is reported HOST UNSUPPORTED — never a pass.

Audit and trust improvements

• A claim/evidence matrix in which every public claim carries one of five statuses, its reproduction
  command, and its limitation — including the claims this release explicitly does not make.
• A public audit kit written for a reviewer who did not build Canary, listing the author's own weakest
  points.
• A tripwire (tooling/probes/v15-evidence-consistency.mjs) that fails when a withdrawn figure is
  presented as current, when a retired statement reappears, or when a required statement of the current
  truth disappears.
• The token headline is withdrawn with no replacement rather than quietly restated: the corrected data
  has two COMPLETE runs that disagree in sign (80.74 % / −19.26 % and 102.91 % / +2.91 %), and at n=1 per cell
  this benchmark cannot resolve an effect of that size in either direction.

Known limitation — an unresolved, unexplained containment-sweep failure

One hosted Windows run failed sweepDescendants finds and kills a live parent's child process
(sweep killed [7564] but not 2144, duration_ms 35980). Four consecutive runs since have been green, and
that green is not presented as a fix:

• no product change was made between the failing run and the green ones — the only delta is a
  diagnostic in a test failure message, which cannot change product behaviour;
• the improved diagnostic emitted no output at all in the green runs, because its failure path never
  fired;
• 14 local reproduction attempts (idle, 48-way CPU load, 2-core pinning) could not reach it.

A missed descendant is a containment concern. It is recorded as OPEN / UNEXPLAINED / NOT FIXED rather
than re-rolled until it looked green, and it is the first target for the next audit.

Operational note

The Windows core leg ran 224.1 min here against 170.9 min on the previous green (e51b6fc) — about
31 % slower, leaving roughly 76 min of headroom under the 300-min cap. Whether that is runner
variance, host load or something in these bytes is not established; it is not classified as a product
defect, and the timeout was not raised.

Docs

• docs/AUDIT-KIT-1.5.md — attack this release
• docs/CLAIM-EVIDENCE-MATRIX-1.5.md — every claim and its limits
• docs/POST-AUDIT-CLOSURE-1.5.md — the seven findings, fixes and regressions
• docs/HARDENED-1.5.md — what the boundary does and does not protect

v1.4.0

What's Changed

• CANARY v1.4.0 release candidate - remote CI confirmation by @criptofn in #7

New Contributors

• @criptofn made their first contribution in #7

Full Changelog: v1.3.0...v1.4.0

v1.3.0

Full Changelog: v1.2.0...v1.3.0

v1.2.0

Full Changelog: v1.1.0...v1.2.0

v1.1.0

Canary is an independent verification layer for coding agents.

The agent builds. Canary verifies. NO PROOF, NO DONE.

A coding agent saying "done" is not evidence. A green suite is useful evidence — but if the same checks were already green before the change, or a stated requirement is covered by no check at all, Canary reports NOT PROVEN instead of READY.

What's new in v1.1

• Language / project support — native discovery and toolchain handling for Node/JS/TS, Python, Rust, Go, plus a universal command-driven project contract for any other ecosystem. Capability is reported separately as UNIVERSAL / NATIVE / OBSERVED / STRONG, and an unknown runner stays INCONCLUSIVE_ONLY.
• Proof discrimination — the sealed plan is re-run against the sealed base commit. Green before and green after is NOT PROVEN, not READY. A required comparison that cannot be established is an objective UNPROVEN obligation naming the reason — never an accidental PASS.
• Requirement coverage — canary task --requirement "<text>" prints each requirement's digest; the operator's canary bind <script> --requirement "<text>" attaches it to a check the sealed plan really runs. An objective requirement with no bound proof stays NOT PROVEN.
• Candidate workflow — canary work / canary finish: an isolated candidate off a frozen base, committed work, verification from outside, and promotion of the exact verified committed bytes only.
• Subjective acceptance — technical proof and human judgment are separate. An acceptance is bound to the exact candidate tree and scope digest, and can never close an objective duty.
• Agent support — Claude Code is GATED (a completion hook can block it); Codex and any other command-line agent are ADVISORY (a marked, removable AGENTS.md block that cannot block anything, and says so); canary mcp exposes the same operations as a transport with no extra authority.
• Observation channels — node --test, pytest and unittest, each bound to an authority (a repo pin, the verifying runtime itself, or an operator-sealed identity), so printed output can never mint a strong verdict.
• Machine interface — global --json, canary result --json (free; it writes nothing), and compact failure payloads that keep evidence out of the model's context.

Two pre-release blockers, closed

• BLOCKER 1 — candidate verify/finish could reach promotion using weaker obligation logic than the discrimination gate. Candidate verification and promotion now consume measured discrimination against the candidate's frozen isolation base; a changed test filename is no longer proof by itself.
• BLOCKER 2 — a required baseline comparison that could not be established could lose its obligation and improve NOT PROVEN to READY. Such comparisons now remain objective UNPROVEN: missing modules, materialization failure, unavailable execution and thrown comparison errors can never improve a verdict.

Both are regression-tested (apps/cli/test/discrimination-completion.test.ts).

Supported ecosystems

Node / JS / TS · Python · Rust · Go (native discovery and toolchain handling) — plus any command-driven project through the universal contract: CI workflow commands, Makefile / Taskfile / justfile, CMake+CTest or Meson, a shipped gradlew / mvnw, pom.xml, build.gradle*, *.sln / *.csproj, Package.swift, build.zig, phpunit.xml, composer.json, Rakefile, mix.exs, shard.yml — or your own canary.project.json (argv arrays, never a shell line).

Verified numbers (this release)

• npm test: 1072 tests · 1068 pass · 0 fail · 4 skip
• npm run verify:productization: 72 PASS · 2 host-bound SKIP · 0 FAIL (exit 0)
• Mutation / adversarial: master-pass 13/13 caught, trust-boundary 14/14 killed, architecture 13/13 caught; architecture closure matrix 39/39; packed-artifact matrix green against the installed tarball bytes.

SKIPS ARE NOT PASSES. The two skips are host-bound real-PTY checks (pre10-acceptance, f3-acceptance-growth): every product assertion in them executed through the repository's in-process terminal driver — the exact gate the product checks — but the OS-level pty allocation itself is not reproducible on this host. This release therefore records 72/74 with 2 disclosed skips, not 74/74. No reliability percentage is derived from these counts.

Benchmarks, honestly

Consolidated 13-fixture corpus (one model, one host): plain 36/36 delivered correct · guarded 33/33 · 0 false dones and 0 false greens in both arms · guarded +40.2% tokens.

This corpus did NOT demonstrate a general correctness advantage for Canary. The plain arm already hit the ceiling at 36/36 on these small fixtures.

One measurable saving, with requirements declared AND bound: plain 125,369 tokens → guarded 100,812 (−19.6%), invisible 91,439 (−27.1%) — 5/5 delivered correct in every arm, zero false dones, zero false greens.

The safe claim, verbatim: in one bound-requirement benchmark, Canary reduced model-token use by 19.6% while maintaining 100% delivered correctness. That is not "Canary saves 19.6% of tokens", not a correctness claim, and not a universal result.

Declared but NOT bound: five requirements left unbound cost the guarded run 1.5–1.7M tokens over 41–45 turns, because the worker was left pursuing a duty only the operator could close. Bind before the handoff: operator declares requirements → canary bind → canary setup → worker handoff.

Historical, and not the headline: bench-r6 measured −58.5% tokens with correctness unchanged — a historical configuration result, not a universal saving. bench-r5's −56.4% is explicitly rejected as a default because it lost one delivered-correct result and produced a false green. Reliability outranks token savings.

Limitations

• LOCAL has documented same-UID / local-root-of-trust limitations. Canary does not claim to prevent a same-UID worker from replacing the local root of trust on this security level.
• HARDENED is NOT available in this release, and it is not "installation away": it is produced by exactly one thing — a boundary measurement in which every required OS-level control is observed present (canary provider status). The provider/broker architecture exists and refuses to serve without a proven separation; it is not marketed as established isolation.
• A bound check establishes declared proof coverage inside Canary's model — not semantic truth. Canary does not claim that binding a check proves the meaning of a requirement.
• Canary does not claim it "never fails open" universally, and does not claim that every promoted change is independently semantically proven.

Install (Node.js 22+)

npm install -g ./canary-rn-cli-1.1.0.tgz   # verify against the published .sha256
canary --version                            # canary 1.1.0
cd your-repo && canary setup && canary doctor

The package is ONE self-contained bundle with zero runtime dependencies.

Commit: c559e5521759f9ab151ab3d0b6a3b4be811b640d · Tree: 0bdc918414aed87cf76f3a5fcd0b763e63ca2713

Release overview · Benchmark method · Benchmark ledgers · Verification journal · Capability levels · Trust architecture

v1.0.0

Canary binds completion to the exact clean committed candidate and its declared task, requirements and sealed authority. Aesthetic acceptance remains separate from technical evidence; explicit numeric targets need matching sealed proof bindings.

• Linux: 667 unit passes, 0 failures, 1 platform skip.
• Windows: 666 unit passes, 0 failures, 4 POSIX-only skips.
• Architecture matrix: 39/39 on source and installed packages, Linux and Windows.
• New authorization mutations: 13/13 caught; master-pass: 13/13; legacy M7–M10: 60 caught, one platform-only skip.
• Productization: 37 passed, one explicitly host-bound external-hook step skipped; 0 failures.

Verdict: PASS WITH DOCUMENTED RESIDUALS. Same-UID forgery, PTY impersonation, test meaning and the other documented local trust limits remain. Skips are not passes. GitHub-hosted CI is separate from these completed local checks.

Install the attached package with Node.js 22+: npm install -g ./canary-rn-cli-1.0.0.tgz, then run canary setup in your Node.js Git project. The package has no runtime npm dependencies.

Authorization contract · Full closure evidence

Commit: 94dc8eef1921d1bc977ab63e113768cf351d265c
Parent: ec73ab7f6060a0fefd53cc2634fccd3aba4bd6af
Tree: 1a7ad2d16070f681cebc650b3ab38cffbbf6ac12