Skip to content

v1.1.0

Choose a tag to compare

View all tags
@criptofn criptofn released this 13 Sep 15:43
· 178 commits to main since this release
v1.1.0
c559e55

Canary is an independent verification layer for coding agents.

The agent builds. Canary verifies. NO PROOF, NO DONE.

A coding agent saying "done" is not evidence. A green suite is useful evidence — but if the same checks were already green before the change, or a stated requirement is covered by no check at all, Canary reports NOT PROVEN instead of READY.

What's new in v1.1

  • Language / project support — native discovery and toolchain handling for Node/JS/TS, Python, Rust, Go, plus a universal command-driven project contract for any other ecosystem. Capability is reported separately as UNIVERSAL / NATIVE / OBSERVED / STRONG, and an unknown runner stays INCONCLUSIVE_ONLY.
  • Proof discrimination — the sealed plan is re-run against the sealed base commit. Green before and green after is NOT PROVEN, not READY. A required comparison that cannot be established is an objective UNPROVEN obligation naming the reason — never an accidental PASS.
  • Requirement coverage — canary task --requirement "<text>" prints each requirement's digest; the operator's canary bind <script> --requirement "<text>" attaches it to a check the sealed plan really runs. An objective requirement with no bound proof stays NOT PROVEN.
  • Candidate workflow — canary work / canary finish: an isolated candidate off a frozen base, committed work, verification from outside, and promotion of the exact verified committed bytes only.
  • Subjective acceptance — technical proof and human judgment are separate. An acceptance is bound to the exact candidate tree and scope digest, and can never close an objective duty.
  • Agent support — Claude Code is GATED (a completion hook can block it); Codex and any other command-line agent are ADVISORY (a marked, removable AGENTS.md block that cannot block anything, and says so); canary mcp exposes the same operations as a transport with no extra authority.
  • Observation channels — node --test, pytest and unittest, each bound to an authority (a repo pin, the verifying runtime itself, or an operator-sealed identity), so printed output can never mint a strong verdict.
  • Machine interface — global --json, canary result --json (free; it writes nothing), and compact failure payloads that keep evidence out of the model's context.

Two pre-release blockers, closed

  • BLOCKER 1 — candidate verify/finish could reach promotion using weaker obligation logic than the discrimination gate. Candidate verification and promotion now consume measured discrimination against the candidate's frozen isolation base; a changed test filename is no longer proof by itself.
  • BLOCKER 2 — a required baseline comparison that could not be established could lose its obligation and improve NOT PROVEN to READY. Such comparisons now remain objective UNPROVEN: missing modules, materialization failure, unavailable execution and thrown comparison errors can never improve a verdict.

Both are regression-tested (apps/cli/test/discrimination-completion.test.ts).

Supported ecosystems

Node / JS / TS · Python · Rust · Go (native discovery and toolchain handling) — plus any command-driven project through the universal contract: CI workflow commands, Makefile / Taskfile / justfile, CMake+CTest or Meson, a shipped gradlew / mvnw, pom.xml, build.gradle*, *.sln / *.csproj, Package.swift, build.zig, phpunit.xml, composer.json, Rakefile, mix.exs, shard.yml — or your own canary.project.json (argv arrays, never a shell line).

Verified numbers (this release)

  • npm test: 1072 tests · 1068 pass · 0 fail · 4 skip
  • npm run verify:productization: 72 PASS · 2 host-bound SKIP · 0 FAIL (exit 0)
  • Mutation / adversarial: master-pass 13/13 caught, trust-boundary 14/14 killed, architecture 13/13 caught; architecture closure matrix 39/39; packed-artifact matrix green against the installed tarball bytes.

SKIPS ARE NOT PASSES. The two skips are host-bound real-PTY checks (pre10-acceptance, f3-acceptance-growth): every product assertion in them executed through the repository's in-process terminal driver — the exact gate the product checks — but the OS-level pty allocation itself is not reproducible on this host. This release therefore records 72/74 with 2 disclosed skips, not 74/74. No reliability percentage is derived from these counts.

Benchmarks, honestly

Consolidated 13-fixture corpus (one model, one host): plain 36/36 delivered correct · guarded 33/33 · 0 false dones and 0 false greens in both arms · guarded +40.2% tokens.

This corpus did NOT demonstrate a general correctness advantage for Canary. The plain arm already hit the ceiling at 36/36 on these small fixtures.

One measurable saving, with requirements declared AND bound: plain 125,369 tokens → guarded 100,812 (−19.6%), invisible 91,439 (−27.1%) — 5/5 delivered correct in every arm, zero false dones, zero false greens.

The safe claim, verbatim: in one bound-requirement benchmark, Canary reduced model-token use by 19.6% while maintaining 100% delivered correctness. That is not "Canary saves 19.6% of tokens", not a correctness claim, and not a universal result.

Declared but NOT bound: five requirements left unbound cost the guarded run 1.5–1.7M tokens over 41–45 turns, because the worker was left pursuing a duty only the operator could close. Bind before the handoff: operator declares requirements → canary bind → canary setup → worker handoff.

Historical, and not the headline: bench-r6 measured −58.5% tokens with correctness unchanged — a historical configuration result, not a universal saving. bench-r5's −56.4% is explicitly rejected as a default because it lost one delivered-correct result and produced a false green. Reliability outranks token savings.

Limitations

  • LOCAL has documented same-UID / local-root-of-trust limitations. Canary does not claim to prevent a same-UID worker from replacing the local root of trust on this security level.
  • HARDENED is NOT available in this release, and it is not "installation away": it is produced by exactly one thing — a boundary measurement in which every required OS-level control is observed present (canary provider status). The provider/broker architecture exists and refuses to serve without a proven separation; it is not marketed as established isolation.
  • A bound check establishes declared proof coverage inside Canary's model — not semantic truth. Canary does not claim that binding a check proves the meaning of a requirement.
  • Canary does not claim it "never fails open" universally, and does not claim that every promoted change is independently semantically proven.

Install (Node.js 22+)

npm install -g ./canary-rn-cli-1.1.0.tgz   # verify against the published .sha256
canary --version                            # canary 1.1.0
cd your-repo && canary setup && canary doctor

The package is ONE self-contained bundle with zero runtime dependencies.

Commit: c559e5521759f9ab151ab3d0b6a3b4be811b640d · Tree: 0bdc918414aed87cf76f3a5fcd0b763e63ca2713

Release overview · Benchmark method · Benchmark ledgers · Verification journal · Capability levels · Trust architecture

Assets 4
Loading