Skip to content

Wave Loop 549 — CLARA coverage regenerated over 496 specs; 730 seals verify 0 #1954

Description

@gHashTag

Wave Loop 549 — CLARA coverage regenerated; 730 seals, 0 verify

Follow-up to #1951 (Variant A: "Decide CLARA — regenerate or narrow").

The question resolved itself

The old conformance/clara_spec_coverage.json was dated 2026-04-05, covered 36
specs against a corpus of 496, and recorded:

"demo_pipeline": { "command": "bash scripts/clara/demo.sh", "result": "20/20 passed, 0 failed, 0 skipped" }

scripts/clara/ does not exist anywhere in this repository. That was a passing test
result no one could reproduce. "Narrow the claim" was never really an option — the
evidence file had to be replaced.

Regenerated (L7-clean, no shell)

New t27c clara-coverage runs every phase as a real subprocess across all 496 specs and
writes schema-v2 with a reproduce field:

parse 496/496   gen_zig 496/496   gen_verilog 496/496   seal 0/496

The seal finding — 730 seals, 0 verify

seal 0/496 is a clean zero, so it was treated as a harness fault until disproved. It
is real:

  • .trinity/seals/ holds 730 files; not one verifies.
  • Seals were last written April 2026480 of them on 2026-04-14, the same day as
    fcf80027 "fix(l3-purity): replace all Unicode with ASCII in 160 .t27 files", which
    modified the specs being sealed. Nothing re-baselined since, across R12–R14 codegen fixes.
  • specs/numeric/gf16.t27 is git-clean and still fails on spec_hash — so this is
    not an artefact of the uncommitted worktree.

Why no gate caught it

Pre-commit Gate 2/4 tests [[ ! -f "$seal_file" ]]file existence. It never
verifies a hash. Presence is not integrity, and only presence was enforced.

There are also two seal-naming schemes: the gate derives basenamegf16.json,
while seal --verify reads a path-derived numeric_triformat-gf16.json. These coincide
only on a case-insensitive filesystem — so on Linux CI the gate would look for a file
that isn't there.

New: t27c seal-audit [--strict] reports the verify rate in one command.
Non-blocking by default.

Deliberately NOT done: the re-seal

seal --save across 496 specs would rewrite 730 provenance records and canonicalise
whatever the current codegen emits, with no independent oracle that it is correct. That
is a human decision, not a side effect of an audit. This is Variant A below.

Claims corrected rather than quietly patched

  • COMPETITORS.md claim 5 withdraws seal-based integrity.
  • README.md splits Seal presence (GREEN) from Seal integrity (RED), and states
    that ./scripts/tri test exits non-zero because of it.
  • CLARA_TRACEABILITY.md downgrades the pipeline row to partial; its reproduction
    block now shows the failing command instead of omitting it.

Suite: 1157 / 1157 passed, 0 failed.


Three cooperation variants for W550

  • Variant A (recommended) — Re-baseline the seals, with a diff review. Run
    seal --save across all 496 specs, but land it as its own commit with the
    gen_hash_* deltas summarised, so the re-baseline is reviewable rather than a
    1,460-hash blob. Then make Gate 2/4 verify instead of merely exist, and fix the
    basename vs path-derived naming split before it bites on Linux CI. This closes the
    only RED row in the README.

  • Variant B — Make the gates check what they name. Three waves have each found one
    gate checking a cheap proxy: scripts/tri (broken, so bypassed), validate-conformance
    (58 false positives), Gate 2/4 (presence, not integrity). Audit all four pre-commit
    gates plus CI, and for each write down the property it claims and the property it
    tests. Fix the gaps. This is the systemic version of A.

  • Variant C (fallback) — Wire --with-sva into SymbiYosys. Unchanged and still the
    cheapest formal claim available: the SVA is emitted but never checked. One .sby
    harness per BitNet module plus a CI job.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions