Skip to content

v0.17.0 — negative assertions carry scope and shelf life

Choose a tag to compare

@Redtropig Redtropig released this 29 Jul 18:07
· 24 commits to main since this release
4a1d45f

The plugin's anti-hallucination contract bound only positive claims — "done", "fixed", "passing". A negative claim — "clang-tidy isn't installed here", "there's no such function" — was bound by nothing, and the skill's own description carried no negative trigger, so it was not even loaded at the moment it was needed.

This release makes the contract bidirectional: a claim that something is absent must state the scope it searched and the date it looked. It also fixes the two sensors that were making unverified negative claims of their own.

The two headline fixes

doc-drift-scan.sh could not see a single one of 0.16.0's own 21 changed files. It was pathspec-limited to C/C++ in a bash-and-markdown repository — and reported that by returning exactly what a clean scan returns: exit 0, empty stdout. "Did not scan" and "scanned, found nothing" shared a channel, which is how total blindness survived a whole release unnoticed.

Now it scans a maintained language whitelist (C/C++, Python, JS/TS, Go, Rust, Ruby, Java, Kotlin, C#, shell), and every early-exit path announces itself on stderr. stdout stays a pure candidate contract.

cpp-tool-discovery.sh searched versioned tools against a hard-coded ladder ending at 22 — a roughly twelve-month fuse. When LLVM 23 ships, an installed clang-tidy-23 would report NOT_FOUND, recreating the exact bug the script was written to prevent. A hard-coded version ceiling is itself an expiring negative assertion. Replaced with glob enumeration: no ladder, no fuse.

Performance

doc-drift-scan.sh, identical range (--base v0.16.0, 44 symbols × 53 docs, 4363 candidates), same machine:

wall clock
pre-release nested loop 5m44.427s
shipped 3.917s

Output byte-identical apart from a trailing colon the old IFS=: read silently truncated.

The first attempt regressed. Collapsing the search to one alternation grep per document left attribution spawning two greps per candidate line, and candidates outnumber symbol-file pairs — it went from an 8m25.794s baseline to a >10min timeout kill. What shipped replaces attribution with one awk pass per document. The CHANGELOG records that history rather than claiming a clean win.

Also in this release

  • Observation dates on negative capability conclusions at seven sites, plus tests/unit/mandated-phrasing.sh — the wording rule was held together by instruction alone through 0.16.0, whose own review found it already drifted.
  • init-verification re-checks inherited negative conclusions at session start. Only negative ones: those fail silently, while a stale positive fails loudly at the next invocation.
  • tests/README.md's "Quick test" block enumerated 7 of the 18 unit tests that exist — anyone following the documented steps ran under 40% of the suite while believing they had run it. Replaced with a glob, matching what CI already does.
  • Design invariant 8 restated bidirectionally, along with every other site carrying the one-directional form — including the meta-skill's Hard Rule 1, injected at every SessionStart.
  • A sweep for the set -u bare-local crash class across scripts/, hooks/, tests/, templates/. Zero instances beyond the one fixed here. Filed as a convention rather than a validate-anchor check, because neither grep nor shellcheck can decide it — a line-order scanner reports clean on code that crashes.

A defect this release shipped, and what caught it

The attribution stage handed awk its symbol list via -v syms="$CHUNK_SYMS". That value contains newlines, and POSIX does not permit a physical newline in a -v assignment — gawk --posix rejects it outright, and macOS's one-true-awk silently attributed nothing. Every symbol assertion failed on macos-latest and windows-latest while the same file reported 21/21 on the GNU-awk dev box and on ubuntu.

Every local check — the byte-for-byte parity diff, both timing runs, all RED/GREEN cycles — ran against a single awk. A green suite on one implementation was not evidence about any other, and nothing in six rounds of plan self-review, seven task reviews, or a whole-branch review could see it. CI was the only observer carrying that dimension.

Fixed by passing values through ENVIRON, which is verbatim. A portability assertion now re-runs the motivating case with awk forced into strict POSIX mode. Against the -v version it is the single failing assertion while the other 21 still pass — exactly the shape that let it through.

Known limitations

  • The new adversarial trigger prompt held 2/3, not 3/3. Read it with its baseline: the same prompt also triggered against the 0.16.0 description, which contained no negative keyword at all (N=1, PASS). So 2/3 is not evidence that adding negative triggers improved triggering — three runs per arm do not show it to be the description's keywords. Recorded beside the case in run-all.sh. The description change is justified on contract grounds, not on measured triggering.
  • Prefix matching is deliberately loose. cancel must match the prose word "Cancellation", so there is no trailing word boundary. On a common-word symbol this floods — 2840 candidates on this repo. Candidates are for a human or agent to judge, never verdicts.
  • Symbol extraction still only sees call/definition-shaped tokens. A changed global, macro, struct field, or a doc sentence naming no symbol at all remains invisible. Frozen as negative assertions in the unit test so a silent pass is never mistaken for coverage.
  • The contract binds only when the skill loads, and it loads from its description. A negative claim made in a turn where the description didn't match is unguarded. init-verification's session-start re-check is the second line of defence — one session late, which is not never, but is also not here.

Verification

Re-run on main at the tagged commit, not carried over from the branch:

suite result
validate-anchor 157
validate-manifests 3
cpp-tool-discovery unit 15
doc-drift-scan unit 22
mandated-phrasing unit 9
windows-compat 19
posix-compat 4
check-coverage 28
measure-context 6807 / 5153, both under the 12000 cap
CI-identical shellcheck gate clean, 66 files
CI 9/9 green

On-box acceptance, Windows + MSVC:

FOUND      clang-tidy    .../VC/Tools/Llvm/x64/bin/clang-tidy        vs-llvm
FOUND      clang-format  .../VC/Tools/Llvm/x64/bin/clang-format      vs-llvm
FOUND      ninja         .../CMake/Ninja/ninja                       vs-cmake
NOT_FOUND  cppcheck      searched:PATH,vs-llvm,vs-cmake,versioned

Full changelog: v0.16.0...v0.17.0