Skip to content

v1.36.0 — 检索结果的可信度判定、空壳捕获的审计、安装脚本数据化

Latest

Choose a tag to compare

@Spc-jgs Spc-jgs released this 24 Aug 03:05
· 26 commits to master since this release
4e370f6

Added

  • search-vault says whether it answered the question. Asked something the Vault does not cover, it returned hits with a score, a heading and a snippet — the shape of a search that succeeded — so a caller could not tell "here is the answer" from "here is the lexically nearest noise". An Agent either cites a Python note for a Feign question, or, worse, concludes the topic is already covered and never captures it. Every response now carries confidence, keyed on IDF-weighted coverage of the typed query: how much of the question's information the winner actually matched. IDF rather than a stop-word list, because a list needs a countable source and question frames like 有什么 have none; typed words only, because letting the ranker's own query expansion certify the ranker's results is circular. Ranking is unchanged.

    The level has two values, not three, because the measurement supports one cut and not two. On the reference Vault, 22 no-answer queries score 0.09–0.54 and 16 correctly-answered ones score 0.32–0.64 — overlapping ranges. 0.30 demotes none of the 16 while catching 20 of the 22; 0.60 would catch all 22 and demote 12 of the 16. So none is a finding and evidence is merely the absence of one: it does not claim the answer is right, and two of eighteen measured questions carry evidence with a wrong top result.

  • The audit reports a web-clip that captured nothing (web-clip-captured-nothing, defect). Two notes on the reference Vault are placeholders left by a blocked fetch, and the audit's only word on them was web-clip-missing-author at hygiene — while retrieval treated them as ordinary captures: hit, cited, and counted as evidence the topic was already covered. empty-template-note could not reach them, because it fires on content_chars == 0 and placeholder prose is still characters; one of the two has no heading at all, failing its other precondition too.

    The criterion is a floor on body content, scoped to web-clip and skipping notes that carry the Vault's draft tag. Ranked by that count, the Vault's 55 web-clips put both shells first and second (100 and 220 characters), the two self-declared drafts next (329, 383), and the smallest real capture at 799. WEB_CLIP_MIN_CONTENT_CHARS = 400 sits just below the geometric midpoint of 220 and 799, leaning toward missing a shell rather than accusing a real note. The type scoping is load-bearing, not cautious: 92 notes of all types fall under 800 characters and 87 of them are legitimate — 45 daily reports, 23 folder indexes, 9 weekly reports. A genuinely short source is the known false positive, named in docs/superpowers/specs/2026-08-24-shell-capture-detection-design.md rather than denied.

Fixed

  • install.ps1 could delete the contents of a symlink target instead of unlinking it. Uninstalling the base Skill under QoderWork and under Codex used Remove-Item -Recurse -Force, while the other seven paths used Remove-OwnedPath. On a directory symlink — which is what a Skill manager creates — the recursive form follows the link and deletes what is on the other side. The parallel hand-copy in install.sh was wrong in a different place: QoderWork's base tested -d where the other eight tested -d || -L, so an installation whose Skill directory was a symlink was skipped entirely by uninstall.

    Both are consequences of the same shape rather than two independent slips: each script carried 16 hand-copied path literals across its install, uninstall and host-validation branches, with no shared source of truth between the two languages. Both now expand one table, so neither inconsistency can be expressed again.

  • A # comment inside a fenced code block is no longer read as a heading. search_vault had no notion of a fence and scanned for ^#[ \t]+ line by line, which polluted three things at once. Two notes on the reference Vault took their title — scored at 6x body — from a line inside a ```bash block, so they lost that weight on their own subject and gained it on a shell comment. Headings the author never wrote scored at 2x. And the passage split, which is the unit ranking works on, was cut short at phantom boundaries: 22 of 199 notes carried 255 such false headings, one of them turning 100 sections into 52. Short passages are barely penalised by BM25, so notes scored high on subjects they only mention in passing.

    The fix reads the fence notion that already exists — link_graph.blank_code_examples, already used by explore-neighborhood and relatedness — rather than writing a fourth scanner. It blanks code line by line while preserving numbering, so one index addresses both the blanked copy and the body: split on the blanked copy, read content from the original. Discarding code outright would trade this defect for a worse one; npx playwright install chromium still has to be findable, and both halves are asserted. On the 20 annotated queries MRR moved 0.885 → 0.900.

Changed

  • Both installers' Skill and host layouts are data. One table per script — SKILL_ROWS/$SkillRows and HOST_ROWS/$HostRows — expanded by install, uninstall and validation alike. Deliberately two tables rather than one product, because hosts are not uniform: Cursor takes retrieval only, and Claude Code additionally migrates a legacy marker block. Steps that are not Skill payload stay in the case/switch, so a host appears there only when it needs an extra action. The guards changed with the shape: "everything installed can be uninstalled" used to be a count of literal occurrences and is now true by construction, leaving one assertion that no branch grows a path outside the table.

  • BM25's parameters are named BM25_K1 and BM25_B at module level instead of being literals inside _bm25_score. They are the textbook defaults, which is exactly why they need a guard — a round number reads as unexamined — and a sweep is now a one-line change rather than an edit to a function body.

Decided and recorded, with the losing side kept

Four rulings, each with the measurement that produced it and what would reopen it.

  • Lowering BM25's length penalty is not the fix for "the fuller note ranks lower". Sweeping b from 0.00 to 1.00, b=0.25 looks best in aggregate (17/18 versus 16/18, MRR 0.972 versus 0.944) — but only 4 of the 18 queries move at all, so three notes decide the whole result, which is the overfitting the issue itself warned about. The direction is wrong where it matters: of four same-source "excerpt versus full" pairs, b=0.25 fixes one and pushes another's full note out of the Top-5, and the average size of a no-answer query's top result grows from 5916 to 10667 bytes. 2026-08-21-rejected-hypotheses.md §6.

  • The adversarial corpus keeps its shape, and no second everyday corpus is built. Both the issue and an earlier fix measured file bytes; _bm25_score normalises by average_scoring_length, a different unit, and on that unit the corpus sits 2.18x above the reference Vault with nothing at all in the 1–5x band where that Vault keeps 39.7% of its notes. Reshaping was measured, not argued: it leaves the failure the issue wanted exposed exactly where it already is (rank 2) while moving three must_see ranks the wrong way and erasing what the dilution family exists to show. The divergence is now pinned at 2.0–2.4 by assertion so that either narrowing or widening it forces the ruling to be re-read. 2026-08-23-adversarial-corpus-shape-decision.md.

  • A helper cannot be told that a capture's fetch failed, so preventing a shell note at write time is closed. Nothing in the package performs network I/O — the fetch belongs entirely to the Agent, and every fact about it a helper receives is a fact the Agent chose to type. The one name that describes the fetch outcome, retrieval_status, is deliberately never persisted: it belongs to the in-run self-check whose opening line is "do not persist it as telemetry". Requiring the Agent to declare it does not rescue the route, because the only witness to a failed fetch is the actor that failed it. 2026-08-21-rejected-hypotheses.md §7, guarded by an assertion that the package still cannot reach the network.

  • A correction to this file's own v1.35.0 entry, and to the record it summarised. That entry says the two shell notes are "prose an Agent composed while breaking that rule". They are not: Terminal Failure Means Zero Writes, and the whole of core/references/web-capture.md, was created on 2026-07-31, and the notes were written on 2026-07-22 and 2026-07-23 — eight and nine days earlier. That day's write path accepted them, because its metadata predicate asked only whether a field was a non-empty string and unknown is non-empty; today's create-note --apply refuses both. A third note, recorded as an author's deliberately brief draft, turns out to say in its own body that its fetch was blocked by Cloudflare — and that misreading was the sentence that killed the structural predicate. Rescored within web-clips and excluding self-declared drafts, that predicate finds the shell with no false positives, so #167 stayed open carrying the measurement instead of being closed by it.