We stress-tested our own tool with 32 AI agents. Then fixed what they found.
A 4-lane adversarial audit (every command file, hostile synthetic vaults against every script, live retrieval quality on a 2,300-note vault, and a competitor teardown) produced 175 findings. This release closes ~160 of them across 24 PRs (#100-#123), in four phases.
Phase A - Data safety: tools that cannot eat your notes
heal_links/triage_linkscould permanently corrupt non-UTF-8 notes and silently rewrite line endings: strict UTF-8 in, byte-exact out, with non-UTF-8 files skipped loudly (#100)- One dangling symlink no longer kills the health check, the healers, or the exporter (#101)
- The link healer only auto-fixes certain matches now: no more foreign-alphabet links "matching" unrelated notes, no
[[C++]]->C.md, no edits inside code fences (#102) - A leading BOM can't hide frontmatter; exports translate faithfully or say they failed; stats stopped double-counting after exports; link_graph actually mirrors vault_health (pinned by a drift-alarm test) (#103-#107)
- A fresh bootstrap passes its own health check with zero findings - now a CI invariant (#108)
Phase B - Retrieval: search 2-4x better, measured honestly
- The eval harness itself was lying (its "lexical" mode was secretly fused; "hybrid" double-counted semantic). Straightened first, so every claim below is real (#109)
- Query-aware defaults: exact tokens go lexical, semantic votes lead the fusion (#110, re-tuned in #116)
- The silent 2,000-file scan cap is gone; ranking learned that volume is not relevance (logs fade, dossiers speak per-section); the index reached 100% coverage via adaptive chunk-splitting (#111-#113)
- Freshness: "current"-intent queries stop surfacing superseded notes and declined offers above the current truth (#114)
- Multilingual default model (bge-m3): non-English queries went from ~0 to working, and English improved too. Migration:
ollama pull bge-m3, then rebuild the index once (--build, ~10 min; existing indexes keep answering correctly until you do) (#115) - The reference numbers are committed in
scripts/eval/BASELINE.md: keyword recall@10 1.0, paraphrase recall@10 0.77, non-English recall@5 0.63 (#116)
Phase C - Commands and docs tell one truth
- 37 hardcoded folder paths across 18 commands now resolve via the folder map -
/obsidian-exportno longer produces an empty snapshot on default-bootstrapped vaults (#117) - 50 banned characters purged from instruction files - the skill stops teaching agents to write what its own validation hook blocks (#118)
- Schema reconciliation: 4 missing note-type schemas added, 8 commands aligned, and the real-world exceptions (quick capture, kanban boards, raw sources, vault-surface files) are now documented law the hook understands (#119)
- 20 "manual for a different machine" bugs fixed, including a status check that read its own output as evidence (#120)
- SKILL.md's six drifted sections now defer to the command files as the single source of truth; README counts match the filesystem; builds ship no Python bytecode (#121)
- Five new CI lints enforce all of the above permanently
Phase D - The front door
- The README's one-line installer pointed at a file that did not exist. It exists now (idempotent), prerequisites are stated up front, there's a path for users with no vault yet, and the bootstrapper never overwrites your files - keep-by-default,
--forceis the consent (#122) - Every platform dist ships a runnable Python project, so the research toolkit works on Codex, Gemini, OpenCode, Hermes, and Pi - not just Claude Code (#123)
After the sprint closed (same day)
- External contribution (one of 17 outside contributors to date): /research and /research-deep silently ignored a
PERPLEXITY_API_KEYset in the config.env- the free-vs-paid decision ran before anything had loaded the file. Root-caused and fixed by @MichaelHabermas (#125, fixes #124), now fenced in CI - The audit's three never-tested surfaces (bg-agent hook, MCP write path, Telegram ingest) ran live; the one real bug found - Telegram ingest hardcoding wiki-style folders - is fixed with folder-map resolution (#126)
Numbers
- Test suite: 30 -> 129 tests, including 11 permanent invariants (byte-safety, ghost resilience, drift alarms, folder/character/schema/roster lints, the showroom rule, front-door safety, dist runnability)
- Full changelog: see
CHANGELOG.mdunder 0.12.0
Breaking / migration notes: the default embedding model is now bge-m3 (OBSIDIAN_EMBED_MODEL overrides; one-time index rebuild recommended); vault_stats accepts --path (old --vault still works); bootstrap keeps existing files by default (use --force for the old overwrite behavior); the semantic index format is v2 (rebuilds automatically invalidate old caches).