fix(server): block path traversal in register_source_from_path#28
Merged
plind-junior merged 4 commits intoMay 20, 2026
Merged
Conversation
kb.register_source_from_path read the contents of any file the process could access, letting an agent register /etc/passwd, ~/.ssh/id_rsa, ~/.aws/credentials etc. as a "source" and then retrieve the bytes via kb.cite or kb.list_sources. Resolve the path (following symlinks) and require it to be inside the KB root before reading. Adds KBStore.resolve_under_root() so both the JSONL handler and the MCP tool share one containment check. Fixes vouchdev#10
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughWalkthroughAdds KBStore.read_under_root and uses it in server/jsonl registration to forbid reading files outside the project root; updates multiple put_* methods to re-raise ValueError from FileExistsError; adds tests for out-of-root rejection, in-root acceptance, symlink TOCTOU rejection, and directory rejection; small typing and session persistence tweaks. ChangesPath Traversal Prevention & Storage Error Chaining
🎯 4 (Complex) | ⏱️ ~45 minutes
🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tests/test_jsonl_server.py (1)
68-104: ⚡ Quick winAdd the symlink-escape regression case.
These tests cover “outside absolute path” and “inside regular file,” but not the explicit symlink escape called out in
#10. Please add a case where a path understore.rootis a symlink to a file outside the root, so future refactors keep thePath.resolve()containment behavior locked in.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_jsonl_server.py` around lines 68 - 104, Add a regression test that asserts symlink-escape is rejected: create an outside file (e.g., secret.txt), create a symlink inside the KBStore root pointing to that outside file, chdir to store.root, call handle_request with method "kb.register_source_from_path" and params.path pointing to the symlink, and assert the response is not ok with error.code "invalid_request" and that the message mentions "project root" and that KBStore.list_sources() remains empty; this ensures the resolve/containment check (Path.resolve()) used by KBStore.register_source_from_path continues to prevent symlinks from escaping the project root (name the test something like test_register_source_from_path_rejects_symlink_escape).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/vouch/storage.py`:
- Around line 114-124: resolve_under_root currently returns a Path after a
containment check allowing a TOCTOU where callers later call
is_file()/read_bytes(); change resolve_under_root to perform the open/read under
the same trusted check and return a stable handle or bytes instead of a raw
Path. Specifically, locate resolve_under_root and its callers (e.g., any code
calling resolve_under_root(...).is_file() or .read_bytes()), perform
Path(...).resolve(), verify it is inside self.root, then open the file
(preferably with open(resolved, "rb") or using os.open with O_NOFOLLOW if you
must reject symlinks) and return the file object or its bytes so callers use
that stable handle rather than re-checking the filesystem; ensure symlinks are
handled per existing semantics and exceptions propagate as ValueError when
outside self.root.
---
Nitpick comments:
In `@tests/test_jsonl_server.py`:
- Around line 68-104: Add a regression test that asserts symlink-escape is
rejected: create an outside file (e.g., secret.txt), create a symlink inside the
KBStore root pointing to that outside file, chdir to store.root, call
handle_request with method "kb.register_source_from_path" and params.path
pointing to the symlink, and assert the response is not ok with error.code
"invalid_request" and that the message mentions "project root" and that
KBStore.list_sources() remains empty; this ensures the resolve/containment check
(Path.resolve()) used by KBStore.register_source_from_path continues to prevent
symlinks from escaping the project root (name the test something like
test_register_source_from_path_rejects_symlink_escape).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 33df703d-14c8-475e-82f6-c8a6de5aa465
📒 Files selected for processing (4)
src/vouch/jsonl_server.pysrc/vouch/server.pysrc/vouch/storage.pytests/test_jsonl_server.py
Member
|
Update the PR description and fix ci |
ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged.
The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself.
CodeRabbit on PR vouchdev#28 noted that resolve_under_root() only validated a pathname snapshot. Callers then re-opened the same name with .is_file() and .read_bytes(), so an attacker who can swap the resolved path for a symlink between the containment check and the read can still exfiltrate an out-of-root file via kb.register_source_from_path. Collapse the validate-then-read into a single trusted helper `KBStore.read_under_root(path)` that returns `(resolved, bytes)`: 1. Path.resolve() chases pre-existing symlinks and the resulting target is checked for containment (existing behaviour -- legitimate in-root symlinks still work). 2. The read goes through `os.open(resolved, O_RDONLY | O_NOFOLLOW)` so a fresh symlink placed at the resolved name *after* the check fails with ELOOP rather than following the swap. 3. `fstat` + `S_ISREG` rejects directories / device nodes / pipes atomically on the same fd, replacing the racy `is_file()` test the callers used to do. Both register_source_from_path handlers (MCP + JSONL) switch to the new helper and drop their now-redundant follow-up checks. Adds tests: - symlink swapped into the resolved name -> rejected via ELOOP - directory at a valid path -> rejected via S_ISREG Existing "outside the root" and "inside the root" tests still pass.
Member
|
LGTM! |
4 tasks
plind-junior
pushed a commit
that referenced
this pull request
May 22, 2026
CodeRabbit on PR #28 noted that resolve_under_root() only validated a pathname snapshot. Callers then re-opened the same name with .is_file() and .read_bytes(), so an attacker who can swap the resolved path for a symlink between the containment check and the read can still exfiltrate an out-of-root file via kb.register_source_from_path. Collapse the validate-then-read into a single trusted helper `KBStore.read_under_root(path)` that returns `(resolved, bytes)`: 1. Path.resolve() chases pre-existing symlinks and the resulting target is checked for containment (existing behaviour -- legitimate in-root symlinks still work). 2. The read goes through `os.open(resolved, O_RDONLY | O_NOFOLLOW)` so a fresh symlink placed at the resolved name *after* the check fails with ELOOP rather than following the swap. 3. `fstat` + `S_ISREG` rejects directories / device nodes / pipes atomically on the same fd, replacing the racy `is_file()` test the callers used to do. Both register_source_from_path handlers (MCP + JSONL) switch to the new helper and drop their now-redundant follow-up checks. Adds tests: - symlink swapped into the resolved name -> rejected via ELOOP - directory at a valid path -> rejected via S_ISREG Existing "outside the root" and "inside the root" tests still pass.
plind-junior
added a commit
that referenced
this pull request
May 22, 2026
…versal fix(server): block path traversal in register_source_from_path
3 tasks
plind-junior
added a commit
that referenced
this pull request
Jul 7, 2026
* fix(server): block path traversal in register_source_from_path kb.register_source_from_path read the contents of any file the process could access, letting an agent register /etc/passwd, ~/.ssh/id_rsa, ~/.aws/credentials etc. as a "source" and then retrieve the bytes via kb.cite or kb.list_sources. Resolve the path (following symlinks) and require it to be inside the KB root before reading. Adds KBStore.resolve_under_root() so both the JSONL handler and the MCP tool share one containment check. Fixes #10 * fix(cli): translate domain errors into clean ClickException output The CLI handlers for approve and reject caught only (ArtifactNotFoundError, ValueError) but proposals.approve() / proposals.reject() raise ProposalError -- a RuntimeError subclass. The four propose-* shortcuts caught nothing at all. Result: a double- approve, an empty rejection reason, an empty claim text, or an unknown source id surfaced a raw Python traceback instead of the one-line `Error: ...` the rest of the CLI emits. Add a small `_cli_errors` context manager that translates ArtifactNotFoundError / ValueError / ProposalError / LifecycleError into click.ClickException, and apply it to every command that calls into proposals, lifecycle, sessions, or storage. The MCP and JSONL servers already do the equivalent in their own envelopes; this brings the human-facing surface in line. Adds tests/test_cli.py with regressions for approve, reject, propose- claim, propose-entity, and show. * chore(lint): chain FileExistsError cause in storage put_ methods ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged. * fix(verify): catch ArtifactNotFoundError and OSError on stored read verify_source() caught FileNotFoundError, but store.read_source_content() raises ArtifactNotFoundError (a KeyError subclass) when the content blob is missing -- so the "stored content missing" graceful path never ran and a single broken source crashed the entire verify_all() sweep, breaking `vouch source verify` and `vouch doctor`. The same call can also raise OSError (permission denied, TOCTOU race between exists() and read_bytes(), underlying I/O error) which was likewise unhandled. Catch both: the existing "missing" path keeps its note, and OSError surfaces as "stored content unreadable: <reason>". Adds three regression tests: - missing-content blob -> graceful per-source failure - unreadable stored content (monkeypatched PermissionError) -> graceful per-source failure with the underlying reason in the note - mixed sweep with one good + one broken source -> verify_all returns both results instead of aborting at the first failure Fixes #30 * chore(ci): unblock lint/type/test gates so the CLI fix can land The CI workflow runs `ruff check`, `mypy`, then `pytest`, and the fix/cli-clean-domain-errors branch was failing all three on pre-existing main-branch issues unrelated to the CLI change itself. * ruff: add `from e` to the seven `raise ValueError(...) from FileExistsError` re-raises added by the recent exclusive-create guards in storage.put_* (B904), and let ruff re-sort the cli.py import block (I001). * mypy: narrow the `kind` value flowing into `ContextItem(type=...)` with a `Literal` cast so the strict-typed field accepts what the search backends actually return. * pytest: `session_end()` mutates a session that `session_start()` has already written, but `put_session()` now uses exclusive create and rejects the second write. Add `KBStore.update_session()` mirroring `update_claim` and have `session_end` call it; existing test_sessions coverage now passes again. No behavior change for the CLI surface — these are infrastructure fixes so the existing test_sessions / lint / type assertions pass on this branch. * chore(ci): unblock mypy + test gates for path-traversal PR The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself. * fix(server): close TOCTOU window between path check and read CodeRabbit on PR #28 noted that resolve_under_root() only validated a pathname snapshot. Callers then re-opened the same name with .is_file() and .read_bytes(), so an attacker who can swap the resolved path for a symlink between the containment check and the read can still exfiltrate an out-of-root file via kb.register_source_from_path. Collapse the validate-then-read into a single trusted helper `KBStore.read_under_root(path)` that returns `(resolved, bytes)`: 1. Path.resolve() chases pre-existing symlinks and the resulting target is checked for containment (existing behaviour -- legitimate in-root symlinks still work). 2. The read goes through `os.open(resolved, O_RDONLY | O_NOFOLLOW)` so a fresh symlink placed at the resolved name *after* the check fails with ELOOP rather than following the swap. 3. `fstat` + `S_ISREG` rejects directories / device nodes / pipes atomically on the same fd, replacing the racy `is_file()` test the callers used to do. Both register_source_from_path handlers (MCP + JSONL) switch to the new helper and drop their now-redundant follow-up checks. Adds tests: - symlink swapped into the resolved name -> rejected via ELOOP - directory at a valid path -> rejected via S_ISREG Existing "outside the root" and "inside the root" tests still pass. * chore(lint): chain FileExistsError cause in storage put_ methods ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged. * chore(ci): unblock mypy + test gates for path-traversal PR The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself. * docs: add banner diagram to README * docs(spec): semantic search as primary retrieval backend Approved design for feat/semantic-search. Embedding (sentence- transformers all-mpnet-base-v2) becomes the primary search backend with FTS5 as fallback. Synchronous-at-write indexing across all six artifact types (claim, page, source, entity, relation, evidence). Maximally functional scope (~3000 LOC): pluggable model adapter registry, sqlite-vec ANN with NumPy fallback, cross-encoder rerank, HyDE query expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG eval harness, and full CLI/MCP/JSONL parity. The writing-plans step will turn the rollout order in section 16 into concrete phased tasks. * docs(plan): semantic search implementation plan 32-task TDD plan for the semantic-search feature: foundation, storage, write hooks across all six artifact types, semantic-primary search integration in MCP/JSONL/CLI, RRF fusion + hybrid, cross-encoder rerank, HyDE expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG scorer, end-to-end integration test, and user docs. Each task: failing test, minimal implementation, passing test, commit. MockEmbedder test double keeps the unit suite fast; the real model is exercised only under @pytest.mark.integration. The evaluation module is named scorer.py rather than eval.py to avoid shadowing the Python builtin and to keep static analysers quiet; the user-facing CLI subcommand remains `vouch eval embedding` (the Click group is registered under the name "eval", with the Python identifier eval_group). * feat(embeddings): add optional-deps extras and pytest markers * feat(embeddings): create package skeleton * feat(embeddings): Embedder ABC, registry, content_hash, MockEmbedder * feat(embeddings): sentence-transformers all-mpnet-base-v2 default adapter * feat(embeddings): sentence-transformers MiniLM-L6 alternative adapter * feat(embeddings): fastembed BGE alternative (no-torch) adapter * fix(ci): satisfy ruff SIM105 + add mypy overrides for optional deps CI ran ruff which flagged SIM105 on the three try/except ImportError guard blocks in src/vouch/embeddings/__init__.py. Replace each with `contextlib.suppress(ImportError)` -- same semantics, satisfies ruff. Also add mypy overrides for numpy / sqlite_vec / sentence_transformers / fastembed so the type check passes in the base [dev] CI install where those optional extras aren't present. The embedding code paths that import these libraries are only reached when the extras are installed at runtime; for the static type check the missing stubs are noise. * fix(ci): skip embeddings test suite when numpy isn't installed CI runs `pip install -e '.[dev]'` which deliberately excludes the optional `[embeddings]` extras. tests/embeddings/test_*.py modules import numpy at top level, so pytest collection fails with ModuleNotFoundError before any test can be deselected. Add tests/embeddings/conftest.py with `pytest.importorskip("numpy")` so the entire embeddings test directory skips gracefully when numpy is absent. Once `pip install vouch[embeddings]` (or numpy itself) is present, the skip is a no-op and the tests run normally. * fix(bundle): skip Pydantic validation for opaque source content files `_validate_content` in bundle.py uses the path's first directory component to look up a Pydantic validator. For sources, both `sources/<sha>/meta.yaml` (the Source model) and `sources/<sha>/content` (raw opaque bytes) hit the same "sources" key, so the validator was being run on the raw content bytes and failing with "1 validation error for Source". Add an early return for any non-meta.yaml path under `sources/` so opaque content bytes are not Pydantic-validated. Only the Source metadata file is checked, which matches the original intent of the PR #13 validation. Unblocks three pre-existing test_bundle.py failures inherited from upstream main. * fix(embeddings): MockEmbedder uses uint32 scaling to avoid NaN/Inf Per CodeRabbit on PR #37: decoding sha256 chunks as raw float32 (via struct.unpack("<f", ...)) produced NaN/Inf for ~1 in 2^7 chunks because most 4-byte patterns aren't finite IEEE754 floats. Norm overflow then made unit normalization unreliable and broke cosine similarity asserts in downstream phases (the dedup test in Phase 6 already had to work around this with a custom _HashEmbedder). Decode each chunk as an unsigned 32-bit integer, scale to [-1.0, 1.0], accumulate in float64 for exact norm, and cast to float32 on return. Deterministic, finite, well-behaved. * feat(embeddings): state.db schema for embedding storage + put/get helpers * feat(embeddings): NumPy brute-force cosine search over embedding_index * feat(embeddings): sqlite-vec ANN path with NumPy fallback * feat(embeddings): query embedding LRU cache * style: fix ruff E402/SIM105/E702/I001 violations from Phase 2 storage tasks * fix(embeddings): use outer-query for WHERE on cosine alias in search_embedding Per CodeRabbit Critical on PR #39/40/43: SQLite does not allow a column alias defined in the SELECT list to be referenced in the WHERE clause of the same query. The old form SELECT ... 1.0 - vec_distance_cosine(vec, ?) AS score ... WHERE ... AND score >= ? raised `no such column: score` at runtime; the catch-all `except sqlite3.OperationalError` silently swallowed it and made every call fall through to the NumPy brute-force path, which works but defeats the whole point of installing sqlite-vec. Wrap the projection in a subquery so the WHERE clause sees the alias. No behavior change for callers that have already been using the fallback (NumPy still produces correct results); the ANN code path now actually runs when sqlite-vec is loaded. * feat(embeddings): write-time embedding hook + wire put_claim * feat(embeddings): search_semantic wrapper with query cache * feat(embeddings): hook all six artifact write paths * feat(embeddings): kb_search defaults to embedding primary, fts5 fallback * feat(embeddings): re-embed on update_claim * feat(embeddings): JSONL _h_search parity with MCP semantic-primary dispatch * style: fix ruff SIM105/E501/I001/RUF059 violations from Phase 3 * fix(ci): silence mypy import-untyped for forward-ref fusion import The hybrid backend branch in kb_search (server.py) and _h_search (jsonl_server.py) lazily imports vouch.embeddings.fusion (added in Phase 5). On Phase 4's branch that module doesn't exist yet, so mypy emits import-untyped. Mark each import with `# type: ignore` and let ruff auto-format the line. * fix(ci): silence mypy import-untyped for forward-ref dedup import The KBStore._embed_and_store helper lazily imports vouch.embeddings.dedup (added in Phase 6). On Phase 3's branch that module does not yet exist, so mypy treats it as untyped and errors out. Mark the import with `# type: ignore[import-not-found,import-untyped, unused-ignore]` so the type check passes here and stays silent once Phase 6 lands. * spec: cut dated 2026-05-21 snapshot; promote schema generator to script * ci(mypy): collapse fusion import to single line for type-ignore * feat(embeddings): RRF/weighted/normalized fusion strategies * docs: add 'What ships today' table to README * kb: approve review-gate fact * docs: add 'When to use vouch' section * fix(cli): honor VOUCH_AGENT in _whoami for consistent actor attribution The CLI propose-* commands and most actor sites called _whoami(), which only checked VOUCH_USER and the OS user — VOUCH_AGENT was silently ignored. The MCP and JSONL servers (server.py, jsonl_server.py) and session_start already honour VOUCH_AGENT, so multi-agent attribution broke as soon as you used the CLI to propose. Prefer VOUCH_AGENT over VOUCH_USER over the OS user so the recorded proposed_by / audit actor matches across all three transports. * docs: add captured example session walkthrough Real propose -> review -> commit -> retrieve loop captured from a sandbox run on 2026-05-21. Includes on-disk YAML, audit log lines, and an honest note about the literal substring backend limitation pre-embeddings. * docs: add screen recording of full vouch loop (VHS tape + GIF) * docs: embed demo GIF under Quick start in README * fix(embeddings): address CodeRabbit review on hybrid + fusion + write hook Bundles the substantive correctness fixes flagged by CodeRabbit on PR #41. CI was already green; these are latent bugs the review caught. storage._embed_and_store - Re-embed when the active embedder model changes, not only when the content hash changes. The previous short-circuit kept stale vectors on disk after a model swap, mixing document embeddings from one model with queries from another. - Truly best-effort: catch any exception during encode/put/meta/dedup so a hook failure can't bubble back to the caller and leave an artifact persisted-but-API-errored. index_db.reset - Also clears embedding_index, query_embedding_cache, embedding_dupes, and embedding_* keys from index_meta. Otherwise `vouch index` left orphaned hits behind after artifacts were removed. index_db.search_embedding (Python fallback) - Normalize the stored vector by its own L2 norm before scoring, so rankings match the sqlite-vec `1 - vec_distance_cosine` path. The previous raw dot product made magnitude leak into the score. index_db.search_semantic - Invalidate the query-vector cache entry when the embedder dim no longer matches what's on disk; otherwise a model swap returned stale vectors living in the wrong space. server.kb_search + jsonl_server._h_search (hybrid branch) - Hybrid now passes min_score into search_semantic (consistent with embedding/auto branches) and wraps the FTS lookup in try/except so an FTS5 error doesn't fail the whole request. - JSONL transport rejects unknown backend values with a clear error instead of silently returning []; matches MCP transport behavior. embeddings.fusion - rrf_fuse validates limit >= 0 and k >= 0 (k = -rank would divide by zero). weighted_fuse validates limit >= 0. Tests - Three new fusion guard tests cover the negative-limit and negative-k paths. * fix(lifecycle): only catch missing-artifact errors in cite() the bare except was treating every error as a missing citation, which hid real bugs. narrow it to ArtifactNotFoundError so unexpected failures actually surface. * fix(context): only catch missing-artifact errors in citation lookup same issue as cite() — a bare except was hiding real errors behind an empty citation list. narrow it to ArtifactNotFoundError. * refactor(sessions): keep tracebacks when crystallize() approve fails we were dropping the traceback and only stringifying the error, which made it painful to diagnose anything other than ProposalError. now logs via logger.exception and includes error_type in the failures list. * fix(context): only catch sqlite errors when FTS5 falls back the retrieval helper had a bare except that would also swallow bugs in the substring fallback path or in our own code. narrow it to sqlite3.Error so only the intended cases (FTS5 missing, db missing, schema mismatch) trigger the fallback. * refactor(health): use typed status filter for pending proposal count we were listing every proposal and comparing p.status.value == 'pending' as a string. list_proposals() already takes a ProposalStatus filter — use it. shorter, type-checked, and avoids the stringly-typed compare. * chore(capabilities): pin pluginApi compat baseline, fail CI on host-compat drift * feat(dual-solve): surface engine logs * fix(sessions): make crystallize retry idempotent for summary pages Partial crystallize runs that already wrote session-{id} summary pages failed on retry with "page already exists". Upsert the summary page and derive its artifact list from all approved session proposals. Fixes #139 Co-authored-by: Cursor <cursoragent@cursor.com> * fix(models): reject empty text/name/title on claim/entity/page the non-empty contract for `Claim.text`, `Entity.name`, and `Page.title` lived only in the `propose_*` helpers, so `store.put_*`, `store.update_*`, and bundle/sync import (via `_validate_content`) all silently accepted empty or whitespace-only values and landed artifacts carrying zero of the field's semantic content. add `@field_validator`s on each field, mirroring the existing `Claim.evidence` min-citation validator (#81/#82): they reject blank input at construction time, so every write path — direct construction, put/update, and import — inherits the check at once. the validators gate on emptiness only and preserve surrounding whitespace of non-blank values. `Source.locator` is deliberately left out of scope — its format question is bigger and deserves its own issue, as the report notes. closes #155 * refactor(models): extract shared _require_non_empty validator helper address review on #300: the three #155 field validators (Claim.text / Entity.name / Page.title) were structurally identical strip-checks. extract a module-level `_require_non_empty(v, label)` helper they all delegate to, keeping the per-field error messages. also fix a mid-entry sentence casing in the CHANGELOG. no behaviour change. * fix: resolve RPC traceback leakage on internal errors Unexpected exceptions in handle_request() included full stack traces in the JSON error envelope, exposing paths and internals to HTTP /rpc clients. Log server-side and return message only. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(review): kb.triage_pending — advisory triage scoring for the pending queue a long `kb.list_pending` forces the reviewer to reconstruct, per proposal, whether the claim fits the existing kb, whether its citations resolve, whether it duplicates something already filed, and whether it contradicts an approved claim. this adds an optional triage pass that scores each pending proposal on those four signals and attaches a `_meta.vouch_triage` block (recommendation/score/signals/rationale) to help a reviewer prioritize, without ever deciding anything itself. read-only by construction: the pass never calls proposals.approve/reject, store.put_*, or store.move_proposal_to_decided — a human still calls kb.approve/kb.reject. citation_quality reuses proposals._payload_block_reason; duplication_risk reuses the propose-time embedding similarity path (embeddings.similarity.find_similar_on_propose) and degrades to a difflib heuristic when the embeddings extra isn't installed. fit uses a separate, lower-threshold embedding search so a near-duplicate hit doesn't also inflate fit and cancel out its own duplication penalty. opt-in via `triage.enabled: true` in config.yaml (default false). registered at all four kb.* surface sites (server.py, jsonl_server.py, capabilities.py, cli.py) plus `vouch triage [proposal-id...]` with `--json` and `--reverse`. * feat(diff): register kb.diff at all four kb.* surface sites vouch diff shipped CLI-only in 0.1.0, an explicit non-goal at the time ("MCP/JSONL parity ... kb.* surface unchanged"). that leaves it the only read method skipping the four-site registration convention documented in CLAUDE.md, so agents talking MCP/JSONL have no way to fetch a revision diff. adds the kb_diff MCP tool, the kb.diff JSONL handler, and the capabilities.METHODS entry, next to the other by-id read tools (kb_read_claim/kb_read_page) with the same unrestricted-read posture. also makes new_id optional for a superseded claim: diff_artifacts and the CLI both resolve it from superseded_by when omitted, erroring clearly when there's no successor (pages still require an explicit new_id — they have no successor pointer). closes #327. * fix(jsonl): define module logger after the import block #361 inserted `_log = logging.getLogger(...)` between two import groups in jsonl_server.py. that statement-among-imports trips ruff's E402 on every import that follows it, so `ruff check src tests` — the lint gate in ci — now fails on the whole repo. move the logger definition below the imports (where storage.py already keeps its own module logger); no behaviour change. * feat(adapters): toml_merge install strategy for codex config.toml .codex/config.toml is codex's primary config file, so the plain-copy path silently skipped any project where codex was already configured and vouch never got wired. add a toml_merge entry flag mirroring json_merge: parse the existing destination with tomllib, deep-merge the template's tables into it (existing user values always win on conflict), and write back. flip the codex T1 entry to toml_merge. writing uses a minimal hand-rolled serializer (tables, arrays, inline tables in arrays, scalars, datetimes) so the dependency set stays unchanged; the output must survive a tomllib round-trip back to the merged data, and anything the serializer can't faithfully re-emit degrades to skipped rather than risking the user's config. closes #384 * feat(adapters): codex T2 agents.md fenced snippet codex reads AGENTS.md for project instructions the way cursor does, but the codex adapter stopped at T1, so a codex session got the kb tools with no standing guidance on recall-first or the review gate. ship adapters/codex/AGENTS.md.snippet as a T2 tier with the standard fence markers, kept in lockstep with cursor's snippet modulo the host name (enforced by a sync test). also close the edited-fence gap in _install_fenced per the ticket's acceptance criteria: a fence body that drifted from the shipped snippet is replaced within the markers (reported as merged) instead of being skipped forever, while user content outside the fence stays untouched. a begin marker without an end marker is treated as corrupt and left alone. closes #385 * feat(adapters): codex T3 skills mirroring the vouch slash commands claude-code T3 ships nine slash commands; codex users got none of the guided flows. ship them as codex skills under the project-local .codex/skills/ as a T3 tier. scope decision the ticket asked to resolve first: codex loads custom prompts only from ~/.codex/prompts/ (user-global) and has deprecated them upstream in favour of skills, which do have a project-local home at <project>/.codex/skills/ in trusted projects. the #179 rule forbids a project-scoped install from touching home-directory state, so prompts are out and skills are in — recorded in the manifest comment and the adapter readme, with a manual-copy pointer for users who still want ~/.codex/prompts. the SKILL.md files are referenced from the openclaw mirror rather than duplicated (same reuse pattern openclaw itself applies to the claude-code commands), so one edit updates every host. a parametrized sync test asserts each installed codex skill body equals the matching claude-code command body, and a scope test asserts every installed path stays under the project target. closes #386 * feat(capture): ingest codex session rollouts into review-gated summaries session auto-capture was claude-code-only: hooks drive capture observe and capture finalize live. codex has no hook stream, but it persists every session as a rollout jsonl under $CODEX_HOME/sessions containing user messages, tool calls, and outputs — everything the existing rollup needs, just after the fact. new cli command `vouch capture ingest-codex [<rollout> | --latest]` parses one rollout through a small dedicated parser (codex_rollout.py) that maps function_call records into the same observation shape capture.observe produces — shell commands with failure detection from exit codes, apply_patch heredocs surfaced as file edits, mcp tools under their own names, session mechanics skipped — then reuses the existing build_summary_body -> propose_page rollup. one code path from observation to proposal, two front doors. the rollout format is not a stable public contract: unknown record types are tolerated, and unreadable, compressed, or meta-less files degrade to a CodexRolloutError with an actionable message and a non-zero exit, never a stack trace. re-ingesting a session is a no-op keyed on the rollout's session id; --latest resolves the newest rollout whose recorded cwd matches the current project. proposals are attributed to the codex actor (VOUCH_AGENT wins when set), respect capture's enabled/min_observations config, and never touch approve(). fixture rollouts use placeholder data only, enforced by a test. closes #387 * feat(adapters): codex T4 hook wiring for automatic session capture with `vouch capture ingest-codex` available, codex can self-capture the way claude-code T4 does. the ticket assumed the `notify` setting in config.toml, but codex only honours notify in user-global config — it is a restricted key in project-local layers — and the #179 rule forbids touching ~/.codex. codex's hooks system is the project-local equivalent: `.codex/hooks.json` (loaded in trusted projects) fires Stop when a turn completes, with a payload carrying the session id and transcript path. T4 ships that file through json_merge, so an existing user hooks.json is deep-merged into, never overwritten, and re-runs don't duplicate the hook. the handler is a new --hook mode on ingest-codex: read the stop payload from stdin, resolve the session's rollout (transcript_path when present, else by session id in the rollout filename), ingest idempotently, and exit 0 no matter what — the same never-break-the- host rule capture observe follows. stop fires per turn, not per session end, so the finalize policy the ticket left open is resolved as idempotent re-ingest: the dedup guard now refreshes a session's still-PENDING proposal in place when the rollout grew (same id, updated summary, audit-logged), reports an unchanged rollout as a no-op, and never resurrects a proposal a human already decided. storage gains update_proposal, a pure-I/O in-place rewrite of a pending proposal file. wiring only — everything still lands through propose_page; nothing touches approve(). closes #388 * test(adapters): live codex install gate against the real cli tests/test_openclaw_plugin_load_real.py set the pattern: exercise the real host cli when it's on PATH, skip cleanly where it isn't. the codex adapter had no equivalent, so regressions in the shipped config only surfaced when a user hit them — the old T1 silent-skip no-op is exactly the class of bug a live gate would have caught. the new suite installs the adapter into a temp project at T4, marks the project trusted inside an isolated CODEX_HOME, and asserts through `codex mcp list --json` that the vouch server is visible: next to a pre-seeded unrelated server on the merge path, alone on the fresh path, and absent for an untrusted project (the config must stay inert where codex says it does). a full-tier check covers the fenced AGENTS.md, the skills, and the Stop hook, and an isolation check pins every written artifact under the temp project. assertions target the observable contract — server listed, config parses, snippet present — not codex internals, so version bumps shouldn't break the suite. no network, no credentials, never touches the real ~/.codex; runs in the normal pytest invocation. closes #389 * docs(adapters): codex adapter docs reflect the tiered install three surfaces went stale once the codex tier work landed. the adapters table row described codex as a one-file config.toml snippet; it now lists the tiered install. the codex adapter readme led with the manual ~/.codex edit; it now leads with `vouch install-mcp codex`, explains the project-local .codex/config.toml choice and the trust requirement, summarizes what each tier adds (mcp wire / AGENTS.md snippet / skills / capture hook) with the merge-safety guarantees, and keeps the manual edit as an explicit fallback section. getting-started treated codex as a one-liner aside inside the claude instructions; the install section now shows both hosts side by side. the prompts-location decision from the T3 ticket and the notify restriction from the T4 ticket stay documented in their sections. no code; placeholder examples only. closes #390 * docs(mcp): design a friendlier mcp surface — profiles + auto-recall the closest competitor (pmb) exposes 10 mcp tools by default and hides the rest behind a profile flag; vouch shows all 58 every turn, which is the main first-touch friendliness gap. this spec adds a minimal-by-default tool profile, fusion-by-default retrieval, a per-prompt auto-recall hook, and compact tool descriptions. all of it is presentation-only — the review gate, the yaml storage, and the protocol method surface are untouched. it also closes the current 2-of-3 surface-parity gap as a bonus. * docs(mcp): plan the friendlier-mcp-surface build six TDD tasks: tool profiles (minimal default), real mcp↔methods parity, fusion-by-default retrieval, near-duplicate drop, per-prompt auto-recall hook, and compact tool descriptions. reconcile the spec: minimal is 8 tools (kb_capabilities kept as the discovery escape hatch) and the recency multiplier is deferred (per-hit timestamps aren't plumbed through _retrieve). * feat(mcp): add tool profiles, expose a minimal surface by default agents saw all 58 kb.* tools every turn — the main first-touch friendliness gap vs pmb, which exposes ~10 by default. add a profile layer applied in run_stdio: minimal (8 core tools) by default, standard (16), or full (58), selected by VOUCH_TOOL_PROFILE / config mcp.tool_profile. exposure only — the protocol surface, jsonl/cli, and the review gate are unchanged. * fix(mcp): harden profile resolution against a non-dict config section a bare `mcp:` key in config.yaml parses to None in YAML, and a non-dict mcp section is also possible from hand-edited configs. the previous config.get("mcp", {}).get("tool_profile") chain only supplied the {} default when the key was absent, so a present-but-None or non-dict value passed through and the chained .get raised AttributeError. run_stdio calls resolve_profile_name outside its try/except, so this would crash vouch serve instead of degrading to the minimal default. guard each nesting level with isinstance, matching the established reflex_cfg pattern in salience.py. * test(capabilities): assert mcp tool set matches the method list the parity test only compared capabilities.METHODS to the jsonl handlers; the 58 mcp tools were never checked, so mcp drift passed ci. enumerate the unfiltered server tools and assert they equal METHODS — the real 3-surface check for the mcp side. * feat(retrieval): fuse embedding + fts5 by default instead of a waterfall _retrieve tried embedding, then fts5, then substring, returning the first non-empty list — so lexical and semantic hits never combined. auto and hybrid now fuse both retrievers with the already-built rrf_fuse and tag hits "hybrid"; explicit embedding/fts5/substring pins are unchanged. existing KBs (config says "auto") benefit with no migration. * fix(volunteer): treat hybrid relevance as rank-relative, not pre-normalized normalize_relevance() grouped "hybrid" with "embedding", returning the raw score unclamped-by-batch on the assumption it was already a 0-1 similarity. now that _retrieve's auto/hybrid path tags fused hits "hybrid" with a rrf_fuse score (bounded by ~2/(k+rank), typically 0.01-0.03), that assumption stopped holding: every fused relevance landed far below DEFAULT_THRESHOLD (0.85), so kb.volunteer_context stopped firing for any KB using the new default backend. batch-normalize hybrid like fts5 and substring instead — restores the confidence-gated push channel (#236). * feat(retrieval): drop near-duplicate items from the context pack a fused pack could surface the same fact from two claims. add a cheap greedy jaccard pass (>=0.85 over the first 40 tokens) that keeps the highest-scored of a near-duplicate cluster, so an agent never reads the same thing twice. * fix(retrieval): dedupe by score so the highest-scored near-dup wins _dedupe_near_duplicates assumed its input arrived in descending-score order, which only holds for the plain retrieval path. when build_context_pack runs with expand_graph=True, graph neighbours are appended in bfs order with decayed scores after the sorted main items, so a higher-scored neighbour could be dropped in favor of an earlier, lower-scored main item. sort inside the function so the invariant holds regardless of caller order. * feat(hooks): inject per-prompt kb context via a userpromptsubmit hook recall used to be either a session-start firehose or an explicit tool call. add a pure, never-raising helper + a hidden `vouch context-hook` command that reads the host's prompt payload on stdin and prints an additionalContext envelope, and wire it into the claude-code adapter's UserPromptSubmit hook — so relevant, approved knowledge is injected every turn with zero tool calls. * fix(hooks): never raise on non-dict payload or missing kb a reviewer reproduced two crash paths that violated task 5's own contract that the prompt hook must never block a turn: build_claude_prompt_hook raised AttributeError on non-dict JSON (null/number/bool/array/string all decode fine but lack .get), and the context-hook CLI command called _load_store(), whose sys.exit(2) on a missing KB is a SystemExit that slips past `except Exception` and exits the process nonzero. guard the payload with an isinstance check before .get, log a breadcrumb when build_context_pack fails so a broken KB isn't silently indistinguishable from "no hits", and swap _load_store() for the existing non-exiting _capture_store() helper so the command has no sys.exit on its path at all. * feat(mcp): serve one-line tool descriptions under non-full profiles full docstrings for every exposed tool are paid on every turn. under minimal and standard, trim each tool's description to its first line (full keeps the complete docstrings), cutting the per-turn context cost. * docs(mcp): document tool profiles note VOUCH_TOOL_PROFILE / mcp.tool_profile (default minimal, values minimal|standard|full) in the transports reference and the claude-code adapter guide; this repo has no mintlify/ tree yet so both live under docs/ and adapters/claude-code/ instead. * fix(retrieval): dedupe by score but keep caller order for the pack _dedupe_near_duplicates sorted by score and returned that score-sorted order, which fought graph-expansion: build_context_pack appends decayed-score neighbours after the ranked hits, and fused primary hits now carry small rrf scores. re-sorting let an irrelevant depth-2 neighbour outrank a real match, and the max_chars tail-pop then evicted the real match instead of the neighbour. keep the keep-decision in descending-score order so the highest-scored member of a near-duplicate cluster still survives, but return survivors in the caller's original order so budget eviction drops the tail (appended neighbours), not the ranked hits. * docs(changelog): note minimal mcp profile, fusion, auto-recall summarize this branch's user-visible changes under [Unreleased]: the minimal-by-default mcp tool profile, rrf fusion for auto/hybrid retrieval, and the per-prompt auto-recall hook in the claude-code adapter. * docs(readme): note the fourth hook, per-prompt recall, and lean mcp profile install-mcp now writes a fourth hook (UserPromptSubmit -> vouch context-hook), and recall fires per prompt, not only at session start. also note the kb.* mcp surface defaults to a lean profile that widens via VOUCH_TOOL_PROFILE. * fix(capabilities): read openclaw.compat from package.json (#417) the host-compat drift check (#237) read openclaw.compat.pluginApi from openclaw.plugin.json. that block moved to package.json when the plugin packaging was split, and openclaw.* is now banned from the manifest (enforced by test_manifest_carries_no_dead_dialect_fields). the #237 reader was left pointing at the old, now-empty location, so host_compat silently degraded to {} and both host-compat assertions failed on the test branch — which every open pr targeting test inherited, including #413. repoint _load_host_compat and the test helpers at package.json. the openclaw.compat.pluginApi key path is identical in both files, so this is purely a source-file change with no behaviour difference. * chore(repo): untrack owner-local web/ and .claude/ (#413) these two directories were committed by accident and reach every first-time contributor on clone. neither is project source: - web/ is the netlify-deployed marketing landing page. it is shipped via the netlify cli and is not meant to live in the repo tree. - .claude/ is a personal claude code config: a settings.json with owner-specific hooks, plus command files that are byte-identical duplicates of adapters/claude-code/.claude/commands/. untrack both (files stay on disk) and gitignore them, anchored to the repo root so src/vouch/web and adapters/claude-code/.claude stay tracked. also ignore coverage.xml and the .netlify/ and .playwright-mcp/ tool-scratch dirs. * chore(merge): resolve changelog and version conflicts with main * chore(merge): update changelog to resolve merge conflicts * chore(merge): update to main's latest versions for clean merge --------- Co-authored-by: plind-junior <138900956+dripsmvcp@users.noreply.github.com> Co-authored-by: alpurkan17 <alpurkan9@gmail.com> Co-authored-by: Tet-9 <igeparallax@gmail.com> Co-authored-by: Yaroslav98214 <diakovichyaroslav30@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: minion1227 <romantymkiv1999@gmail.com> Co-authored-by: RealDiligent <brave.challenge007@gmail.com> Co-authored-by: jsdevninja <topit89807@gmail.com>
plind-junior
added a commit
that referenced
this pull request
Jul 7, 2026
* fix(server): block path traversal in register_source_from_path kb.register_source_from_path read the contents of any file the process could access, letting an agent register /etc/passwd, ~/.ssh/id_rsa, ~/.aws/credentials etc. as a "source" and then retrieve the bytes via kb.cite or kb.list_sources. Resolve the path (following symlinks) and require it to be inside the KB root before reading. Adds KBStore.resolve_under_root() so both the JSONL handler and the MCP tool share one containment check. Fixes #10 * fix(cli): translate domain errors into clean ClickException output The CLI handlers for approve and reject caught only (ArtifactNotFoundError, ValueError) but proposals.approve() / proposals.reject() raise ProposalError -- a RuntimeError subclass. The four propose-* shortcuts caught nothing at all. Result: a double- approve, an empty rejection reason, an empty claim text, or an unknown source id surfaced a raw Python traceback instead of the one-line `Error: ...` the rest of the CLI emits. Add a small `_cli_errors` context manager that translates ArtifactNotFoundError / ValueError / ProposalError / LifecycleError into click.ClickException, and apply it to every command that calls into proposals, lifecycle, sessions, or storage. The MCP and JSONL servers already do the equivalent in their own envelopes; this brings the human-facing surface in line. Adds tests/test_cli.py with regressions for approve, reject, propose- claim, propose-entity, and show. * chore(lint): chain FileExistsError cause in storage put_ methods ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged. * fix(verify): catch ArtifactNotFoundError and OSError on stored read verify_source() caught FileNotFoundError, but store.read_source_content() raises ArtifactNotFoundError (a KeyError subclass) when the content blob is missing -- so the "stored content missing" graceful path never ran and a single broken source crashed the entire verify_all() sweep, breaking `vouch source verify` and `vouch doctor`. The same call can also raise OSError (permission denied, TOCTOU race between exists() and read_bytes(), underlying I/O error) which was likewise unhandled. Catch both: the existing "missing" path keeps its note, and OSError surfaces as "stored content unreadable: <reason>". Adds three regression tests: - missing-content blob -> graceful per-source failure - unreadable stored content (monkeypatched PermissionError) -> graceful per-source failure with the underlying reason in the note - mixed sweep with one good + one broken source -> verify_all returns both results instead of aborting at the first failure Fixes #30 * chore(ci): unblock lint/type/test gates so the CLI fix can land The CI workflow runs `ruff check`, `mypy`, then `pytest`, and the fix/cli-clean-domain-errors branch was failing all three on pre-existing main-branch issues unrelated to the CLI change itself. * ruff: add `from e` to the seven `raise ValueError(...) from FileExistsError` re-raises added by the recent exclusive-create guards in storage.put_* (B904), and let ruff re-sort the cli.py import block (I001). * mypy: narrow the `kind` value flowing into `ContextItem(type=...)` with a `Literal` cast so the strict-typed field accepts what the search backends actually return. * pytest: `session_end()` mutates a session that `session_start()` has already written, but `put_session()` now uses exclusive create and rejects the second write. Add `KBStore.update_session()` mirroring `update_claim` and have `session_end` call it; existing test_sessions coverage now passes again. No behavior change for the CLI surface — these are infrastructure fixes so the existing test_sessions / lint / type assertions pass on this branch. * chore(ci): unblock mypy + test gates for path-traversal PR The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself. * fix(server): close TOCTOU window between path check and read CodeRabbit on PR #28 noted that resolve_under_root() only validated a pathname snapshot. Callers then re-opened the same name with .is_file() and .read_bytes(), so an attacker who can swap the resolved path for a symlink between the containment check and the read can still exfiltrate an out-of-root file via kb.register_source_from_path. Collapse the validate-then-read into a single trusted helper `KBStore.read_under_root(path)` that returns `(resolved, bytes)`: 1. Path.resolve() chases pre-existing symlinks and the resulting target is checked for containment (existing behaviour -- legitimate in-root symlinks still work). 2. The read goes through `os.open(resolved, O_RDONLY | O_NOFOLLOW)` so a fresh symlink placed at the resolved name *after* the check fails with ELOOP rather than following the swap. 3. `fstat` + `S_ISREG` rejects directories / device nodes / pipes atomically on the same fd, replacing the racy `is_file()` test the callers used to do. Both register_source_from_path handlers (MCP + JSONL) switch to the new helper and drop their now-redundant follow-up checks. Adds tests: - symlink swapped into the resolved name -> rejected via ELOOP - directory at a valid path -> rejected via S_ISREG Existing "outside the root" and "inside the root" tests still pass. * chore(lint): chain FileExistsError cause in storage put_ methods ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged. * chore(ci): unblock mypy + test gates for path-traversal PR The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself. * docs: add banner diagram to README * docs(spec): semantic search as primary retrieval backend Approved design for feat/semantic-search. Embedding (sentence- transformers all-mpnet-base-v2) becomes the primary search backend with FTS5 as fallback. Synchronous-at-write indexing across all six artifact types (claim, page, source, entity, relation, evidence). Maximally functional scope (~3000 LOC): pluggable model adapter registry, sqlite-vec ANN with NumPy fallback, cross-encoder rerank, HyDE query expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG eval harness, and full CLI/MCP/JSONL parity. The writing-plans step will turn the rollout order in section 16 into concrete phased tasks. * docs(plan): semantic search implementation plan 32-task TDD plan for the semantic-search feature: foundation, storage, write hooks across all six artifact types, semantic-primary search integration in MCP/JSONL/CLI, RRF fusion + hybrid, cross-encoder rerank, HyDE expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG scorer, end-to-end integration test, and user docs. Each task: failing test, minimal implementation, passing test, commit. MockEmbedder test double keeps the unit suite fast; the real model is exercised only under @pytest.mark.integration. The evaluation module is named scorer.py rather than eval.py to avoid shadowing the Python builtin and to keep static analysers quiet; the user-facing CLI subcommand remains `vouch eval embedding` (the Click group is registered under the name "eval", with the Python identifier eval_group). * feat(embeddings): add optional-deps extras and pytest markers * feat(embeddings): create package skeleton * feat(embeddings): Embedder ABC, registry, content_hash, MockEmbedder * feat(embeddings): sentence-transformers all-mpnet-base-v2 default adapter * feat(embeddings): sentence-transformers MiniLM-L6 alternative adapter * feat(embeddings): fastembed BGE alternative (no-torch) adapter * fix(ci): satisfy ruff SIM105 + add mypy overrides for optional deps CI ran ruff which flagged SIM105 on the three try/except ImportError guard blocks in src/vouch/embeddings/__init__.py. Replace each with `contextlib.suppress(ImportError)` -- same semantics, satisfies ruff. Also add mypy overrides for numpy / sqlite_vec / sentence_transformers / fastembed so the type check passes in the base [dev] CI install where those optional extras aren't present. The embedding code paths that import these libraries are only reached when the extras are installed at runtime; for the static type check the missing stubs are noise. * fix(ci): skip embeddings test suite when numpy isn't installed CI runs `pip install -e '.[dev]'` which deliberately excludes the optional `[embeddings]` extras. tests/embeddings/test_*.py modules import numpy at top level, so pytest collection fails with ModuleNotFoundError before any test can be deselected. Add tests/embeddings/conftest.py with `pytest.importorskip("numpy")` so the entire embeddings test directory skips gracefully when numpy is absent. Once `pip install vouch[embeddings]` (or numpy itself) is present, the skip is a no-op and the tests run normally. * fix(bundle): skip Pydantic validation for opaque source content files `_validate_content` in bundle.py uses the path's first directory component to look up a Pydantic validator. For sources, both `sources/<sha>/meta.yaml` (the Source model) and `sources/<sha>/content` (raw opaque bytes) hit the same "sources" key, so the validator was being run on the raw content bytes and failing with "1 validation error for Source". Add an early return for any non-meta.yaml path under `sources/` so opaque content bytes are not Pydantic-validated. Only the Source metadata file is checked, which matches the original intent of the PR #13 validation. Unblocks three pre-existing test_bundle.py failures inherited from upstream main. * fix(embeddings): MockEmbedder uses uint32 scaling to avoid NaN/Inf Per CodeRabbit on PR #37: decoding sha256 chunks as raw float32 (via struct.unpack("<f", ...)) produced NaN/Inf for ~1 in 2^7 chunks because most 4-byte patterns aren't finite IEEE754 floats. Norm overflow then made unit normalization unreliable and broke cosine similarity asserts in downstream phases (the dedup test in Phase 6 already had to work around this with a custom _HashEmbedder). Decode each chunk as an unsigned 32-bit integer, scale to [-1.0, 1.0], accumulate in float64 for exact norm, and cast to float32 on return. Deterministic, finite, well-behaved. * feat(embeddings): state.db schema for embedding storage + put/get helpers * feat(embeddings): NumPy brute-force cosine search over embedding_index * feat(embeddings): sqlite-vec ANN path with NumPy fallback * feat(embeddings): query embedding LRU cache * style: fix ruff E402/SIM105/E702/I001 violations from Phase 2 storage tasks * fix(embeddings): use outer-query for WHERE on cosine alias in search_embedding Per CodeRabbit Critical on PR #39/40/43: SQLite does not allow a column alias defined in the SELECT list to be referenced in the WHERE clause of the same query. The old form SELECT ... 1.0 - vec_distance_cosine(vec, ?) AS score ... WHERE ... AND score >= ? raised `no such column: score` at runtime; the catch-all `except sqlite3.OperationalError` silently swallowed it and made every call fall through to the NumPy brute-force path, which works but defeats the whole point of installing sqlite-vec. Wrap the projection in a subquery so the WHERE clause sees the alias. No behavior change for callers that have already been using the fallback (NumPy still produces correct results); the ANN code path now actually runs when sqlite-vec is loaded. * feat(embeddings): write-time embedding hook + wire put_claim * feat(embeddings): search_semantic wrapper with query cache * feat(embeddings): hook all six artifact write paths * feat(embeddings): kb_search defaults to embedding primary, fts5 fallback * feat(embeddings): re-embed on update_claim * feat(embeddings): JSONL _h_search parity with MCP semantic-primary dispatch * style: fix ruff SIM105/E501/I001/RUF059 violations from Phase 3 * fix(ci): silence mypy import-untyped for forward-ref fusion import The hybrid backend branch in kb_search (server.py) and _h_search (jsonl_server.py) lazily imports vouch.embeddings.fusion (added in Phase 5). On Phase 4's branch that module doesn't exist yet, so mypy emits import-untyped. Mark each import with `# type: ignore` and let ruff auto-format the line. * fix(ci): silence mypy import-untyped for forward-ref dedup import The KBStore._embed_and_store helper lazily imports vouch.embeddings.dedup (added in Phase 6). On Phase 3's branch that module does not yet exist, so mypy treats it as untyped and errors out. Mark the import with `# type: ignore[import-not-found,import-untyped, unused-ignore]` so the type check passes here and stays silent once Phase 6 lands. * spec: cut dated 2026-05-21 snapshot; promote schema generator to script * ci(mypy): collapse fusion import to single line for type-ignore * feat(embeddings): RRF/weighted/normalized fusion strategies * docs: add 'What ships today' table to README * kb: approve review-gate fact * docs: add 'When to use vouch' section * fix(cli): honor VOUCH_AGENT in _whoami for consistent actor attribution The CLI propose-* commands and most actor sites called _whoami(), which only checked VOUCH_USER and the OS user — VOUCH_AGENT was silently ignored. The MCP and JSONL servers (server.py, jsonl_server.py) and session_start already honour VOUCH_AGENT, so multi-agent attribution broke as soon as you used the CLI to propose. Prefer VOUCH_AGENT over VOUCH_USER over the OS user so the recorded proposed_by / audit actor matches across all three transports. * docs: add captured example session walkthrough Real propose -> review -> commit -> retrieve loop captured from a sandbox run on 2026-05-21. Includes on-disk YAML, audit log lines, and an honest note about the literal substring backend limitation pre-embeddings. * docs: add screen recording of full vouch loop (VHS tape + GIF) * docs: embed demo GIF under Quick start in README * fix(embeddings): address CodeRabbit review on hybrid + fusion + write hook Bundles the substantive correctness fixes flagged by CodeRabbit on PR #41. CI was already green; these are latent bugs the review caught. storage._embed_and_store - Re-embed when the active embedder model changes, not only when the content hash changes. The previous short-circuit kept stale vectors on disk after a model swap, mixing document embeddings from one model with queries from another. - Truly best-effort: catch any exception during encode/put/meta/dedup so a hook failure can't bubble back to the caller and leave an artifact persisted-but-API-errored. index_db.reset - Also clears embedding_index, query_embedding_cache, embedding_dupes, and embedding_* keys from index_meta. Otherwise `vouch index` left orphaned hits behind after artifacts were removed. index_db.search_embedding (Python fallback) - Normalize the stored vector by its own L2 norm before scoring, so rankings match the sqlite-vec `1 - vec_distance_cosine` path. The previous raw dot product made magnitude leak into the score. index_db.search_semantic - Invalidate the query-vector cache entry when the embedder dim no longer matches what's on disk; otherwise a model swap returned stale vectors living in the wrong space. server.kb_search + jsonl_server._h_search (hybrid branch) - Hybrid now passes min_score into search_semantic (consistent with embedding/auto branches) and wraps the FTS lookup in try/except so an FTS5 error doesn't fail the whole request. - JSONL transport rejects unknown backend values with a clear error instead of silently returning []; matches MCP transport behavior. embeddings.fusion - rrf_fuse validates limit >= 0 and k >= 0 (k = -rank would divide by zero). weighted_fuse validates limit >= 0. Tests - Three new fusion guard tests cover the negative-limit and negative-k paths. * fix(lifecycle): only catch missing-artifact errors in cite() the bare except was treating every error as a missing citation, which hid real bugs. narrow it to ArtifactNotFoundError so unexpected failures actually surface. * fix(context): only catch missing-artifact errors in citation lookup same issue as cite() — a bare except was hiding real errors behind an empty citation list. narrow it to ArtifactNotFoundError. * refactor(sessions): keep tracebacks when crystallize() approve fails we were dropping the traceback and only stringifying the error, which made it painful to diagnose anything other than ProposalError. now logs via logger.exception and includes error_type in the failures list. * fix(context): only catch sqlite errors when FTS5 falls back the retrieval helper had a bare except that would also swallow bugs in the substring fallback path or in our own code. narrow it to sqlite3.Error so only the intended cases (FTS5 missing, db missing, schema mismatch) trigger the fallback. * refactor(health): use typed status filter for pending proposal count we were listing every proposal and comparing p.status.value == 'pending' as a string. list_proposals() already takes a ProposalStatus filter — use it. shorter, type-checked, and avoids the stringly-typed compare. * chore(capabilities): pin pluginApi compat baseline, fail CI on host-compat drift * feat(dual-solve): surface engine logs * fix(sessions): make crystallize retry idempotent for summary pages Partial crystallize runs that already wrote session-{id} summary pages failed on retry with "page already exists". Upsert the summary page and derive its artifact list from all approved session proposals. Fixes #139 Co-authored-by: Cursor <cursoragent@cursor.com> * fix(models): reject empty text/name/title on claim/entity/page the non-empty contract for `Claim.text`, `Entity.name`, and `Page.title` lived only in the `propose_*` helpers, so `store.put_*`, `store.update_*`, and bundle/sync import (via `_validate_content`) all silently accepted empty or whitespace-only values and landed artifacts carrying zero of the field's semantic content. add `@field_validator`s on each field, mirroring the existing `Claim.evidence` min-citation validator (#81/#82): they reject blank input at construction time, so every write path — direct construction, put/update, and import — inherits the check at once. the validators gate on emptiness only and preserve surrounding whitespace of non-blank values. `Source.locator` is deliberately left out of scope — its format question is bigger and deserves its own issue, as the report notes. closes #155 * refactor(models): extract shared _require_non_empty validator helper address review on #300: the three #155 field validators (Claim.text / Entity.name / Page.title) were structurally identical strip-checks. extract a module-level `_require_non_empty(v, label)` helper they all delegate to, keeping the per-field error messages. also fix a mid-entry sentence casing in the CHANGELOG. no behaviour change. * fix: resolve RPC traceback leakage on internal errors Unexpected exceptions in handle_request() included full stack traces in the JSON error envelope, exposing paths and internals to HTTP /rpc clients. Log server-side and return message only. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(review): kb.triage_pending — advisory triage scoring for the pending queue a long `kb.list_pending` forces the reviewer to reconstruct, per proposal, whether the claim fits the existing kb, whether its citations resolve, whether it duplicates something already filed, and whether it contradicts an approved claim. this adds an optional triage pass that scores each pending proposal on those four signals and attaches a `_meta.vouch_triage` block (recommendation/score/signals/rationale) to help a reviewer prioritize, without ever deciding anything itself. read-only by construction: the pass never calls proposals.approve/reject, store.put_*, or store.move_proposal_to_decided — a human still calls kb.approve/kb.reject. citation_quality reuses proposals._payload_block_reason; duplication_risk reuses the propose-time embedding similarity path (embeddings.similarity.find_similar_on_propose) and degrades to a difflib heuristic when the embeddings extra isn't installed. fit uses a separate, lower-threshold embedding search so a near-duplicate hit doesn't also inflate fit and cancel out its own duplication penalty. opt-in via `triage.enabled: true` in config.yaml (default false). registered at all four kb.* surface sites (server.py, jsonl_server.py, capabilities.py, cli.py) plus `vouch triage [proposal-id...]` with `--json` and `--reverse`. * feat(diff): register kb.diff at all four kb.* surface sites vouch diff shipped CLI-only in 0.1.0, an explicit non-goal at the time ("MCP/JSONL parity ... kb.* surface unchanged"). that leaves it the only read method skipping the four-site registration convention documented in CLAUDE.md, so agents talking MCP/JSONL have no way to fetch a revision diff. adds the kb_diff MCP tool, the kb.diff JSONL handler, and the capabilities.METHODS entry, next to the other by-id read tools (kb_read_claim/kb_read_page) with the same unrestricted-read posture. also makes new_id optional for a superseded claim: diff_artifacts and the CLI both resolve it from superseded_by when omitted, erroring clearly when there's no successor (pages still require an explicit new_id — they have no successor pointer). closes #327. * fix(jsonl): define module logger after the import block #361 inserted `_log = logging.getLogger(...)` between two import groups in jsonl_server.py. that statement-among-imports trips ruff's E402 on every import that follows it, so `ruff check src tests` — the lint gate in ci — now fails on the whole repo. move the logger definition below the imports (where storage.py already keeps its own module logger); no behaviour change. * feat(adapters): toml_merge install strategy for codex config.toml .codex/config.toml is codex's primary config file, so the plain-copy path silently skipped any project where codex was already configured and vouch never got wired. add a toml_merge entry flag mirroring json_merge: parse the existing destination with tomllib, deep-merge the template's tables into it (existing user values always win on conflict), and write back. flip the codex T1 entry to toml_merge. writing uses a minimal hand-rolled serializer (tables, arrays, inline tables in arrays, scalars, datetimes) so the dependency set stays unchanged; the output must survive a tomllib round-trip back to the merged data, and anything the serializer can't faithfully re-emit degrades to skipped rather than risking the user's config. closes #384 * feat(adapters): codex T2 agents.md fenced snippet codex reads AGENTS.md for project instructions the way cursor does, but the codex adapter stopped at T1, so a codex session got the kb tools with no standing guidance on recall-first or the review gate. ship adapters/codex/AGENTS.md.snippet as a T2 tier with the standard fence markers, kept in lockstep with cursor's snippet modulo the host name (enforced by a sync test). also close the edited-fence gap in _install_fenced per the ticket's acceptance criteria: a fence body that drifted from the shipped snippet is replaced within the markers (reported as merged) instead of being skipped forever, while user content outside the fence stays untouched. a begin marker without an end marker is treated as corrupt and left alone. closes #385 * feat(adapters): codex T3 skills mirroring the vouch slash commands claude-code T3 ships nine slash commands; codex users got none of the guided flows. ship them as codex skills under the project-local .codex/skills/ as a T3 tier. scope decision the ticket asked to resolve first: codex loads custom prompts only from ~/.codex/prompts/ (user-global) and has deprecated them upstream in favour of skills, which do have a project-local home at <project>/.codex/skills/ in trusted projects. the #179 rule forbids a project-scoped install from touching home-directory state, so prompts are out and skills are in — recorded in the manifest comment and the adapter readme, with a manual-copy pointer for users who still want ~/.codex/prompts. the SKILL.md files are referenced from the openclaw mirror rather than duplicated (same reuse pattern openclaw itself applies to the claude-code commands), so one edit updates every host. a parametrized sync test asserts each installed codex skill body equals the matching claude-code command body, and a scope test asserts every installed path stays under the project target. closes #386 * feat(capture): ingest codex session rollouts into review-gated summaries session auto-capture was claude-code-only: hooks drive capture observe and capture finalize live. codex has no hook stream, but it persists every session as a rollout jsonl under $CODEX_HOME/sessions containing user messages, tool calls, and outputs — everything the existing rollup needs, just after the fact. new cli command `vouch capture ingest-codex [<rollout> | --latest]` parses one rollout through a small dedicated parser (codex_rollout.py) that maps function_call records into the same observation shape capture.observe produces — shell commands with failure detection from exit codes, apply_patch heredocs surfaced as file edits, mcp tools under their own names, session mechanics skipped — then reuses the existing build_summary_body -> propose_page rollup. one code path from observation to proposal, two front doors. the rollout format is not a stable public contract: unknown record types are tolerated, and unreadable, compressed, or meta-less files degrade to a CodexRolloutError with an actionable message and a non-zero exit, never a stack trace. re-ingesting a session is a no-op keyed on the rollout's session id; --latest resolves the newest rollout whose recorded cwd matches the current project. proposals are attributed to the codex actor (VOUCH_AGENT wins when set), respect capture's enabled/min_observations config, and never touch approve(). fixture rollouts use placeholder data only, enforced by a test. closes #387 * feat(adapters): codex T4 hook wiring for automatic session capture with `vouch capture ingest-codex` available, codex can self-capture the way claude-code T4 does. the ticket assumed the `notify` setting in config.toml, but codex only honours notify in user-global config — it is a restricted key in project-local layers — and the #179 rule forbids touching ~/.codex. codex's hooks system is the project-local equivalent: `.codex/hooks.json` (loaded in trusted projects) fires Stop when a turn completes, with a payload carrying the session id and transcript path. T4 ships that file through json_merge, so an existing user hooks.json is deep-merged into, never overwritten, and re-runs don't duplicate the hook. the handler is a new --hook mode on ingest-codex: read the stop payload from stdin, resolve the session's rollout (transcript_path when present, else by session id in the rollout filename), ingest idempotently, and exit 0 no matter what — the same never-break-the- host rule capture observe follows. stop fires per turn, not per session end, so the finalize policy the ticket left open is resolved as idempotent re-ingest: the dedup guard now refreshes a session's still-PENDING proposal in place when the rollout grew (same id, updated summary, audit-logged), reports an unchanged rollout as a no-op, and never resurrects a proposal a human already decided. storage gains update_proposal, a pure-I/O in-place rewrite of a pending proposal file. wiring only — everything still lands through propose_page; nothing touches approve(). closes #388 * test(adapters): live codex install gate against the real cli tests/test_openclaw_plugin_load_real.py set the pattern: exercise the real host cli when it's on PATH, skip cleanly where it isn't. the codex adapter had no equivalent, so regressions in the shipped config only surfaced when a user hit them — the old T1 silent-skip no-op is exactly the class of bug a live gate would have caught. the new suite installs the adapter into a temp project at T4, marks the project trusted inside an isolated CODEX_HOME, and asserts through `codex mcp list --json` that the vouch server is visible: next to a pre-seeded unrelated server on the merge path, alone on the fresh path, and absent for an untrusted project (the config must stay inert where codex says it does). a full-tier check covers the fenced AGENTS.md, the skills, and the Stop hook, and an isolation check pins every written artifact under the temp project. assertions target the observable contract — server listed, config parses, snippet present — not codex internals, so version bumps shouldn't break the suite. no network, no credentials, never touches the real ~/.codex; runs in the normal pytest invocation. closes #389 * docs(adapters): codex adapter docs reflect the tiered install three surfaces went stale once the codex tier work landed. the adapters table row described codex as a one-file config.toml snippet; it now lists the tiered install. the codex adapter readme led with the manual ~/.codex edit; it now leads with `vouch install-mcp codex`, explains the project-local .codex/config.toml choice and the trust requirement, summarizes what each tier adds (mcp wire / AGENTS.md snippet / skills / capture hook) with the merge-safety guarantees, and keeps the manual edit as an explicit fallback section. getting-started treated codex as a one-liner aside inside the claude instructions; the install section now shows both hosts side by side. the prompts-location decision from the T3 ticket and the notify restriction from the T4 ticket stay documented in their sections. no code; placeholder examples only. closes #390 * docs(mcp): design a friendlier mcp surface — profiles + auto-recall the closest competitor (pmb) exposes 10 mcp tools by default and hides the rest behind a profile flag; vouch shows all 58 every turn, which is the main first-touch friendliness gap. this spec adds a minimal-by-default tool profile, fusion-by-default retrieval, a per-prompt auto-recall hook, and compact tool descriptions. all of it is presentation-only — the review gate, the yaml storage, and the protocol method surface are untouched. it also closes the current 2-of-3 surface-parity gap as a bonus. * docs(mcp): plan the friendlier-mcp-surface build six TDD tasks: tool profiles (minimal default), real mcp↔methods parity, fusion-by-default retrieval, near-duplicate drop, per-prompt auto-recall hook, and compact tool descriptions. reconcile the spec: minimal is 8 tools (kb_capabilities kept as the discovery escape hatch) and the recency multiplier is deferred (per-hit timestamps aren't plumbed through _retrieve). * feat(mcp): add tool profiles, expose a minimal surface by default agents saw all 58 kb.* tools every turn — the main first-touch friendliness gap vs pmb, which exposes ~10 by default. add a profile layer applied in run_stdio: minimal (8 core tools) by default, standard (16), or full (58), selected by VOUCH_TOOL_PROFILE / config mcp.tool_profile. exposure only — the protocol surface, jsonl/cli, and the review gate are unchanged. * fix(mcp): harden profile resolution against a non-dict config section a bare `mcp:` key in config.yaml parses to None in YAML, and a non-dict mcp section is also possible from hand-edited configs. the previous config.get("mcp", {}).get("tool_profile") chain only supplied the {} default when the key was absent, so a present-but-None or non-dict value passed through and the chained .get raised AttributeError. run_stdio calls resolve_profile_name outside its try/except, so this would crash vouch serve instead of degrading to the minimal default. guard each nesting level with isinstance, matching the established reflex_cfg pattern in salience.py. * test(capabilities): assert mcp tool set matches the method list the parity test only compared capabilities.METHODS to the jsonl handlers; the 58 mcp tools were never checked, so mcp drift passed ci. enumerate the unfiltered server tools and assert they equal METHODS — the real 3-surface check for the mcp side. * feat(retrieval): fuse embedding + fts5 by default instead of a waterfall _retrieve tried embedding, then fts5, then substring, returning the first non-empty list — so lexical and semantic hits never combined. auto and hybrid now fuse both retrievers with the already-built rrf_fuse and tag hits "hybrid"; explicit embedding/fts5/substring pins are unchanged. existing KBs (config says "auto") benefit with no migration. * fix(volunteer): treat hybrid relevance as rank-relative, not pre-normalized normalize_relevance() grouped "hybrid" with "embedding", returning the raw score unclamped-by-batch on the assumption it was already a 0-1 similarity. now that _retrieve's auto/hybrid path tags fused hits "hybrid" with a rrf_fuse score (bounded by ~2/(k+rank), typically 0.01-0.03), that assumption stopped holding: every fused relevance landed far below DEFAULT_THRESHOLD (0.85), so kb.volunteer_context stopped firing for any KB using the new default backend. batch-normalize hybrid like fts5 and substring instead — restores the confidence-gated push channel (#236). * feat(retrieval): drop near-duplicate items from the context pack a fused pack could surface the same fact from two claims. add a cheap greedy jaccard pass (>=0.85 over the first 40 tokens) that keeps the highest-scored of a near-duplicate cluster, so an agent never reads the same thing twice. * fix(retrieval): dedupe by score so the highest-scored near-dup wins _dedupe_near_duplicates assumed its input arrived in descending-score order, which only holds for the plain retrieval path. when build_context_pack runs with expand_graph=True, graph neighbours are appended in bfs order with decayed scores after the sorted main items, so a higher-scored neighbour could be dropped in favor of an earlier, lower-scored main item. sort inside the function so the invariant holds regardless of caller order. * feat(hooks): inject per-prompt kb context via a userpromptsubmit hook recall used to be either a session-start firehose or an explicit tool call. add a pure, never-raising helper + a hidden `vouch context-hook` command that reads the host's prompt payload on stdin and prints an additionalContext envelope, and wire it into the claude-code adapter's UserPromptSubmit hook — so relevant, approved knowledge is injected every turn with zero tool calls. * fix(hooks): never raise on non-dict payload or missing kb a reviewer reproduced two crash paths that violated task 5's own contract that the prompt hook must never block a turn: build_claude_prompt_hook raised AttributeError on non-dict JSON (null/number/bool/array/string all decode fine but lack .get), and the context-hook CLI command called _load_store(), whose sys.exit(2) on a missing KB is a SystemExit that slips past `except Exception` and exits the process nonzero. guard the payload with an isinstance check before .get, log a breadcrumb when build_context_pack fails so a broken KB isn't silently indistinguishable from "no hits", and swap _load_store() for the existing non-exiting _capture_store() helper so the command has no sys.exit on its path at all. * feat(mcp): serve one-line tool descriptions under non-full profiles full docstrings for every exposed tool are paid on every turn. under minimal and standard, trim each tool's description to its first line (full keeps the complete docstrings), cutting the per-turn context cost. * docs(mcp): document tool profiles note VOUCH_TOOL_PROFILE / mcp.tool_profile (default minimal, values minimal|standard|full) in the transports reference and the claude-code adapter guide; this repo has no mintlify/ tree yet so both live under docs/ and adapters/claude-code/ instead. * fix(retrieval): dedupe by score but keep caller order for the pack _dedupe_near_duplicates sorted by score and returned that score-sorted order, which fought graph-expansion: build_context_pack appends decayed-score neighbours after the ranked hits, and fused primary hits now carry small rrf scores. re-sorting let an irrelevant depth-2 neighbour outrank a real match, and the max_chars tail-pop then evicted the real match instead of the neighbour. keep the keep-decision in descending-score order so the highest-scored member of a near-duplicate cluster still survives, but return survivors in the caller's original order so budget eviction drops the tail (appended neighbours), not the ranked hits. * docs(changelog): note minimal mcp profile, fusion, auto-recall summarize this branch's user-visible changes under [Unreleased]: the minimal-by-default mcp tool profile, rrf fusion for auto/hybrid retrieval, and the per-prompt auto-recall hook in the claude-code adapter. * docs(readme): note the fourth hook, per-prompt recall, and lean mcp profile install-mcp now writes a fourth hook (UserPromptSubmit -> vouch context-hook), and recall fires per prompt, not only at session start. also note the kb.* mcp surface defaults to a lean profile that widens via VOUCH_TOOL_PROFILE. * fix(capabilities): read openclaw.compat from package.json (#417) the host-compat drift check (#237) read openclaw.compat.pluginApi from openclaw.plugin.json. that block moved to package.json when the plugin packaging was split, and openclaw.* is now banned from the manifest (enforced by test_manifest_carries_no_dead_dialect_fields). the #237 reader was left pointing at the old, now-empty location, so host_compat silently degraded to {} and both host-compat assertions failed on the test branch — which every open pr targeting test inherited, including #413. repoint _load_host_compat and the test helpers at package.json. the openclaw.compat.pluginApi key path is identical in both files, so this is purely a source-file change with no behaviour difference. * chore(repo): untrack owner-local web/ and .claude/ (#413) these two directories were committed by accident and reach every first-time contributor on clone. neither is project source: - web/ is the netlify-deployed marketing landing page. it is shipped via the netlify cli and is not meant to live in the repo tree. - .claude/ is a personal claude code config: a settings.json with owner-specific hooks, plus command files that are byte-identical duplicates of adapters/claude-code/.claude/commands/. untrack both (files stay on disk) and gitignore them, anchored to the repo root so src/vouch/web and adapters/claude-code/.claude stay tracked. also ignore coverage.xml and the .netlify/ and .playwright-mcp/ tool-scratch dirs. * chore(merge): resolve changelog and version conflicts with main * chore(merge): update changelog to resolve merge conflicts * chore(merge): update to main's latest versions for clean merge * docs(readme): restore incubated by gittensor acknowledgment * chore: remove banner.svg and its reference in README * chore: remove desktop, spec, migrations dirs and pre-commit config * fix(ci): restore missing storage and context constants Add back KB_FORMAT_VERSION, SCHEMA_VERSION, SCHEMA_VERSION_FILENAME, _starter_config to storage.py and _RETRACTED_CLAIM_STATUSES to context.py to fix import errors in test suite. * fix(ci): restore coherent backing modules to match callers the test-branch merges (2ddd1f9 onward) took main's older, smaller storage.py / context.py / index_db.py / cli.py / server.py / health.py / jsonl_server.py / bundle.py / sessions.py while keeping the newer caller modules (lifecycle, proposals, graph, notify, sync, provenance, codex_rollout, triage). that desync left 31 mypy errors — callers referencing methods the truncated backing no longer defined (put_relation_idempotent, _validate_claim_refs, update_page, update_proposal, index_prov_edge, get_meta) — so the test job failed at mypy before pytest ever ran. restore src/ and tests/ to bb06565, the last coherent snapshot, where the fuller backing matches the callers. this also brings back the reindex cli command the recall-eval job invokes (the truncated cli only had index), fixing the second red job. keep __version__ at 1.2.2 so the four version sites stay in lockstep. --------- Co-authored-by: plind-junior <138900956+dripsmvcp@users.noreply.github.com> Co-authored-by: alpurkan17 <alpurkan9@gmail.com> Co-authored-by: Tet-9 <igeparallax@gmail.com> Co-authored-by: Yaroslav98214 <diakovichyaroslav30@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: minion1227 <romantymkiv1999@gmail.com> Co-authored-by: RealDiligent <brave.challenge007@gmail.com> Co-authored-by: jsdevninja <topit89807@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #10.
kb.register_source_from_path(both MCP and JSONL transports) used to read any file the validator process could open and stored it as a "source". An agent could name/etc/passwd,~/.ssh/id_rsa,~/.aws/credentialsetc. and then exfiltrate the bytes viakb.citeorkb.list_sources. The fix forces both transports through a single containment check rooted at the project root.Root Cause
Both entrypoints joined the caller-supplied path verbatim and read it:
kb_register_source_from_path_h_register_source_from_pathNeither resolved symlinks before checking containment, and neither imposed a containment check at all — the only validation was
path.is_file().Fix
Adds
KBStore.resolve_under_root(path)insrc/vouch/storage.py. It callsPath(path).resolve()(following symlinks) and then enforcesis_relative_to(self.root). Both transports now route through it before reading. Symlink resolution happens before the containment check, so a symlink that lives inside the project but points outside is rejected.The
locatorstored on theSourcerecord is now the resolved path inside the project root (was the user-supplied string before), so the record is honest about what bytes it represents.Lint cleanup (
2a439a5)Inherited from main: ruff's
B904rule fires on 7 pre-existingraise ValueError(...)statements insideexcept FileExistsError:blocks instorage.py(put_claim,put_page,put_entity,put_relation,put_evidence,put_session,put_proposal). All recent merges to main fail CI for the same reason. Added the missingfrom eclauses so this PR's CI can go green. No behavior change — the chained exception now carries__cause__but the public message and type are unchanged.Test Plan
tests/test_jsonl_server.py::test_register_source_from_path_rejects_outside_roottmp_path(outside project root) and asserts the JSONL response is{ok: false, error.code: "invalid_request"}with"project root"in the message, and thatstore.list_sources()stays empty.ruff check src tests— passes locallypytest -q(excluding the pre-existing flakytests/test_sessions.pytimestamp-collision tests) — 55 passedSummary by CodeRabbit