chore(release): prepare 1.3.0#473
Merged
Merged
Conversation
fix: track backend label separately in JSONL search handler (#14)
fix: add existence guards to all put_* methods (#12)
…act-guard fix(proposals): refuse to overwrite existing artifact on approve
kb.register_source_from_path read the contents of any file the process could access, letting an agent register /etc/passwd, ~/.ssh/id_rsa, ~/.aws/credentials etc. as a "source" and then retrieve the bytes via kb.cite or kb.list_sources. Resolve the path (following symlinks) and require it to be inside the KB root before reading. Adds KBStore.resolve_under_root() so both the JSONL handler and the MCP tool share one containment check. Fixes #10
The CLI handlers for approve and reject caught only (ArtifactNotFoundError, ValueError) but proposals.approve() / proposals.reject() raise ProposalError -- a RuntimeError subclass. The four propose-* shortcuts caught nothing at all. Result: a double- approve, an empty rejection reason, an empty claim text, or an unknown source id surfaced a raw Python traceback instead of the one-line `Error: ...` the rest of the CLI emits. Add a small `_cli_errors` context manager that translates ArtifactNotFoundError / ValueError / ProposalError / LifecycleError into click.ClickException, and apply it to every command that calls into proposals, lifecycle, sessions, or storage. The MCP and JSONL servers already do the equivalent in their own envelopes; this brings the human-facing surface in line. Adds tests/test_cli.py with regressions for approve, reject, propose- claim, propose-entity, and show.
ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged.
verify_source() caught FileNotFoundError, but store.read_source_content() raises ArtifactNotFoundError (a KeyError subclass) when the content blob is missing -- so the "stored content missing" graceful path never ran and a single broken source crashed the entire verify_all() sweep, breaking `vouch source verify` and `vouch doctor`. The same call can also raise OSError (permission denied, TOCTOU race between exists() and read_bytes(), underlying I/O error) which was likewise unhandled. Catch both: the existing "missing" path keeps its note, and OSError surfaces as "stored content unreadable: <reason>". Adds three regression tests: - missing-content blob -> graceful per-source failure - unreadable stored content (monkeypatched PermissionError) -> graceful per-source failure with the underlying reason in the note - mixed sweep with one good + one broken source -> verify_all returns both results instead of aborting at the first failure Fixes #30
The CI workflow runs `ruff check`, `mypy`, then `pytest`, and the fix/cli-clean-domain-errors branch was failing all three on pre-existing main-branch issues unrelated to the CLI change itself. * ruff: add `from e` to the seven `raise ValueError(...) from FileExistsError` re-raises added by the recent exclusive-create guards in storage.put_* (B904), and let ruff re-sort the cli.py import block (I001). * mypy: narrow the `kind` value flowing into `ContextItem(type=...)` with a `Literal` cast so the strict-typed field accepts what the search backends actually return. * pytest: `session_end()` mutates a session that `session_start()` has already written, but `put_session()` now uses exclusive create and rejects the second write. Add `KBStore.update_session()` mirroring `update_claim` and have `session_end` call it; existing test_sessions coverage now passes again. No behavior change for the CLI surface — these are infrastructure fixes so the existing test_sessions / lint / type assertions pass on this branch.
The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself.
CodeRabbit on PR #28 noted that resolve_under_root() only validated a pathname snapshot. Callers then re-opened the same name with .is_file() and .read_bytes(), so an attacker who can swap the resolved path for a symlink between the containment check and the read can still exfiltrate an out-of-root file via kb.register_source_from_path. Collapse the validate-then-read into a single trusted helper `KBStore.read_under_root(path)` that returns `(resolved, bytes)`: 1. Path.resolve() chases pre-existing symlinks and the resulting target is checked for containment (existing behaviour -- legitimate in-root symlinks still work). 2. The read goes through `os.open(resolved, O_RDONLY | O_NOFOLLOW)` so a fresh symlink placed at the resolved name *after* the check fails with ELOOP rather than following the swap. 3. `fstat` + `S_ISREG` rejects directories / device nodes / pipes atomically on the same fd, replacing the racy `is_file()` test the callers used to do. Both register_source_from_path handlers (MCP + JSONL) switch to the new helper and drop their now-redundant follow-up checks. Adds tests: - symlink swapped into the resolved name -> rejected via ELOOP - directory at a valid path -> rejected via S_ISREG Existing "outside the root" and "inside the root" tests still pass.
…versal fix(server): block path traversal in register_source_from_path
ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged.
The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself.
Resolve conflicts in context.py, sessions.py, storage.py (keep main). fix: block bad writes in import_apply per review feedback (#13) import_apply now captures schema validation issues and skips the file instead of passing a throwaway list.
fix(cli): translate domain errors into clean ClickException output
fix(bundle): reject path traversal in import (CVE-2007-4559, #9)
fix(verify): catch ArtifactNotFoundError on missing stored content
Approved design for feat/semantic-search. Embedding (sentence- transformers all-mpnet-base-v2) becomes the primary search backend with FTS5 as fallback. Synchronous-at-write indexing across all six artifact types (claim, page, source, entity, relation, evidence). Maximally functional scope (~3000 LOC): pluggable model adapter registry, sqlite-vec ANN with NumPy fallback, cross-encoder rerank, HyDE query expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG eval harness, and full CLI/MCP/JSONL parity. The writing-plans step will turn the rollout order in section 16 into concrete phased tasks.
32-task TDD plan for the semantic-search feature: foundation, storage, write hooks across all six artifact types, semantic-primary search integration in MCP/JSONL/CLI, RRF fusion + hybrid, cross-encoder rerank, HyDE expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG scorer, end-to-end integration test, and user docs. Each task: failing test, minimal implementation, passing test, commit. MockEmbedder test double keeps the unit suite fast; the real model is exercised only under @pytest.mark.integration. The evaluation module is named scorer.py rather than eval.py to avoid shadowing the Python builtin and to keep static analysers quiet; the user-facing CLI subcommand remains `vouch eval embedding` (the Click group is registered under the name "eval", with the Python identifier eval_group).
CI ran ruff which flagged SIM105 on the three try/except ImportError guard blocks in src/vouch/embeddings/__init__.py. Replace each with `contextlib.suppress(ImportError)` -- same semantics, satisfies ruff. Also add mypy overrides for numpy / sqlite_vec / sentence_transformers / fastembed so the type check passes in the base [dev] CI install where those optional extras aren't present. The embedding code paths that import these libraries are only reached when the extras are installed at runtime; for the static type check the missing stubs are noise.
CI runs `pip install -e '.[dev]'` which deliberately excludes the
optional `[embeddings]` extras. tests/embeddings/test_*.py modules
import numpy at top level, so pytest collection fails with
ModuleNotFoundError before any test can be deselected.
Add tests/embeddings/conftest.py with `pytest.importorskip("numpy")`
so the entire embeddings test directory skips gracefully when numpy
is absent. Once `pip install vouch[embeddings]` (or numpy itself) is
present, the skip is a no-op and the tests run normally.
`_validate_content` in bundle.py uses the path's first directory component to look up a Pydantic validator. For sources, both `sources/<sha>/meta.yaml` (the Source model) and `sources/<sha>/content` (raw opaque bytes) hit the same "sources" key, so the validator was being run on the raw content bytes and failing with "1 validation error for Source". Add an early return for any non-meta.yaml path under `sources/` so opaque content bytes are not Pydantic-validated. Only the Source metadata file is checked, which matches the original intent of the PR #13 validation. Unblocks three pre-existing test_bundle.py failures inherited from upstream main.
…s cannot fork the hash chain (#263) without the lock, two log_event calls racing on the same kb both observe the same prev_hash from _last_hash, both append to audit.log.jsonl, and verify_chain reports "previous hash mismatch" at the second concurrent entry — permanently breaking the tamper-evidence guarantee #244/#253 added. reachable from vouch serve handling concurrent agents, vouch serve alongside cli commands on the same kb, scripted backgrounded approvals, and scheduled expire / sync runs. holds an exclusive lock on a sibling audit.log.jsonl.lock file for the duration of read-prev-hash → derive → append. uses fcntl.flock on posix and msvcrt.locking on windows. the audit log itself is never opened in a mode that could truncate it. dynamic __import__ keeps mypy clean on both platforms. new regression spawns four multiprocessing.spawn workers writing 20 events each and asserts verify_chain.ok plus the full 80 events landed — threads hide the race behind the gil. a second test asserts the lock is released on an exception mid-write so the next call succeeds. Fixes #262
…pic (closes #315) (#347) * feat(experts): kb.experts — rank entities by evidence density on a topic Closes #315. Add a read-only kb.experts query: given a free-text topic, rank the entities carrying the most matched evidence (count / recency / citation weightings) identically across mcp / jsonl / cli. Aggregates approved, live claims only — excludes superseded/archived/redacted so a non-live claim never inflates a score; no proposals, writes, network, or llm. Ranking lives in a new src/vouch/experts.py, wired at the four registration sites. * test(experts): cover the kb.experts JSONL request/response envelope The suite exercised rank_experts() directly but not the kb.experts JSONL entrypoint. Add two envelope tests mirroring tests/test_jsonl_server.py: a well-formed request returns {id, ok, result} with the ranking under result["experts"], and a request missing the required `topic` param returns the {id, ok: false, error} failure envelope (code "missing_param"). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: plind <59729252+plind-junior@users.noreply.github.com> Co-authored-by: e11734937-beep <e11734937@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
…sals (#323) (#353) reviewers scan the pending queue linearly and every item looks the same weight; the ones that most deserve a hard look blend in. the near- duplicate direction already surfaces (find_similar_on_propose); this surfaces the outlier direction. scores every pending claim proposal worst-first with reason codes: - thin_evidence: evidence count at or below a floor (barely cited) - contradicts_many: declares contradictions against >= n approved live claims (retired claims don't count) - far_from_corpus: nearest-neighbour cosine to the approved claim corpus below a floor (an outlier). embedding-derived, so it degrades gracefully to no code when the embeddings extra is absent — the same swallow find_similar_on_propose does — leaving the two non-embedding codes still computed. thresholds resolve from review.anomaly.* (mirroring similarity_threshold). read-only by construction: a hint for the reviewer that emits no proposal, writes no artifact, and never rejects or quarantines — the human gate is untouched. scoring lives in a new anomaly.py, not storage.py. shipped cli-only (`vouch flag-anomalies`); the kb.flag_anomalies method and a list-pending --flag-anomalies flag are a separate concern. Co-authored-by: plind <59729252+plind-junior@users.noreply.github.com>
…268) the citation gate and uncited_items listed claims dropped by the max_chars budget, so a pack could fail require_citations for items the caller never received. evaluate uncited only over the returned item list. fixes #174. Co-authored-by: Yaroslav98214 <diakovichyaroslav30@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: plind <59729252+plind-junior@users.noreply.github.com>
…shes (#360) * fix: resolve list_pages crash on corrupt page files list_pages() lacked the _load_or_skip resilience applied to other list_* paths, so one bad pages/*.md file crashed status, search, lint, and MCP tools. Skip unreadable pages with a warning instead. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(scripts): add repro script for list_pages crash (PR #360) Gives maintainers a one-command proof of the pre-fix crash path and the fixed skip-and-continue behavior. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>
Add a numbered, imperative install guide for agents (closes #329): detect host via install-mcp --list, wire MCP, verify capabilities, init KB, and run a propose-to-human-approve smoke test. Link from llms.txt and distinguish from getting-started.md. Co-authored-by: luciferlive112116 <luciferlive112116@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com>
contradict() had no guard against claim_a == claim_b, unlike its sibling supersede() which explicitly rejects self-reference. calling it with the same id on both sides wrote a self-loop contradicts reference and flipped the claim to contested with no real counterparty -- durable, nonsensical state with no way to recover short of manual yaml editing. Co-authored-by: plind <59729252+plind-junior@users.noreply.github.com>
* docs: add local dev install to readme, fix dev setup steps following the documented dev setup verbatim fails twice on a clean machine: `python -m venv` breaks on stock ubuntu (only `python3` exists), and `make check` after `pip install -e '.[dev]'` fails at the mypy step because src/vouch/web imports jinja2, which lives in the web extra. ci installs '.[dev,web]' (.github/workflows/ci.yml), so the docs now match: python3 + '.[dev,web]' in readme, contributing, and claude.md. also adds the previously missing local-development install block to the readme's install section. verified end-to-end on a fresh clone: install, vouch --version, vouch status, and a green make check (1092 passed). * docs: note webapp build step for source checkouts `vouch console` serves prebuilt assets that only release wheels ship (hatch_build.py bundles webapp/dist); a fresh clone has no build, so the command errors with "no built vouch console found" even after `pip install -e '.[dev,web]'`. document `make webapp-build` (one-time, needs node) and `make console` (backend + console dev server) in the readme dev block and contributing dev setup. verified in a fresh clone on the test branch: after `make webapp-build`, `vouch console` serves the react console at http 200. * docs: make readme dev block a full runnable sequence the dev-install block listed clone + venv + pip but left the webapp to prose, so following the commands alone never got the console running. the block now carries the whole verified path: editable install, `vouch --version` smoke check, `make console` for the backend + vite dev-server pair, and `make check` for the ci gate. verified in a fresh clone on the test branch: `make console` brings up `vouch serve --transport http` on :8731 (streamablehttp session manager started) and the vite console dev server, node deps auto-installed on first run.
# Conflicts: # src/vouch/cli.py # src/vouch/index_db.py # src/vouch/jsonl_server.py # src/vouch/server.py # src/vouch/storage.py # webapp/src/main.tsx
date the changelog and reconstruct entries lost in earlier merge conflict resolutions: transcript viewer, review-gated delete, session split, codex adapter, mcp tool profiles, per-prompt recall, triage, fusion-by-default, and the path-traversal fix. refresh the roadmap to reflect the shipped 1.x line and the road to connected kbs.
# Conflicts: # CHANGELOG.md
|
Important Review skippedToo many files! This PR contains 181 files, which is 31 over the limit of 150. To get a review, narrow the scope: Upgrade to a paid plan to raise the limit. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (181)
You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
added
docs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
15 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
releases the 1.3.0 line: the web console (
vouch console, dashboard, chat with llm answers, session transcript viewer), review-gated artifact delete, session-split summaries, the codex adapter tier, mcp tool profiles with per-prompt auto-recall, retrieval fusion by default, and the audit-log file lock.release prep in this pr beyond the feature branch itself:
version already agrees at 1.3.0 across pyproject.toml, openclaw.plugin.json, package.json, and
__init__.py. full gate green locally: pytest (non-embeddings), mypy src, ruff.tag v1.3.0 follows once this merges.