feat(synthesize): file good answers back as page proposals - #737
Open
philluiz2323 wants to merge 2 commits into
Open
feat(synthesize): file good answers back as page proposals#737philluiz2323 wants to merge 2 commits into
philluiz2323 wants to merge 2 commits into
Conversation
kb.synthesize already produces citation-bearing prose traceable to
approved claims, with a confidence grade - everything needed to file
a page proposal was already computed, just not wired to propose_page.
turning a good answer into a page meant manually copying the text and
re-typing the cited claim ids into a separate kb.propose_page call.
add an opt-in file_as_page parameter: when set, files the answer back
as a PAGE proposal via the existing propose_page, citing the same
claim ids the answer already cites. still fully gated - propose_page
files a pending proposal, never approves anything, identical to every
other write path in this codebase.
skip filing (not an error, not a crash) when the answer is empty or
cites nothing - an uncited "answer" is the kb saying it doesn't know,
not knowledge worth filing - or when propose_page itself fails (e.g. a
title collision): the synthesis already succeeded, and losing that to
a secondary optional step would be worse than reporting why filing
didn't happen. both outcomes surface via new page_proposal_id (None
when skipped/failed) and page_proposal_skipped_reason result fields.
extracted the existing no-llm path into _deterministic_synthesize so a
new _maybe_file_page_proposal helper can wrap either backend's result
the same way - the llm path already produces the identical
{query, answer, claims, pages, gaps, _meta} shape.
registered on mcp (kb_synthesize), jsonl (kb.synthesize), and cli
(vouch synthesize --file-as-page [--page-title T] [--agent A]).
opt-in and additive - file_as_page defaults to False, so every
existing caller's result shape is unchanged.
Closes vouchdev#736
diff-coverage flagged two branches in _maybe_file_page_proposal as untested: the gaps-section body append (only reachable when file_as_page=True AND the query is partially covered - the existing tests either fully cover or fully miss), and the except (ProposalError, ArtifactNotFoundError) skip path. add a partially-covered query (some cited claims, some uncovered terms) asserting the gaps land in the filed page body, and a monkeypatched propose_page failure asserting it degrades to a reported skip reason without losing the already-successful synthesis.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
kb.synthesizegains an opt-infile_as_page: bool = Falseparameter(plus
page_title: str | None = None). When set, the synthesized answeris filed back as a
PAGEproposal via the existingpropose_page, citingthe same claim ids the answer already cites. Registered on all three
transports plus CLI: MCP
kb_synthesize(..., file_as_page, page_title),JSONL
kb.synthesizeparams,vouch synthesize --file-as-page [--page-title T] [--agent A].Why
ROADMAP.md's 1.4 milestone: "Good synthesized answers can be filed back
as page proposals — gated by review like every other write."
kb.synthesizealready produces exactly that shape — citation-bearing prose traceable to
approved claims — but nothing wired it to
propose_page; turning a goodanswer into a page meant manually copying the text and re-typing the
cited claim ids into a separate call.
Closes #736
Still fully gated:
propose_pagefiles a pending proposal, the same asevery other write path in this codebase — nothing here approves anything.
Filing is skipped (not an error) when the answer is empty or cites
nothing — an uncited "answer" is the KB saying it doesn't know, not
knowledge worth filing — or when
propose_pageitself fails (e.g. atitle collision): the synthesis already succeeded, and losing that to a
secondary, optional step would be a worse failure than reporting why
filing didn't happen. Both outcomes surface via new
page_proposal_id(
Nonewhen skipped or failed) andpage_proposal_skipped_reasonresultfields.
Internally, the existing no-LLM path is extracted into
_deterministic_synthesizeso a new_maybe_file_page_proposalhelpercan wrap either backend's result the same way — the LLM path already
produces the identical
{query, answer, claims, pages, gaps, _meta}shape, so one wrapper covers both without duplicating the filing logic.
What might break
Nothing — purely additive and opt-in.
file_as_pagedefaults toFalse,so every existing caller (MCP, JSONL, CLI) gets back the exact same
result shape as before; the new
page_proposal_id/page_proposal_skipped_reasonkeys are only present whenfile_as_pagewas requested. No on-disk shape, object model, or
kb.*method removal —propose_pageand the review gate itself are completely untouched.VEP
Not applicable — no object model, on-disk layout, bundle format, or
audit-log shape change. A new opt-in parameter on an existing read method
that calls an existing, unmodified write path.
Tests
make check-equivalent: ruff clean (src+tests); mypyclean;
tests/test_capabilities.py,tests/test_cli.py,tests/test_hot_memory.py,tests/test_synthesize.pyall pass(the 3 pre-existing LLM-backend test failures are a known Windows
python3-stub-invocation environment issue, confirmed presentidentically on
testHEAD viagit stashcomparison, unrelated tothis change)
tests/test_synthesize.py:files a pending proposal with the right claims/body/actor, custom
title, skip-on-empty-answer,
ValueErrorwithoutproposed_by,default-off backward compatibility, plus JSONL and CLI
(
CliRunner) end-to-end casesCHANGELOG.mdupdated under## [Unreleased]