Skip to content

feat(capture): make real-time capture opt-in, drop the per-tool-call hook - #690

Closed
dripsmvcp wants to merge 2 commits into
vouchdev:testfrom
dripsmvcp:fix/coerce-bool-tail
Closed

feat(capture): make real-time capture opt-in, drop the per-tool-call hook#690
dripsmvcp wants to merge 2 commits into
vouchdev:testfrom
dripsmvcp:fix/coerce-bool-tail

Conversation

@dripsmvcp

Copy link
Copy Markdown
Contributor

closes #602

what

capture.realtime is a new config key, default false. vouch capture observe returns immediately when it is off — before any file i/o — and the
shipped claude-code adapter no longer registers a PostToolUse hook at all.

on a busy session that hook was hundreds of python process spawns, each
loading the kb store, to append a line to an ephemeral buffer whose only
consumer is the once-per-session rollup. the knowledge itself has been batched
since passive answer memory landed: capture.answer_mode defaults to
session, so claims are extracted once at finalize from the full transcript.

the Stopvouch capture answer hook goes with it, per the issue. under the
default answer mode it is a pure deferral — it fires, decides it is in session
mode, and exits. SessionStart, UserPromptSubmit and SessionEnd are the
three that carry weight, and that is what the adapter ships now.

the compatibility half — item 3, which the issue is right to call

non-optional

summarize gates on len(observations) + len(changed_files) < min_observations, so an empty buffer would have left any session touching
fewer than 3 files with no summary at all.

capture.finalize now reconstructs the session's tool activity from the
transcript it already reads (capture.observations_from_transcript):

  • each tool_use block is paired with the tool_result carrying its id, so
    the Bash branch still tells a failed command from a clean one — is_error
    is folded into the text summarize_tool greps, and it wins over content
  • secrets are masked exactly as observe masks them, because these records
    roll into a committed page and the append-only audit log
  • an unreadable transcript yields [] rather than costing the session its
    summary

same observation shape, same rollup, one more front door onto it —
codex_rollout already does precisely this for codex rollouts.

when realtime is on

both sources then describe the same tool calls. session_split._merge_observations
unions them on tool_use_id (falling back to tool/summary/cmd identity for
older buffers that predate the id) and orders by timestamp, so a call recorded
by both counts once against min_observations and renders once in the
activity list.

compatibility

no stored artifact changes shape. installed hosts keep their current
.claude/settings.json until they re-run vouch install-mcp — noted in the
changelog. capture observe stays on the CLI and stays callable, it just
defaults to a no-op, so this is not a surface change and needs no VEP.

tests

new in tests/test_capture.py: the default is off and a quoted "true" still
enables it; observe writes no buffer file when off; reconstruction returns
the right tools in transcript order with ids, timestamps, the failed-Bash
summary and the files/cmd fields, skips unobserved tools, masks secrets,
and survives a missing path; finalize with realtime off clears
min_observations on reconstructed activity alone and renders it into the
page body; and with realtime on, a call in both the buffer and the transcript
is counted and rendered once.

existing suites that seed the buffer through observe now opt into
realtime explicitly (fixtures in test_capture.py/test_session_split.py,
call-site configs in test_capture_answer.py/test_adopt.py/
test_session_transcript.py), and the adapter-shape assertions in
test_capture.py/test_install_adapter.py now pin the absence of
PostToolUse/Stop rather than their presence.

full gate green locally: pytest tests/ --ignore=tests/embeddings,
mypy src, ruff check src tests.

…hook

new config key `capture.realtime`, default false. `capture observe`
returns immediately when it is off, before any file i/o, and the shipped
claude-code adapter no longer registers a `PostToolUse` hook at all.

on a busy session that hook was hundreds of python process spawns, each
loading the kb store, to append a line to an ephemeral buffer whose only
consumer is the once-per-session rollup — the knowledge itself is already
batched, since `capture.answer_mode` has defaulted to `session` since
passive answer memory landed.

the `Stop` -> `capture answer` hook goes with it. under the default
answer mode it is a pure deferral: it fires, decides it is in session
mode, and exits. `SessionStart`, `UserPromptSubmit` and `SessionEnd` are
the three that carry weight.

removing the buffer without replacing it would have cost small sessions
their summary — `summarize` gates on
`len(observations) + len(changed_files) < min_observations`, so with an
empty buffer only the git-diff backstop counts. `finalize` now
reconstructs the session's tool activity from the transcript it already
reads (`observations_from_transcript`), pairing each tool_use with the
tool_result carrying its id so a failed bash still reads as failed, and
masking secrets the same way `observe` does. same observation shape, same
rollup, one more front door — codex_rollout already does this for codex.

with `realtime: true` both sources describe the same calls, so
`_merge_observations` dedups on `tool_use_id` (falling back to
tool/summary/cmd for older buffers) and orders by timestamp: a call
recorded twice counts once, in the buffer's position.

no stored artifact changes shape, and installed hosts keep their current
settings.json until they re-run `vouch install-mcp`.
@dripsmvcp
dripsmvcp requested a review from plind-junior as a code owner July 30, 2026 21:50
@github-actions github-actions Bot added docs documentation, specs, examples, and repo guidance adapters agent host adapters and install manifests storage kb storage, migrations, schemas, and proposals tests tests and fixtures size: M 200-499 changed non-doc lines labels Jul 30, 2026
…erge

the diff-coverage gate wants 100% of changed python. the uncovered lines
were all the tolerant branches — which are the ones worth a test anyway,
since a transcript is the host's file and one malformed row must not cost
the session its summary.

one messy transcript exercises every skip in a single pass: blank and
whitespace lines, undecodable json, a bare-string document, a non-dict
message, a non-list content, a bare-string block, tool_use with a missing
and a non-string name, an orphan tool_result, and an unparseable
timestamp degrading to 0.0. plus the max_observations ceiling.

for the merge: id-based dedup, the no-id fallback on tool/summary/cmd for
buffers that predate tool_use_id, timestamp ordering, and the
short-circuit when either side is empty.
@github-actions github-actions Bot added size: L 500-999 changed non-doc lines and removed size: M 200-499 changed non-doc lines labels Jul 30, 2026
@plind-junior

Copy link
Copy Markdown
Member

the argument holds and the compatibility half is the part that makes it shippable — summarize gating on len(observations) + len(changed_files) < min_observations means an empty buffer would have cost every small session its summary entirely, and reconstructing the observations from the transcript finalize already reads is a better answer than lowering the threshold. pairing each tool_use with the tool_result carrying its id, so the Bash branch can still tell a failed command from a clean one, is the detail that would have been easy to drop and would have quietly degraded every summary.

checked the sibling-adapter question, since removing a hook from one adapter usually leaves another behind: adapters/codex/hooks.json registers only UserPromptSubmit (context-hook) and Stop (capture ingest-codex) — no PostToolUse — so codex was never paying this cost and needs no matching change. nothing left behind there.

the thing i would want resolved is what capture.realtime: true actually does after this. the shipped .claude/settings.json no longer registers PostToolUse at all, and that is the only thing that ever called vouch capture observe on this adapter. so an operator who sets the key to true gets no realtime capture — they also have to hand-add a hook the installer no longer writes and the docs no longer describe. the install.yaml comment explains why the hook is gone by default but not how to get it back.

that leaves a config key whose "on" state is unreachable through any shipped path, which is the shape that ages into a dead flag. either install-mcp should emit the hook when the key is set, or the key should go and the answer becomes "realtime capture is not supported on claude-code; use the transcript path" — both are defensible, but the current middle state documents an option the product does not provide.

the merge-order note is worth stating too: #679 drives correction capture from UserPromptSubmit, which is one of the three hooks this PR keeps, so the two are compatible — but they both edit capture behaviour and whichever lands second should re-check the adapter's hook list rather than assume it.

@plind-junior

Copy link
Copy Markdown
Member

duplicated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

adapters agent host adapters and install manifests docs documentation, specs, examples, and repo guidance size: L 500-999 changed non-doc lines storage kb storage, migrations, schemas, and proposals tests tests and fixtures

Projects

None yet

Development

Successfully merging this pull request may close these issues.

make real-time capture opt-in: default capture.realtime off, drop PostToolUse observe from adapters

2 participants