Skip to content

Aggregate parser content-capture gaps in burn diagnose #79

@willwashburn

Description

@willwashburn

Context

#59 / #75 added a per-invocation, per-adapter ingest-time warning when a parser commits turns containing tool calls but emits zero tool_result ContentRecords. The warning fires at most once per burn invocation per adapter and is gated on fresh affected sessions, which is correct for live ingest UX but means there's no persistent, queryable view of the gap. After a single warned invocation, a user re-running burn later sees nothing — even though the underlying degradation (burn waste falling back to even-split attribution for those sessions) is still in effect.

PR #75 explicitly deferred this as a follow-up:

Out of scope (deferred)

  • Aggregate count surfaced in burn diagnose as a permanent state report — issue calls this out as a separate follow-up.

Issue #59's implementation sketch made the same call:

Aggregation can also be surfaced in burn diagnose as a permanent state report rather than a per-invocation warning. Both layers are useful; consider shipping the warning first and then wiring the aggregate count into diagnose as a separate follow-up.

Today burn diagnose is strictly per-session — it requires a sessionId positional and bails with exit 2 when none is provided (packages/cli/src/commands/diagnose.ts:18-22). There's no "global state" mode that scans the ledger and reports adapter-level health. The gap-counting helper already exists and is exported (countToolCallGaps in packages/cli/src/ingest.ts), but its results are thrown away after each ingest pass.

Proposal

Add a no-argument variant (or --all flag) to burn diagnose that walks the ledger and emits a per-adapter content-capture health report. For each adapter (claude / codex / opencode) present in the ledger:

Implementation outline:

  1. In packages/cli/src/commands/diagnose.ts, branch when args.positional[0] is absent (today this is exit 2). Run ingestAll(), then for each adapter scan turns + content via queryAll / readContent and apply the existing countToolCallGaps(turns, content) helper per session.
  2. Render a small fixed-width table (sessions / withToolCalls / gapped / orphanToolCalls / degraded%) similar to the existing Top files by cost table at diagnose.ts:98-107.
  3. Honor --json for machine-readable output, matching the existing per-session JSON branch at diagnose.ts:43-60.
  4. Skip rows where contentMode is hash-only / off for that adapter's most recent ingest, with a one-line note explaining why no gap signal is available — mirrors the warning's gating in ingest.ts.

This is a read-only state report. It does not change ingest behavior, does not gate exit code, and does not modify the existing per-session diagnose path.

Acceptance criteria

  • burn diagnose (no positional arg) exits 0 and prints a per-adapter content-capture gap table instead of the current exit-2 "missing session id" error.
  • burn diagnose --json (no positional) emits { adapters: [{ adapter, sessions, sessionsWithToolCalls, gappedSessions, orphanToolCalls, degradedPct }, ...] }.
  • The existing burn diagnose <sessionId> behavior is unchanged (same output, same exit codes, same --json shape).
  • On a ledger where codex/opencode have tool calls but zero tool_result records, the report shows the gap counts and degradedPct > 0.
  • On a ledger where the parser captures content correctly, the report shows gappedSessions: 0 for that adapter.
  • Adapters with no sessions in the ledger are omitted (or shown with zeroes — pick one and document).
  • Test in packages/cli/src/commands/diagnose.test.ts (new file or extended) using a synthetic ledger with mixed gap / non-gap sessions across adapters.

Out of scope

Refs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions