Skip to content

burn plans: honor fidelity (mark monthly spend totals low-confidence on partial usage) #108

@willwashburn

Description

@willwashburn

Context

PR #76 (#41 first cut) shipped summarizeFidelity and hasMinimumFidelity in @relayburn/analyze, but stopped short of wiring those helpers into the commands that consume the ledger. Quoting PR #76's deferred-work paragraph:

burn compare, burn waste, burn limits, burn plans behavior gating on fidelity class — the helpers (hasMinimumFidelity, summarizeFidelity) are now in place; wiring them into each command is the natural follow-up.

burn plans (packages/cli/src/commands/plans.ts) lists plans and runs computePlanUsage against queryAll(...) to render monthly spend against each plan's budget. computePlanUsage sums costForTurn(...) across the cycle, so a turn whose source omitted token fields contributes \$0 and silently makes a plan look under-budget. Quoting #41:

burn limits / burn plans (#5, #39) … should permit partial usage data where enough exists for spend totals … should mark projections as low-confidence when the underlying fidelity is partial.

Proposal

In packages/cli/src/commands/plans.ts (specifically the runList path that renders plan status) and computePlanUsage (packages/analyze/src/plan-usage.ts):

  1. Permissive filter. Like limits, plans is allowed to include partial / aggregate-only / cost-only data — incomplete totals are still useful as a lower bound. Do not default-exclude.
  2. Annotate per-plan confidence. Extend PlanUsage with a fidelity field: { confidence: 'high' | 'low', summary: FidelitySummary } computed over the cycle's contributing turns. confidence === 'high' when every turn in the cycle is full or usage-only with hasInputTokens && hasOutputTokens; otherwise low.
  3. Surface in the human-readable table. When any plan's cycle has low confidence, render a note column or a footer line: note: 3 of 412 turns this cycle lack per-turn token data — totals are a lower bound. Suppress when all cycles are full (avoid noise in the common case, matching summary's pattern).
  4. JSON contract. Where plans already emits JSON (the --json path on runList), add the fidelity block per plan. Programmatic consumers reading monthly spend get the same confidence + FidelitySummary shape summary --json already establishes.
  5. cost-only sources count toward spend. A cost-only source provides a price even when it provides no tokens — spend totals should include it. Mark the plan low-confidence on the token-coverage axis but do not drop the cost.

Acceptance criteria

  • burn plans continues to render monthly totals when the cycle slice contains partial / aggregate-only / cost-only turns.
  • Each rendered plan row shows a low-confidence note when any contributing turn lacks per-turn token coverage; full-fidelity cycles show no note.
  • PlanUsage exposes a fidelity field ({ confidence, summary }) and --json emits it per plan.
  • cost-only source contributions count toward spend totals, with the cycle marked low-confidence on the token-coverage axis.
  • New tests in packages/analyze/src/plan-usage.test.ts cover: high-confidence cycle (all full), low-confidence cycle (partial turn), cost-only contributions counted toward spend.
  • New tests in packages/cli/src/commands/plans.test.ts cover the rendered note and the JSON shape.

Out of scope

  • The add / remove / set-reset-day subcommands — they don't read the ledger.
  • Cursor-style providers that lack spend tracking entirely (Collector: Cursor — won't implement (data moved server-side) #22) — they already render the "spend tracking unavailable" note independent of fidelity.
  • Per-day breakdowns of confidence within a cycle. Cycle-level binary flag is enough for the first cut.

Refs

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions