Skip to content

cli: fold burn by-tool into burn summary --by-tool #156

@willwashburn

Description

@willwashburn

Summary

burn by-tool is structurally a summary mode — same query path, same filters, different aggregator and output shape. Move it under summary as --by-tool, matching the existing --by-provider, --by-subagent-type, and --subagent-tree mode flags.

Background

burn summary already has three "mode flags" that swap the grouping axis and the output shape:

  • --by-provider — table grouped by provider instead of model (commands/summary.ts:50,63-65)
  • --by-subagent-type — aggregated subagent stats, different columns
  • --subagent-tree <session-id> — tree layout, not a table at all

burn by-tool does the same thing in spirit:

  • Same prelude: ingestAllloadPricingqueryAll with the same filters (commands/by-tool.ts:21-23 is literally the prefix of commands/summary.ts:52-54).
  • Different aggregator: attributeCostToTools (by-tool.ts:59-112) splits turn N's input cost across turn N-1's tool_use blocks.
  • Different output shape: tool | calls | attributedCost instead of model | turns | input | output | reasoning | cacheRead | cacheCreate | cost.

Filter parity gap: by-tool accepts --since, --project, --session, --provider but is missing --workflow and --agent (by-tool.ts:11-15). Folding into summary closes that gap by inheritance.

Proposed shape

burn summary --by-tool [--since 7d] [--project <path>] [--session <id>]
                       [--workflow <id>] [--agent <id>] [--provider <p>] [--json]

Output columns stay as today (tool | calls | attributedCost), and the existing attribution-method footer carries over verbatim:

attributedCost = (turn N input cost) split evenly across tool_use blocks in turn N-1, grouped by tool name.
unattributed cost (no prior tool call, e.g. first turn): $X.XX

This is exactly the same precedent --subagent-tree set: a mode flag that produces an output shape unrelated to the default model/provider table. Don't try to reshape by-tool's columns into the default schema — that's a worse design than the current standalone command.

Implementation Steps

No users yet — this lands as a clean break with no deprecation alias path.

Single PR.

  1. Move the aggregator:
    • Lift attributeCostToTools (commands/by-tool.ts:59-112) into commands/summary.ts (or a sibling helper file commands/summary-by-tool.ts if summary.ts is getting large — it's already 441 lines).
  2. Wire up the mode in runSummary:
    • Read args.flags['by-tool'] alongside the existing byProvider, subagentTreeFlag, subagentTypeFlag checks (summary.ts:48-61).
    • Add a renderByToolMode(args, turns, pricing) branch parallel to renderSubagentTreeMode / renderSubagentTypeMode. Same dispatch shape, returns Promise<number>.
    • Mode precedence: explicit subagent flags first (existing), then --by-tool, then default --by-provider | by-model. Document the precedence in the help block.
  3. Mode exclusivity:
    • If --by-tool is combined with --by-provider / --by-subagent-type / --subagent-tree, exit non-zero with a clear error: "--by-tool cannot be combined with --by-provider/--by-subagent-type/--subagent-tree". Mirrors how the existing modes implicitly assume one-axis-at-a-time.
  4. JSON contract:
    • summary --by-tool --json emits { ingest, turns, byTool: [{ tool, calls, attributedCost }], unattributed, fidelity }. Keep the fidelity block parallel to the model/provider JSON shapes (summary.ts:69-101) so programmatic consumers see one schema family.
  5. CLI dispatch & help:
    • Update the help block in cli.ts:22-78 — drop the burn by-tool line, add --by-tool to the summary flag list with a one-line note about the different output shape and metric.
    • Drop the case 'by-tool' branch from cli.ts:95-96.
  6. Tests:
    • Move commands/by-tool.test.ts content into summary.test.ts (or keep as summary-by-tool.test.ts) — same fixtures, point them at runSummary with --by-tool.
    • Add a mode-exclusivity test (--by-tool --by-provider should fail with the expected message).
  7. Docs: README, AGENTS.md, landscape.md, CHANGELOG [Unreleased] on @relayburn/cli. Most user-facing examples are burn by-tool --since 7d — update each.

Files touched (scope)

  • packages/cli/src/commands/by-tool.ts — removed
  • packages/cli/src/commands/summary.ts — new --by-tool branch + aggregator helper
  • packages/cli/src/commands/by-tool.test.ts — merged into summary.test.ts or renamed
  • packages/cli/src/cli.ts — dispatch + help block
  • README.md, AGENTS.md, landscape.md, packages/cli/CHANGELOG.md

No external API change — attributeCostToTools is currently un-exported from @relayburn/analyze (it lives in the CLI). Stays internal.

Risks / questions

  • Metric mixing. attributedCost (the by-tool number) and cost (the by-model/by-provider number) are not the same number. The existing footer in by-tool.ts:44-46 explains why; that explanation must ride along into the new mode's output, not be lost in a denser combined help block. Worth reviewing the JSON schema for the same reason — programmatic consumers should be able to tell which metric they're looking at.
  • Help-block density. summary already lists eight flags; adding --by-tool plus its precedence note pushes it close to the readability cliff. Consider promoting the mode flags to a small "Modes:" subsection in the help text rather than mixing them with filters.

Out of scope

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions