Skip to content

zaakirio/ingot

Repository files navigation

Ingot

Ingot

Join AI coding-agent token spend with git history and answer: what did the tokens buy?

Companies are spending heavily on AI coding agents. Leadership tracks token spend, dashboards show tokens per day and dollars per model, and nobody measures any of it against what actually shipped. There is no token:deliverable metric. Ingot builds one: it ingests agent-usage telemetry, reads your git history, joins the two by working directory and time window, and reports cost per commit, cache efficiency, and the spend that produced no commits at all.

How it works

usage adapters          deliverable adapter
(claude-code,           (git log --numstat)
 langfuse-export)              |
      |                        |
      v                        v
   normalized sqlite store (tokens + metadata only)
                 |
                 v
   join: cwd prefix -> repo, timestamp -> window
                 |
                 v
   metrics -> markdown / JSON / static HTML report
  1. Usage adapters parse agent telemetry into normalized usage events: timestamp, model, token counts, session id, working directory. Two adapters ship today (Claude Code transcripts and Langfuse exports); an adapter is one module with a single scan() function, so more can be added.
  2. The deliverable adapter reads commit history from your repos via git log --numstat and classifies each commit: tests touched, docs touched, source touched, revert or not.
  3. The join attributes usage to repos by longest working-directory prefix match and buckets both sides into time windows (days or ISO weeks).
  4. Metrics price the tokens from a config-declared cost table and divide spend by what landed.

Quickstart

uv sync
cp ingot.example.yaml ingot.yaml   # edit sources, repos, prices
uv run ingot scan                  # ingest into ingot.db
uv run ingot report                # markdown to stdout
uv run ingot report --format json --out report.json
uv run ingot html --out report.html

ingot scan is incremental and idempotent: usage events are deduplicated by message id, and re-running a scan only adds what is new. Prices are applied at report time, so changing the cost table never requires a rescan.

Configuration

See ingot.example.yaml for the full annotated reference. The three decisions that matter:

  1. Sources. For Claude Code, include is a strict allowlist of project-directory globs. Directories that do not match are never opened, which is how you keep work projects out of a personal analysis (and vice versa).
  2. Repos. Commits in these repos are the deliverable side of the join. Each entry takes an optional name used as the label in reports, so published output does not need to contain your filesystem paths.
  3. Pricing. USD per million tokens per model, including cache read and cache write rates, with the source of each price commented.

Privacy guarantees

Ingot is built to be pointed at real transcripts, so the privacy rules are hard guarantees, not defaults:

  1. No conversation text is ever stored. Adapters extract token counts and routing metadata (timestamp, model, session id, cwd, branch) from an explicit field allowlist and never read message.content or Langfuse input/output payloads.
  2. The directory allowlist is strict. A Claude Code project directory whose name does not match an include glob is never opened, let alone parsed.
  3. Both guarantees are tested. The fixture transcripts plant sentinel strings in every text-bearing field; tests/test_privacy.py scans them and asserts the sentinels appear nowhere in the raw bytes of the resulting sqlite database, and that non-allowlisted directories contribute nothing.

Worked example: this repo's own development

Real numbers from the machine this project was built on. The scan allowlisted the eleven Claude Code project directories of one personal inference-engineering workspace (~/.claude/projects/-*-inf-eng*) and joined them against five repos: four portfolio projects (anvil, sluice, assay, bellows) and an eval harness (crucible). Observed period: 2026-06-07 to 2026-07-07.

spend              $561.2785 across 3,790 api responses in 45 sessions
total tokens       1,083,985,397
  fresh input      408,377
  output           3,662,911
  cache read       1,058,462,021
  cache write      21,452,088

by model
  claude-fable-5     $198.2326    110,967,533 tok      665 responses
  claude-sonnet-4-6  $146.4594    367,017,236 tok    1,309 responses
  claude-sonnet-5    $131.4129    514,877,893 tok    1,236 responses
  claude-opus-4-8     $85.1737     91,122,735 tok      580 responses

cache efficiency
  hit rate           98.0% of prompt tokens served from cache
  saved by reads     $3,224.4751 vs paying fresh-input price
  net cache win      $3,200.9242 after the $23.5510 write premium

deliverables
  commits in period  51
  cost per commit    $11.0055 overall (all spend / all commits)
  crucible           $231.3977 attributed, 47 commits, $4.9234 per commit
                     (+24,097 / -1,274 lines, 18 test-touching commits at
                      $12.8554 per test-touching commit, 23 docs, 0 reverts)

unattributed spend
  no repo match      $304.2035
  repo, no commits   $32.5890
  total              $336.7925 (60.0% of spend)
  zero-commit        36 of 45 sessions ($309.2405)

Two things worth reading out of these numbers rather than past them. First, the cache is doing enormous work: 98% of prompt tokens were cache reads, and without caching this month would have cost roughly $3,200 more. Second, the 60% unattributed number is the honest one: most sessions ran in the workspace root repo, which is deliberately not one of the five configured deliverable repos, and much of that spend was research, planning, and writing rather than commits to these repos (the four portfolio repos also had exactly one commit each at scan time, being days old). That is exactly the conversation this tool exists to make concrete; see HONESTY below before quoting any of it.

The full report for this run is checked in at docs/example-report.html.

Metric definitions

  • Cost per commit is attributed spend on a repo divided by commits authored in that repo during the observed period.
  • Cache hit rate is cache-read tokens as a share of all prompt tokens (cache read + fresh input + cache write).
  • Cache savings is what the cache-read tokens would have cost at the fresh-input price, minus what they cost at the cache-read price; the write premium is subtracted to get the net win.
  • Unattributed spend is usage whose working directory matches no configured repo, plus usage attributed to a repo in a window where that repo received no commits.
  • Zero-commit sessions are sessions none of whose (repo, window) cells contain a commit.

HONESTY

Read this before quoting any number from an Ingot report.

Correlation, not causation. Ingot joins spend and commits by directory and time window. It cannot see whether the tokens caused the commit, and it will happily attribute a hand-written commit to whatever agent session was open in that directory that day.

Commits are a proxy, not value. A one-line configuration fix can be worth more than a thousand-line refactor. Commit counts, insertions, and deletions measure motion, and motion is not the same thing as progress.

A zero-commit session is not a wasted session. Research, code review, debugging a production incident, and deciding not to build something are all legitimately commit-free. Ingot reports unattributed spend precisely so that this conversation can be had with numbers instead of vibes, not so the number can be read as waste.

Token efficiency can be gamed. The moment cost-per-commit becomes a target, it stops being a good measure: commits get smaller, work gets split, and the metric improves while nothing else does. Goodhart's law applies in full.

What this tool is actually for. Making the spend conversation concrete: which projects the tokens flow to, how much of the prompt bill the cache absorbs, what a shipped change costs in tokens this month versus last. It is a lens for budgets and workflows. It is not a performance review, and using it to rank people is a misuse of the tool.

Development

uv sync
uv run pytest
docker build -t ingot .

The test suite is fully offline: it uses small fixture transcripts and builds a throwaway git repo in a temp directory.

License

MIT

About

Token:deliverable analytics for AI coding agents - what did the tokens buy? Usage telemetry x git history, cost per shipped commit, cache economics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors