Skip to content

AviVAvi/TokenScope

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

TokenScope

A token profiler for Claude Code. It parses the session transcripts Claude Code already writes to disk (~/.claude/projects/**/*.jsonl) and tells you where your tokens actually went — and what to change to spend fewer of them.

ccusage tells you what you spent. tokenscope tells you why — and (soon) proves the fix worked.

Why a profiler, not an estimator

Predicting what a task will cost is guesswork — agent runs vary 3–5x on the same prompt. But every session's real usage is already recorded with per-request precision (input, output, cache reads/writes, per tool call). Measuring beats guessing, so v1 measures.

Usage

npx @asmitbohra/tokenscope            # overview of every project + top waste findings
npx @asmitbohra/tokenscope glow-os    # per-session breakdown for one project
npx @asmitbohra/tokenscope codemap .  # generate CODEMAP.md for a repo

Or install globally for the short command:

npm i -g @asmitbohra/tokenscope   # then: tokenscope, tokenscope <project>, tokenscope codemap .

No runtime dependencies; Node ≥ 18. From a clone, the TypeScript source also runs directly on Node ≥ 23.6:

# Overview of every project + top waste findings
node src/cli.ts

# Per-session breakdown for one project (name substring match)
node src/cli.ts glow-os

# Generate CODEMAP.md — a signatures-only map of a repo the agent can read
# instead of exploring whole files
node src/cli.ts codemap path/to/repo

The /tokenscope skill (judgment layer)

The CLI is deliberately deterministic — it measures, it never guesses. The skill adds the judgment layer: a Claude Code session that runs the profiler, reads your project's CLAUDE.md and recent prompts to understand the workflow, and then tells you which recommendations to apply and which ones are scoring deliberate cost as waste.

Install (copies into your personal skills directory):

cp -r skills/tokenscope ~/.claude/skills/tokenscope

Then in any Claude Code session: /tokenscope (or just ask for a token audit).

Tuning to your workflow (.tokenscope.json)

tokenscope scores context hygiene, not value — it has no model of your workflow and can't distinguish "expensive because wasteful" from "expensive because verification is the product". Drop a .tokenscope.json in a project root to teach it:

{
  "verification": true,
  "mute": ["split-sessions", "front-load-specs"]
}

"verification": true suppresses output-filtering advice (truncating output you're measuring manufactures false confidence). "mute" silences any recommendation by id.

What it reports

  • Per-project / per-session totals — requests, context tokens (input + cache read + cache write), output, and API-equivalent cost using real model pricing incl. cache multipliers (reads 0.1×, writes 1.25×/2×).
  • Context Score (0–100, A–F) — per-project hygiene grade from five measured signals: cache efficiency (30), error rate (20), re-read waste (20), compaction churn (15), MCP hygiene (15). High spend ≠ low score; only waste costs points. Each component also shows its approximate dollar impact, because hygiene points and dollars diverge — cache efficiency usually dominates real cost.
  • Structural detector — stats hot files on disk: a 5,000-line append-only doc that gets fully re-read every session is a structure problem (no cheap entry point), so it gets a head/archive-split recommendation with the measured math, not a "add summaries" platitude.
  • Recommendations — every finding maps to a concrete fix with your own numbers as evidence: paste-ready CLAUDE.md blocks (key-file summaries, environment notes classified from your actual error messages, output-filtering rules), claude mcp remove commands for servers you configured but never call, and session-hygiene advice.
  • Codemap — generates CODEMAP.md, a signatures-only skeleton of a repo (files, exports, functions, line anchors) that agents can consult instead of reading whole files.
  • MCP audit — cross-references servers in ~/.claude.json against tool calls in transcripts to find schema-tax with zero payoff.
  • Context by tool — how many tokens each tool (Read, Bash, Grep, MCP tools…) injected into context.
  • Waste findings:
    • repeated-read — same file read 3+ times in a session
    • huge-read — single file read worth 15k+ tokens
    • error-loop — 5+ failed tool calls (each retry re-bills the growing context)
    • verbose-commands — shell output flooding the context
    • low-cache-hit — sessions where prompt caching mostly missed
    • compaction-churn — sessions that outgrew the context window repeatedly

Accuracy notes

  • Token totals and cost come from the API's own usage objects in the transcript — exact.
  • Per-tool context sizes are estimated at ~4 chars/token from tool-result payloads — approximate, clearly a lower bound on their total cost (content re-bills on every subsequent request until cached/compacted).
  • "API-equivalent cost" is what the usage would bill via the API; subscription plans prepay this, so read it as value consumed, not a bill.

Layout

src/
  cli.ts        entry point / project discovery / commands
  parser.ts     JSONL transcript → structured session data
  analyzer.ts   metrics + waste-finding heuristics
  score.ts      Context Score (0-100, with $ impact per component)
  recommend.ts  findings → evidence-backed fixes (structure-aware)
  codemap.ts    CODEMAP.md generator (signatures-only repo map)
  mcp.ts        configured-vs-used MCP server audit
  report.ts     terminal rendering
  pricing.ts    model pricing + cache multipliers

Author

Built by Asmit Bohra

Found something wild in your own transcripts? Open an issue — real-world waste patterns shape the roadmap.

About

See where your Claude Code tokens actually go. Profiles the session logs Claude Code already writes, scores each project's context hygiene 0–100, and generates fixes backed by your own measured numbers. Local, zero deps, no telemetry.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Sponsor this project

 

Packages

 
 
 

Contributors