mem stats

mem-stats measures what your agent sessions really consume, using the logs the client already writes on your machine: Claude Code transcripts and Codex rollouts. It records tokens per request, what fills the context (MEM included), where the prompt cache was rebuilt and why, compactions and fresh starts, per project and for the whole machine. Everything goes into one local SQLite ledger per user that holds counts, never content; it makes no model calls and sends nothing anywhere.

Version	0.1.1
Commands	MEM STATS with subcommands SETUP, STATUS, RESET, REPORT, WATCH, INGEST, PRUNE, INIT, INVENTORY, READ
Executable content	Yes: five Python runners (stats.py, claude_code.py, codex.py, hook_ingest.py, setup.py)
External actions	No
Requirements	Python 3.11+ (standard library and SQLite). Python 3.14+ to read Codex rollouts older than a week. Optional: tiktoken for exact token counts

Why it exists

An agent re-sends the whole conversation with every request. What a session costs is therefore mostly what it carries, not what it adds: every page read, every tool result and every tool definition is paid again on each later request, usually as cache reads, and sometimes as a full cache rewrite after a pause. None of this is visible from the chat window.

mem-stats makes it visible, so you can answer practical questions with numbers instead of impressions: How much of my input is MEM? Which knowledge base pages are the heaviest? Which tool definitions am I paying for and never using? How often did the cache get rebuilt, and why? Which project is the expensive one? It is also the measuring base that mem-optimize builds on.

How it works

One ledger per user

There is a single ledger per user, covering every project on the machine. Each request records the project it belongs to (the folder its session started in), so REPORT --project PATH gives any project's figures. A per-project ledger would add nothing, so the setup wizard does not even ask about scope.

Automatic collection

A setup wizard installs a small hook in the client: Stop and SessionEnd for Claude Code (SessionEnd also covers /clear and exit), Stop for Codex. At the end of every agent turn the hook queues the session's transcript or rollout and returns at once, printing nothing. Nothing reaches the model, so collection costs no tokens and never makes a turn fail.

A background worker then does the work:

• Only one worker runs per ledger, under a lock. A lock left behind by a worker that died is taken over after 30 minutes.
• It waits a few seconds so the client can finish writing the turn, then runs the right adapter on each queued file (for Claude Code, subagent transcripts included).
• Several sessions open at once simply queue. The ledger is SQLite in WAL mode with deterministic event ids, so concurrent writers never duplicate an event.
• A queued file leaves the queue only after its import succeeded. A failed one is retried by the next worker runs, at most three times, then dropped with a line in the run log.
• Every process starts without a visible window.

History backfill

At setup you can import your history: every past session of the chosen clients on this machine is read once, in the foreground. On a long history this can take a few minutes. The backfill takes the same lock as the worker, waiting for a running one first, so turns that end meanwhile just queue.

Imports are incremental, which keeps the per-turn cost small even on very large transcripts: a request whose events the ledger already holds is skipped, as long as they were computed with the same knowledge bases and the same token counting. A run with other knowledge bases or another tokenizer recomputes them; --full recomputes everything.

Re-ingesting is safe. A measured usage event that arrives again with the same counts replaces the stored one; with different counts it is reported as a conflict and skipped, never an abort. Derived data is replaced by the latest run, except that a run with less context (fewer knowledge bases found) never overwrites richer data.

Counts, never content

The ledger stores a closed set of event kinds (usage, attribution, composition, breakdown, overhead, lifecycle, coverage, metric, delivery, inventory), and each kind accepts a closed set of fields. An unknown field is rejected, identifiers are short and contain no spaces, paths contain no control characters. The ledger therefore cannot hold file contents, prompts, answers, credentials or tool output. What it holds: identifiers, token counts, modes, hashes, KB page paths, tool names and project folders.

MEM attribution

For Claude Code sessions, mem-stats estimates how much of your input is MEM: the knowledge base pages read, the MEM pointer in the client's instruction file, and each extension separately. It recognizes a knowledge base as a folder holding both MEM.md and MEM.config.md; the hook and the backfill look for one under each session's own project folder. The estimate is sized on the measured usage of each request, and follows messages preserved across a compaction into the next requests. The report gives MEM's share both against total input and against non-MEM input, so you can see what the knowledge base costs relative to everything else in the conversation.

Reports

A report is built from the ledger on demand, as readable Markdown (--format text) or JSON (the default). You can choose sections and filter by project, period or session:

Section	What it shows
provider_usage, money	Tokens by bucket (input, cache reads, 5-minute and 1-hour cache writes, output) per client and model; API-equivalent cost with a price card. Unknown is null, never zero
projects	Per project: sessions, requests, tokens, cost, MEM share
mem_attribution	MEM's share of input (Claude Code)
context_composition	Prompt tokens by source (your messages, assistant output, attachments, each tool, each MCP server), new versus carried
fixed_overhead	System prompt, tool definitions and listings carried by every request; per tool, its weight, calls per 1,000 requests and last call; the tools never or rarely called (at most once per 1,000 requests)
kb_pages	The knowledge base pages that cost most
cache_breaks	Requests that rewrote at least half of a 10k+ prompt, by likely cause (idle time, model change, context reduced), with the cost of the rewrite
costliest	Top sources, pages and overhead items in one place
lifecycle	Compactions (with context before and after) and /clear fresh starts
subagents	Delegated runs by agent type
coverage	What the adapters could not account for: pending requests, anomalies, untested client versions, skipped rollouts, conflicts
extension_metrics	Numeric metrics reported by other extensions

Other sections (extensions, payload, inventory_by_session) cover installed extensions and instrumented reads. MEM STATS WATCH renders the same report again whenever the ledger changes, as a live local view. A report costs tokens only when you put it into a conversation.

Installing and first setup

Install it from the extension catalogue, or accept the agent's offer when you first type a MEM STATS command. Because it ships code, installing it needs extensions_allow_executable_content: true (see below).

Installing collects nothing by itself. Run MEM STATS SETUP afterwards: the wizard is what starts collection. It follows a plan, confirm, apply pattern and writes nothing until you confirm:

1. Clients: Claude Code, Codex, or both. Those installed on the machine are pre-selected. For Codex, the wizard tells you that a new or changed hook runs only after you approve it with /hooks in the CLI or in the app's hook review.
2. Import the history: yes (recommended) or no.
3. Plan: the agent shows you the full plan, every settings file, hook command and copy.
4. Apply: after one confirmation, setup copies the runners to a versioned folder in your home directory, creates the ledger, backs up each settings file, adds the hooks, and runs the backfill if you chose it.

Run MEM STATS SETUP again to change your answers, and after updating the extension, so the copied runners are refreshed. A second run replaces the hook entries instead of duplicating them.

Commands

Command	What it does
MEM STATS SETUP	Setup wizard: questions, full plan, one confirmation. Installs the hook, creates the ledger, imports the history
MEM STATS STATUS	Hooks present per client, ledger path and size, last ingestion and last error, lock and queue. Read-only
MEM STATS RESET	Removes the hooks mem-stats added, and only those. The ledger stays unless you explicitly ask to delete it
MEM STATS REPORT	The report, as text or JSON; filter with --project PATH, --since/--until, --session; pick --sections; --top N for ranked lists; --prices CARD for costs
MEM STATS WATCH	The report, refreshed whenever the ledger changes (--interval, default 2 seconds)
MEM STATS INGEST	Import by hand: a whole machine's history, one workspace, one transcript or rollout
MEM STATS PRUNE	Deletes whole sessions last active before a date, never half a session
MEM STATS INIT	Creates the ledger
MEM STATS INVENTORY	Records a snapshot of a knowledge base's extensions
MEM STATS READ	An instrumented full read of a knowledge base file

Configuration

There are no extensions_mem_stats_* keys in MEM.config.md. The choices are the wizard's answers:

Answer	Default	Meaning
clients	Claude Code	Which clients get the hook: claude, codex
backfill	true	Import every existing session of the chosen clients once, at setup
ledger	~/.mem/mem-stats/state/stats.sqlite	Advanced: another ledger path
python	The interpreter running setup	Interpreter the hook runs with
adapter_python	Same as python	Interpreter for the adapters, for example a Python 3.14 to read old Codex rollouts

Price cards are optional. The extension ships an Anthropic API card; you can add your own cards under the extension's custom/ folder. Each request is priced with the matching card that was valid at its date. Under a subscription the totals value your usage at API rates: they are a valuation, not what you paid.

Safety and permissions

• Executable content gate. Every subcommand runs code. If extensions_allow_executable_content is not true in MEM.config.md (absent means false), the agent must not run the runners. It tells you the setting is off, says plainly that mem-stats is installed but does nothing (nothing collected, no report, no wizard), and asks whether to enable the setting, naming the five runners and noting that the setting covers every installed extension with executable content. It changes MEM.config.md only after an explicit yes; on a no it runs nothing and does not ask again in that session.
• Settings are written only after you confirm the plan. Setup backs up each settings file first and records what it added.
• It only touches its own hooks. Hook entries are recognized by the runner name, so other tools' hooks, mem-optimize's included, are never modified, and RESET removes only what mem-stats added.
• No network, no model calls. Local collection sends nothing anywhere.
• No content. The ledger's closed schema rejects anything that is not a count, an identifier or a path; the run log records times, exit codes and counts only.
• Codex hook trust. Codex runs a new or changed hook only after you approve it with /hooks (CLI) or in the app's hook review. Until then, nothing is collected from Codex.

Where its files and data live

The ledger is outside your repository, so measurements never end up in a commit.

Location	Content
~/.mem/mem-stats/<version>/runner/	Copy of the runners used by the hooks
~/.mem/mem-stats/state/stats.sqlite	The ledger
~/.mem/mem-stats/state/ingest.jsonl	Run log: times, exit codes, counts, never content
~/.mem/mem-stats/state/	Also the ingestion lock and queue, next to the ledger
~/.mem/mem-stats/setup-log.json, backups/<stamp>/	What setup added, and the settings files as they were
~/.claude/settings.json	Claude Code hooks (Stop, SessionEnd)
~/.codex/hooks.json	Codex hook (Stop)
extensions/mem-stats/ in the knowledge base	The extension itself; custom/ for your own price cards, preserved by updates

Limits and known issues

• MEM attribution, context composition and KB pages are Claude Code only. For Codex the extension records usage (reasoning tokens kept apart, duplicate responses on resume counted once), lifecycle and coverage.
• Codex in the VS Code extension on Windows runs no hooks while openai/codex#33413 is open. There, ingest by hand with MEM STATS INGEST.
• Old Codex rollouts need Python 3.14+. Codex recompresses rollouts older than a week to .jsonl.zst, and only Python 3.14's standard library reads zstd. With an older Python they are skipped and counted under coverage; set adapter_python to a 3.14 interpreter to include them.
• Sizes are estimates without tiktoken. Without it, sizes are labelled as character estimates.
• Some Claude Code output counts are lower bounds. A request that never settled (mostly subagent requests whose transcript kept only early streaming usage) is kept and flagged; its output is a minimum.
• Client versions the adapters were not tested against are still parsed if well-formed, and reported under coverage.
• Costs are API-equivalent valuations, not invoices, and require a matching price card; unpriced requests are counted by reason.

See also

• Specification: https://github.com/Ryadel/MEM/blob/main/src/extensions/mem-stats/index.md
• Extensions
• mem-optimize, which acts on what mem-stats measures. mem-stats does not depend on it; some mem-optimize features need mem-stats