v2.10.2 — Close the Bash blind spot

A gap-closing release, motivated by a real user (me, on this user's repo) experiencing the product fail to deliver where they actually work.

What was broken

smart_read (v2.10) only intercepts the Read tool. In autonomous/agentic Claude Code sessions, Claude reaches for Bash to view files (cat/less/more/bat <bigfile>) — that bypassed smart_read entirely, the whole file dumped into context, and Receipts didn't see it. Documented honestly in v2.10.1's limitations.md, now actually closed.

What's new

smart_bash.py — PreToolUse hook on Bash. Intercepts the narrow, unambiguous case of a pure file dump and hands Claude the same outline smart_read does. The parser is the load-bearing part:

• intercepts exactly: cat/less/more/bat <file>, single positional, no flags, zero shell metacharacters in the entire command
• explicitly does not intercept: pipes, redirects, chains, substitutions, head/tail (their 10-line default is smaller than the outline — intercepting would be net-negative), sed, quoted args, multiple files, anything ambiguous
• once-per-file-per-session dedupe shared with smart_read so the same file isn't double-nagged
• slice events tagged via=bash for Receipts crediting
• env-disable: CONTEXT_OS_SMART_BASH=0

Why "perfection" required the narrow scope

The danger isn't missing an intercept — it's blocking a legitimate complex Bash command. So the parser's invariant is: any doubt → allow. The CI gate exercises 30+ adversarial shapes (pipes, &&, ;, $(), backticks, globs, tildes, quoted args, comments, line continuations…) and asserts every one passes through.

Tests

• python/evals/runners/savings_test.py now ~100 assertions (up from 49), covering: parser whitelist correctness, parser pass-through on every risky shape, end-to-end intercept/dedupe/disable, non-existent-file/small-file/non-Bash-tool guards.
• ranker_floor still green; setup.sh embeds byte-identical.

Honest scope

This closes the Bash blind spot for the safe whitelist. The genuinely-ambiguous cases (sed/awk/quoted scripts/multi-file) remain invisible to Receipts — that's the right call (false-positive blocking real Bash work would be worse than missing some coverage).

Install

curl -fsSL https://raw.githubusercontent.com/sravan27/context-os/main/setup.sh | bash

v2.10.1 — fix dropped-suggestions bug + show the compounding win honestly

A bug-fix + honesty release. Both came out of running context-os on its own sessions.

Fixed: /savings showed 0 searches avoided

auto_context logs the session id truncated to 12 chars; savings_tracker compared it against the full 36-char UUID, so every retrieval suggestion was silently dropped → 0 assisted hits, even though auto_context was firing on every prompt. (Slices used the correct comparison, which is why they showed up and searches didn't.) Fixed, with a regression test using a real UUID so it can't recur. The 39→41-assertion suite missed it because synthetic tests used short session ids where [:12] == full.

Added: the compounding win is now visible (and honestly labelled)

The headline tokens saved only counted the first load of a sliced file. It hid the bigger effect: that file body would have been re-sent on every turn until compaction. /savings now shows a separate "Budget freed" line, measured from the real transcript (kept-out tokens × turns the file persisted, capped at an inter-compaction window so one long autonomous session can't distort it).

Framed honestly: budget-freed is context-budget, not dollars — prompt caching makes re-sent content cheap, so it mostly means you hit compaction / the context limit later, not a 10× bill cut. The conservative first-load number stays the headline; milestone/$/streak ride that floor.

Also

• README + Show HN draft reframed: lead with the proven, version-independent claim (retrieval quality, CI-gated), and caption the −40.9% A/B as cold-cache --print, N=36 — not a warm-session bill saving.
• New: replay_history.py — validate the whole stack on your own Claude Code history, $0, no API key.

curl -fsSL https://raw.githubusercontent.com/sravan27/context-os/main/setup.sh | bash
python3 python/evals/runners/replay_history.py   # validate on your own transcripts

v2.10.0 — Structural slicing: read the slice, not the file

The compounding cost nobody attacks

auto_context kills first-turn exploration. v2.10 attacks the bigger, compounding cost: every file Claude reads is re-sent on every later turn until compaction. An 800-line file read at turn 3 is paid for ~40 more times — when Claude needed one 40-line function.

The assembly — one repo graph, now used for compression, not just retrieval

When Claude issues a whole-file Read of a big indexed file, smart_read.py (PreToolUse) intercepts it and hands back the file's outline — every symbol with its exact line range and signature, rendered straight from the graph with zero file content:

[context-os] `payment.py` is 847 lines. Read the slice you need:
  L12-45    class PaymentProcessor
  L47-89      def charge(self, amount, method)
  L340-410  def validate_card(number, cvv)
  … +28 symbols
  e.g. Read("payment.py", offset=47, limit=43) for the block at L47.

Claude re-reads offset=47, limit=43 — 43 lines, not 847. The body never enters context, so it's never re-sent. Unlike the first-turn saving, this one compounds across every remaining turn.

• Fires once per file per session (no loop, no nag), only above a line threshold, only when the graph has structure to slice — otherwise falls through to the size guard.
• /outline <file> renders the same map on demand.
• CONTEXT_OS_SMART_READ=0 to disable; CONTEXT_OS_SMART_READ_MIN to tune.

Measured, and folded into Receipts

build_repo_graph.py now stores each symbol's line range + signature. Every interception is logged; /savings now shows two measured sources:

  Searches avoided   412   (auto_context → straight to file)
  Big reads sliced    96   (smart_read → outline, not whole file)

  Where it came from
  Avoided searches    1,402,000 tok
  Sliced big reads      938,000 tok

Per-slice credit is conservative — whole-file tokens minus outline tokens — and ignores the larger avoided per-turn re-sends, so it under-claims.

Why it's novel

A repo graph used not for retrieval but for structural compression of reads. Retrieval (skip exploration) + slicing (read less) + causal measurement of both, on one graph — an assembly no other Claude Code tool ships.

Rigor

savings_test.py — 39 assertions incl. outline rendering, line-range slicing, once-per-session dedupe, sliced-read pass-through, disable flag, slice→receipts crediting, two-source receipts. ranker_floor green. All 5 setup.sh-embedded hook copies byte-identical to canonical.

8 Python hooks · 13 slash commands · 30 techniques.

curl -fsSL https://raw.githubusercontent.com/sravan27/context-os/main/setup.sh | bash

v2.9.3 — Receipts, measured (not estimated)

The upgrade: measurement, not estimation

v2.9.2 credited a flat 8k tokens per hit. A sharp reviewer's first objection — "that number is made up." So it no longer is.

savings_tracker.py now reconstructs each prompt's first turn from the transcript and classifies it:

• assisted — Claude's first tool action was a Read of a file auto_context surfaced, with no Glob/Grep first. An exploration replaced.
• explored — it searched before finding the target. The exploration's token cost is read directly off the real tool_result sizes in your transcript.

Each avoided search is credited the measured average cost of an exploration in that same session — not a constant:

[context-os] receipt: 1 prompt went straight to the right file → ~5,000 tokens
saved (a search cost ~5,000 tok here, measured across 1 that still explored).

$ /savings
  Searches avoided            412   (opened the right file with no Glob/Grep)
  How it's measured
  A search cost        14,200 tokens on average — measured
                       from 287 of your own prompts that still explored
  100% of the savings above is measured this way.

Why it matters

This is the in-product, per-user version of the live A/B: not "trust our benchmark," but "here's what it saved you, measured on your repo." The per-search credit is clamped ≤15k (below the 21k aggregate the A/B measured) and sessions with nothing to measure fall back to a clearly-labelled 8k estimate — so the total under-claims, never over-claims.

Rigor

• python/evals/runners/savings_test.py — 29 assertions: causal assisted-vs-explored classification, measured-vs-estimate per-hit selection, exploration-cost measurement from real tool_result sizes, session isolation, milestone, streak, fail-open on garbage/missing transcript, report rendering.
• Wired into CI. ranker_floor still green — auto_context retrieval untouched.
• setup.sh's embedded hook is byte-identical to the canonical source.

Install

curl -fsSL https://raw.githubusercontent.com/sravan27/context-os/main/setup.sh | bash

v2.9.2 — Receipts: make the savings visible

The addiction loop

auto_context saves ~40% of first-turn tokens — but silently. Invisible wins build no habit and earn no word-of-mouth. v2.9 closes the loop the way GitHub's contribution graph and Spotify Wrapped did: by making an accumulating quantity visible.

New surfaces

• savings_tracker.py (Stop hook) — reads the transcript for files Claude opened, joins them against the suggestions auto_context logged that session, and credits each match as a hit (a Glob/Grep exploration the agent skipped). Maintains .context-os/savings/{ledger.jsonl,total.json}. Prints a one-line receipt at session end + a celebration when the running total crosses a milestone.
• /savings — dashboard: all-time saved, this week, hit-rate, day-streak, dollar value, rate-limit runway ("~47 prompts before the window"), and a copy-paste shareable card.
• statusLine meter — live 💰 2.3M saved · 5d🔥, reading the cached total. The ambient trigger.
• auto_context.py logs each suggestion to suggestions.jsonl — data we control, robust against transcript-format churn.

$ /savings
  All-time saved      2,340,000 tokens  (~$14.04)
  This week             612,000 tokens  (~$3.67)
  Runway bought            ~47 prompts before the rate window
  Hits                        412  (files context-os surfaced that you opened)
  Hit-rate                    71%  ███████████████████░░░░░
  Streak                        5  consecutive days 🔥
  ╭─────────────────────────────────────────────╮
  │            context-os · receipts            │
  │  2,340,000 tokens saved                     │
  │  412 hits over 38 sessions                  │
  ╰─────────────────────────────────────────────╯

Honesty

A hit is conservative — a suggested file that was also Read, counted once per session. tokens-saved defaults to 8k/hit, under half the ~21k measured in the live A/B, so the running total under-claims. CONTEXT_OS_SAVINGS_PER_HIT tunes it; CONTEXT_OS_SAVINGS=0 disables. Local-only, no phone-home.

Quality

python/evals/runners/savings_test.py — 17 assertions (hit detection, no over-count, session isolation, path normalization, milestone crossing, streak math, fail-open on garbage/missing transcript, empty-state report). Wired into CI. ranker_floor still green — auto_context retrieval unaffected.

Install

curl -fsSL https://raw.githubusercontent.com/sravan27/context-os/main/setup.sh | bash

Adds the Stop hook, the /savings command, the report backend, and the statusLine meter. 7 Python hooks, 12 slash commands, 29 techniques.

v2.9.1 — AI agent cost leak audit slots

AI agent cost-leak audit funnel

This release tightens the private-repo audit path around the existing context-os GitHub Action and local checker.

• Free local check: python3 python/agent_cost_leak_check.py --repo . --json
• Audit scope: https://sravan27.github.io/money-27-proof/agent-cost-leak-audit.html
• Two paid 48-hour audit slots are open this week for teams already spending real money on Claude Code, Codex, Cursor, or internal coding agents.

The OSS hook and Action stay MIT/free. The paid audit is for private repo scorecards, workflow/context-loop diagnosis, a CI leak gate where useful, and one concrete repo/workflow patch.

v2.9.0 — agent cost leak GitHub Action

What's new

v2.9.0 turns the agent cost leak checker into a versioned GitHub Action that teams can drop into CI:

- uses: sravan27/context-os@v2.9.0
  with:
    max-score: "40"

The checker scans repo shape for coding-agent cost-leak signals before Claude Code, Codex, Cursor, or internal agents waste context on generated files, missing repo guidance, large tracked blobs, or unclear validation paths.

Included

• Root composite Action: action.yml
• Local stdlib checker: python/agent_cost_leak_check.py
• CI threshold mode: --max-score
• GitHub Step Summary output for Action runs
• Action smoke test in the repo CI
• Private-repo audit path for teams that want a 48-hour report and concrete fix path

Try it locally

python3 python/agent_cost_leak_check.py --repo . --max-score 40

Paid audit page: https://sravan27.github.io/money-27-proof/agent-cost-leak-audit.html

v2.8.0 — cross-repo generalization (3 OSS repos, 36 prompts, +18.2% aggregate MRR)

What's new

The v2.7 claim "beats every lexical baseline" rested on one dogfood repo. v2.8 proves it generalizes to three real OSS repos that are not in our fixture set.

Cross-repo evidence (36 hand-labeled prompts, pinned SHAs)

Repo (lang, files)	auto_context MRR	Best baseline	Δ
axios/axios (JS, 214)	0.382	bm25-path 0.252	+0.130
BurntSushi/ripgrep (Rust, 100)	0.503	bm25-path 0.459	+0.044
psf/requests (Py, 36)	0.750	bm25-symbols 0.875	−0.125

Weighted aggregate across 36 prompts: auto_context 0.545 vs best baseline 0.461 → +18.2%.

The single loss (psf/requests) is honest: prompts in that set use exact class names (PreparedRequest, HTTPError, CaseInsensitiveDict), which is the lexical-retrieval ceiling regime where bm25-symbols caps. We win the cross-repo aggregate, in every language, but don't pretend to win every repo.

Ranker improvements (net-positive, kept)

• plural ↔ singular stem variants in extract_tokens()
• case-fold dedupe of path tokens (no triple-counting Request/request/requests)
• df-discriminativity scaling: disc = 1 - df/N for path_substr bonus
• file-level score aggregation: sum candidate scores per file, pick best-scoring line as representative

Tried and reverted (net-negative on synthetic)

• symbol_part (token matches camel/snake segment of symbol)
• symbol_phrase (compound-symbol substring in normalized prompt)

Quality gates

• Synthetic MRR 0.984 (was 0.969 in v2.7) · P@3 0.698
• Dogfood MRR 0.756 · top-1 0.600 · +0.142 over bm25-symbols
• All 9 ranker_floor regression gates green
• New multi_repo_eval.py exits non-zero if (a) weighted aggregate fails to beat every baseline or (b) any repo falls below the avg of the five baselines

Reproduce

git clone https://github.com/sravan27/context-os && cd context-os
python3 python/evals/runners/ranker_floor.py        # 9 hard gates
python3 python/evals/runners/multi_repo_eval.py     # 3 OSS repos, 36 prompts

First run of multi_repo_eval.py clones the three pinned repos to ~/.cache/context-os-multi-repo/ (~30MB total). Re-runs use the cache.

Files

• python/evals/runners/multi_repo_eval.py — runner, dual acceptance criterion
• python/evals/multi_repo_prompts/{axios,ripgrep,requests}.json — 36 hand-labeled prompts
• python/evals/reports/multi-repo-eval.md — full report (per-repo tables, weighted aggregate, per-prompt detail)
• hooks/python/auto_context.py — ranker improvements
• docs/PITCH.md, docs/FOR-CLAUDE-CODE-TEAM.md, docs/PROPOSAL.md, docs/REVIEW-CHECKLIST.md — updated with v2.8 cross-repo evidence

v2.7.0 — Pitch-ready: ranker regression gate, 50k latency, ROI math

TL;DR

Acquisition-hardening release. Same ranker that got the −40.9% live-Claude result; harder evidence and a CI gate that prevents silent regression.

Three new docs for the Claude Code team (in order, ~30 min total):

• docs/PITCH.md — 5-min leadership read
• docs/REVIEW-CHECKLIST.md — 20-min engineer walkthrough
• docs/SECURITY.md — enterprise/privacy review

What's new

Ranker regression gate (CI-enforced)

python/evals/runners/ranker_floor.py asserts 9 hard floors across synthetic / dogfood / baseline margins. Any PR that regresses retrieval quality red-lines the build.

Gate	Floor	Current
synthetic MRR	0.920	0.969
synthetic P@3	0.640	0.703
dogfood MRR	0.720	0.789
dogfood top-1	0.580	0.667
MRR lift over bm25-symbols (synth)	+0.060	+0.094
MRR lift over naive-filename	+0.300	+0.406
MRR lift over random	+0.500	+0.562
dogfood MRR lift over bm25-symbols	+0.080	+0.208

50k-file latency measurement

After path_df precomputation in the graph schema (v1→v2), the hook is O(tokens) per query, not O(files × tokens). Scale measurements:

Files	p99 (v2.6)	p99 (v2.7)	SLA
100	25ms	23ms	1000ms
1,000	45ms	41ms	1000ms
10,000	175ms	118ms	1000ms
25,000	420ms	284ms	1000ms
50,000	830ms	589ms	1000ms (1.7× under)

ROI math

Conservative back-of-envelope (1M users × 400 prompts/month × 8.2K tokens saved × $6/1M tokens):

	Without auto_context	With	Delta
Tokens/user/month	20M	11.8M	−8.2M
Cost/user/month	$120	$71	−$49
Across 1M users/year	—	—	~$588M

90% discounted for cache reuse / cohort overlap / power-user skew is still low-nine-figures/year.

Retrieval (held across the rebuild)

• Synthetic MRR 0.969 · P@3 0.703 (unchanged)
• Dogfood MRR 0.789 · top-1 0.667 (was 0.800; 49→50 files shifted path_df; honest number)
• Beats every lexical baseline on real-repo prompts

CI

.github/workflows/ci.yml now runs ranker_floor.py as a hard gate after the dogfood step. Every PR regenerates every report; nothing ships without the floor holding.

No install action needed

v1 graphs are auto-detected; path_df is recomputed on first load. No rebuild required for existing users. curl -fsSL https://raw.githubusercontent.com/sravan27/context-os/main/setup.sh | bash pulls 2.7.0.

Reproduce everything in 5 minutes

git clone https://github.com/sravan27/context-os && cd context-os
python3 python/evals/runners/ranker_floor.py        # 9 hard gates, ~45s
python3 python/evals/runners/autocontext_eval.py    # MRR 0.969 · P@3 0.703
python3 python/evals/runners/dogfood_eval.py        # real-repo MRR 0.789
python3 python/evals/runners/robustness_test.py     # 18/18 adversarial
python3 python/evals/runners/latency_bench.py       # p99 SLA

Every number in the pitch is the output of one of those scripts. Nothing hand-edited. The green CI badge is the contract.

Contact

Email sridharsravan@icloud.com. We have code, time, and a strong bias for shipping. Happy to walk the upstream port with the Claude Code team.

v2.6.0 — beats every lexical baseline on its own repo

TL;DR

auto_context — a ~400-line stdlib Python hook that injects file:line · symbol · imports candidates into the first Claude Code turn — now beats every lexical baseline on its own repo.

Live Claude A/B (unchanged from v2.5, re-validated)

• −40.9% aggregate tokens on 36 real claude --print calls, 95% bootstrap CI [32.7%, 48.9%]
• 6/6 prompt-level wins, 16/18 per-run wins (Wilson CI [67.2%, 96.9%])
• Paired t-test p = 5.06e-07, Cohen's d = 1.84 (large effect)
• Wall-clock −35.3% (mean 11.80s → 7.64s)

Offline retrieval (Python/TS/Rust fixtures, 32 hand-labeled prompts)

• MRR 0.969 · P@3 0.703 · Coverage 1.000
• +0.094 MRR over BM25-symbols (textbook lexical baseline)
• +0.407 MRR over naive-filename

Dogfood on this repo (real heterogeneous codebase, 49 src, 440 symbols)

Method	MRR	Top-1	P@3
auto_context	0.800	0.667	0.322
bm25-symbols	0.619	0.533	0.244
bm25-path	0.536	0.467	0.256
naive-filename	0.483	0.400	0.322
grep-count	0.283	0.133	0.111
random	0.061	0.000	0.000

+0.181 MRR over BM25-symbols on real-repo prompts — not just synthetic fixtures.

Operational

• p99 hook latency: 23ms @ 100 files, 46ms @ 1k, 173ms @ 10k (5× under 1s SLA)
• 18/18 adversarial robustness cases pass (unicode, 100k prompts, null bytes, corrupt graph, regex bombs, shell meta, path injection)
• Ablation: path_substr (ΔMRR −0.062) + path_exact (ΔMRR −0.016) load-bearing. No dead weight.

What changed vs v2.5

Ranker upgrades (hooks/python/auto_context.py):

1. IDF-weighted symbol + path matches, dampened & capped at 1.6
2. Basename-in-prompt detection across under/space/none normalizations (+15)
3. Multi-token path coverage bonus (+2 per extra token, cap +8)
4. NL → code term expansion (~20 curated mappings)
5. Graph-aware stopwords: promote test/file/find/... back to tokens when the graph has files named that way

CI: 5 new eval steps (ablation, robustness, latency, baseline-comparison, dogfood) now run on every PR.

Installer: setup.sh heredoc synced — installed users get the upgraded ranker immediately.

Install

curl -fsSL https://raw.githubusercontent.com/sravan27/context-os/main/setup.sh | bash

All evidence is reproducible: cd python/evals && ls runners/ → every number in this release has a Python script behind it.