context-guru v0.1.0 — First Public Release

Provider-agnostic context engineering for LLM agents.
Shrink the tokens every request carries — losslessly, or lossy-but-reversibly — without touching the agent.

What is context-guru?

context-guru is a single Go core that reduces the token cost of LLM-agent traffic.
The same core runs as an HTTP proxy/gateway (drop-in, any language, zero agent changes) or as an in-process plugin inside larger platforms (AuthBridge, bifrost).

It operates on the messages array — dropping redundant tool output, collapsing superseded runs, projecting large reads down to what's relevant — and every reduction is safe by construction:

Fail open, always — any component error or panic reverts that component only; the original request is always a valid fallback.
Never worse — a component that would grow a message is reverted. You never pay to compact.
Reversible — every lossy drop leaves a <<cg:HASH>> marker and stashes the original, recoverable via a model-callable context_guru_expand tool or GET /expand.

Benchmark — the cheapest & highest-reward arm on SWE-bench Verified

Evaluated live, end-to-end, with the claude-code agent on aws/claude-sonnet-5, against a no-compaction baseline, headroom (request-stream proxy), and rtk (Rust Token Killer, shell-level Bash-output hook). All 50 tasks scored under all four arms.

dimension	baseline	context-guru	headroom	rtk
tasks solved	43 / 50	44 / 50	40 / 50	43 / 50
total billed cost	$31.98	$27.77 (−13.2%)	$30.30 (−5.3%)	$29.09 (−9.0%)
cache-read tokens	102.8M	84.5M (−17.8%)	96.4M (−6.3%)	91.7M (−10.8%)
cache-write tokens	1.855M	1.847M (−0.4%)	1.839M (−0.9%)	1.835M (−1.1%)
mean steps / task	36.1	31.1 (−13.9%)	35.1 (−2.8%)	33.2 (−8.0%)
added latency / req	—	117 ms	63 ms	0 ms
tool LLM cost	$0	$0.31	$0	$0

context-guru is the cheapest arm and solves the most tasks. It cuts billed cost 13.2% vs no compaction, driven by a 17.8% cache-read reduction, by freezing each compaction and replaying it byte-identically every turn so the saving compounds across the session's cache-reads while never mutating the cached prefix.

Full four-way study, per-task/per-component breakdowns, real before→after examples, and reproduction instructions: docs/RESULTS.md.

Architecture

Agent → [Host Adapter]
             ↓
         apply.Body  (wire normalization, Anthropic tool_result expansion)
             ↓
         Pipeline  (ordered components, fail-open, never-worse)
             ↓
         Upstream Provider
             ↓
         Expand Loop  (resolve <<cg:HASH>> markers from Store if model calls context_guru_expand)
             ↓
         Agent

One core, three integration points — proxy, AuthBridge in-process plugin, bifrost LLMPlugin — all running the same components package.

Shared state:

Store (store/) — TTL+LRU in-memory session state (rewind slot for original stash, sticky slot for byte-stable cross-turn re-application). Default: 1800 s TTL, 1000 entries.
Session (session/) — resolved from x-context-guru-session header, or falls back to sha256(system_prompt + first_user_message)[:16] for zero-cooperation keying.

The Pipeline — 13 Components

Components implement one of two lossiness-typed interfaces: Reformat (lossless) or Offload (lossy, must stash originals and return cache keys to prove reversibility).

Reformat — lossless repack

Component	What it does
`format`	Re-encodes pretty-printed JSON tool output as compact JSON
`toon`	Re-encodes a uniform JSON array as TOON (header once, one row per item)
`cacheinject`	Adds an Anthropic `cache_control` breakpoint on a stable prefix boundary

Offload — drop bytes, stash the original, leave a recoverable marker

Component	What it does
`dedup`	Replaces a byte-identical earlier tool output with a pointer
`failed_run`	Collapses superseded test/build runs, keeps the latest in full
`cmdfilter`	Shrinks structured command output (pytest, npm, make, …) via a declarative DSL
`extract`	Deterministic noise collapse — repeated lines, blank runs, progress bars
`extract_llm`	A cheap model writes a sandboxed Starlark filter that trims to what's relevant
`collapse`	Head/tail window on any oversized output — last-resort fallback
`mask`	Age-based GC — keeps the newest N tool outputs, stashes older ones
`skeleton`	Replaces code-block function bodies with `{ … }`, keeps signatures (needs `cg_skeleton` build tag, tree-sitter)
`smartcrush`	Keeps anchor items of a long JSON array, drops the middle
`summarize`	Compresses the middle of the trajectory into one LLM-written summary (run alone)

Configuration Presets

Named presets cover the most common use cases. Pick one via --preset or PRESET env; fine-tune with a full YAML config (--config/CONFIG).

Preset	Pipeline	Use
`codesmart` (default)	`format, dedup, failed_run, cmdfilter, extract_llm, extract, cacheinject`	SWE-bench-winning config; LLM pass no-ops without `CHEAP_MODEL`
`codesafe`	`format, dedup, failed_run, cmdfilter, extract, collapse, cacheinject`	Same as `codesmart`, deterministic-only, zero model calls
`balanced`	`format, dedup, failed_run, cmdfilter, cacheinject`	Conservative — no noise extraction
`safe`	`format, cacheinject`	Lossless only
`agent`	`format, dedup, failed_run, mask, extract, cacheinject`	Long agentic sessions
`coding`	`format, skeleton, cmdfilter, cacheinject`	Code-reading agents (needs `cg_skeleton`)
`mcp`	`format, smartcrush, cacheinject`	MCP / JSON-array-heavy workloads
`summarize`	`summarize`	Trajectory compression — run alone
`off`	(passthrough)	A/B baseline control arm

Key Flags & Environment Variables

Flag / Env	Default	Purpose
`--preset` / `PRESET`	`codesmart`	Pipeline preset when no `--config`
`--config` / `CONFIG`	—	Full YAML config (overrides preset)
`LISTEN_ADDR`	`:4000`	Proxy listen address
`--anthropic-upstream` / `ANTHROPIC_UPSTREAM`	`https://api.anthropic.com`	Anthropic backend
`--openai-upstream` / `OPENAI_UPSTREAM`	`https://api.openai.com`	OpenAI backend
`ANTHROPIC_API_KEY` / `OPENAI_API_KEY`	—	Real key injected on forward (gateway mode); empty = pass client auth through
`CHEAP_MODEL` (+`CHEAP_MODEL_*`)	—	Dedicated cheap model for `extract_llm` / `summarize`
`FORCE_MODEL`	—	Pin all requests to one model (eval-containers `EVAL_MODEL`)
`INJECT_EXPAND`	`auto`	Tool injection: `auto` / `always` / `never`

Per-request headers:

Header	Effect
`x-context-guru-session: <id>`	Explicit session key
`x-context-guru-bypass: true`	Skip pipeline (A/B baseline)
`x-context-guru-pipeline: <names>`	Per-request pipeline override (comma-separated)

HTTP Routes

Endpoint	Purpose
`POST /anthropic/v1/messages`	Anthropic Messages API proxy
`POST /openai/v1/chat/completions`	OpenAI Chat Completions proxy
`GET /healthz`	Health check
`GET /stats`	Token-weighted savings rollup (JSON)
`GET /expand?id=<hash>`	Recover a stashed original by marker hash

Integration Options

Option	Description	Where
Proxy / gateway	`context-guru-proxy` in front of the provider; includes eval-containers gateway image	`proxy/`, `cmd/context-guru-proxy/`
In-process plugin	AuthBridge (Rossoctl sidecar) plugin importing this module, running the same pipeline on `pctx.Body`	Plugin lives in `cortex`; reuses `apply.Body` + `expand/`
bifrost LLMPlugin	Run the pipeline as a `PreRequestHook` inside any bifrost deployment	`adapters/bifrost/`

Quickstart (60 seconds)

Requires Go 1.26 and a C toolchain (CGO_ENABLED=1).

# Build
CGO_ENABLED=1 go build -tags cg_skeleton -o bin/context-guru-proxy ./cmd/context-guru-proxy

# Run (ships with the SWE-bench-winning config by default)
./bin/context-guru-proxy                          # listens on :4000

# Point any agent at it — one port serves both API dialects
export ANTHROPIC_BASE_URL=http://localhost:4000/anthropic
export OPENAI_BASE_URL=http://localhost:4000/openai/v1
claude                                            # or any OpenAI-compatible agent

# Watch the savings
curl -s localhost:4000/stats | jq

Or pull the Docker image:

docker build -t context-guru:local .
docker run -p 4000:4000 \
  -e ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
  context-guru:local

Omit -tags cg_skeleton for a pure-Go build (no CGO required); the skeleton component is simply inert, everything else works.

Technical Stack

Language: Go 1.26
Module: github.com/rossoctl/context-guru
Key dependencies: bifrost core (provider-agnostic chat schema), tree-sitter + language grammars (code parsing), gjson/sjson (byte-lossless JSON splicing), Starlark (sandboxed LLM-written filters), tiktoken-go (token counting)
Build tags: cg_skeleton — enables tree-sitter binding for the skeleton component (optional)
Version stamping: -ldflags "-X .../buildinfo.Version=v0.1.0 -X .../buildinfo.Commit=<sha>"

Documentation

Architecture & design — component model, fail-open pipeline, store, session, expand loop, metrics
All components — how every component works, live before→after, config options
Benchmark results — four-way SWE-bench Verified study with per-task and per-component breakdowns
Integrations — proxy gateway vs AuthBridge plugin, request path diagrams
Setup & deployment — build, Docker, eval-containers gateway, SWE-bench run-through
Preset reference — choosing the right preset for your workload
Docs site

What's included in v0.1.0

This is the first public release of context-guru. Everything in the repo ships:

✅ 13 pipeline components (3 reformat, 10 offload)
✅ Named presets (codesmart, codesafe, balanced, safe, agent, coding, mcp, summarize, off)
✅ Standalone HTTP proxy binary (context-guru-proxy) — Anthropic + OpenAI on one port
✅ Gateway mode with credential injection (eval-containers compatible)
✅ Docker image
✅ bifrost LLMPlugin adapter
✅ In-process apply.Body API for AuthBridge / custom host integration
✅ TTL+LRU session store with rewind + sticky slots
✅ Reversibility via <<cg:HASH>> markers + server-side expand loop
✅ Per-request pipeline override and bypass headers
✅ /stats token-weighted savings aggregator
✅ context_guru_expand model-callable tool (auto-injected)
✅ Dynamic context-window triggers (fraction of model window, per-request)
✅ Declarative DSL filter engine for cmdfilter (pytest, npm, make builtins + custom)
✅ tree-sitter code skeleton parsing (14 languages, optional build tag)
✅ Starlark sandboxing for LLM-written filters (extract_llm)
✅ Comprehensive docs and benchmark reproduction instructions

Notes

No breaking changes — this is the initial release; the public API (components, apply, config, expand, schema) is stabilising toward 1.0.
CGO is optional — omit -tags cg_skeleton for a pure-Go build; skeleton becomes a no-op.
Model costs — extract_llm and summarize call a cheap model only when CHEAP_MODEL is configured; without it they safely no-op (codesmart behaves as codesafe).
License: Apache-2.0

A Rossoctl platform component.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

context-guru v0.1.0

Choose a tag to compare

Sorry, something went wrong.

Sorry, something went wrong.

Uh oh!

No results found

context-guru v0.1.0 — First Public Release

What is context-guru?

Benchmark — the cheapest & highest-reward arm on SWE-bench Verified

Architecture

The Pipeline — 13 Components

Reformat — lossless repack

Offload — drop bytes, stash the original, leave a recoverable marker

Configuration Presets

Key Flags & Environment Variables

HTTP Routes

Integration Options

Quickstart (60 seconds)

Technical Stack

Documentation

What's included in v0.1.0

Notes

Uh oh!