Reduce LLM context bloat, control token usage, improve latency, and make AI applications more efficient — without changing the underlying model.
CtxLean is an open-source, provider-agnostic AI context optimization layer that sits between your application and an LLM provider.
It analyzes outgoing AI requests, identifies unnecessary context, applies optimization strategies, manages token budgets, and gives you visibility into where your context is being spent.
🚧 Status: Pre-Alpha / Design Phase. Nothing here is published yet. The code blocks, CLI output, and config examples below describe the target design — Star/watch the repo to follow progress, and don't
pip installyet — there's nothing on PyPI.
┌──────────────────────┐
│ AI Application │
│ │
│ RAG / Agent / MCP / │
│ Coding Assistant │
└──────────┬───────────┘
│
▼
┌──────────────────────────────────────┐
│ CtxLean │
│ │
│ Context Analysis │
│ Token Budgeting │
│ Deduplication │
│ Context Compression │
│ Semantic Selection │
│ Prompt / Prefix Caching │
│ Cost Estimation │
│ Request Analytics │
└──────────┬───────────────────────────┘
│
▼
┌──────────────────────────────────────┐
│ LLM Provider │
│ │
│ OpenAI / Gemini / Claude / Ollama / │
│ vLLM / Other Providers │
└──────────────────────────────────────┘
- Why CtxLean?
- Core Idea
- Features
- Provider Support
- Architecture
- Installation
- Planned Usage
- Configuration
- CLI
- Benchmarking
- Design Principles
- Privacy
- Use Cases
- CtxLean vs Traditional LLM Gateway
- Technology Stack
- Contributing
- License
Modern AI applications often send far more context than the model actually needs.
A simple request such as:
"Fix the authentication bug."
can result in a request containing:
System Prompt
+
Conversation History
+
Repository Files
+
Documentation
+
Git Diff
+
Tool Results
+
MCP Context
+
Retrieved Documents
+
User Prompt
The actual user instruction might be only a few tokens. The surrounding context can be thousands or millions of tokens.
This creates several problems:
- 💰 Higher token costs
- 🐌 Higher latency
- 🧠 Context-window pressure
- 🔁 Repeated computation
- 📦 Unnecessary context transmission
- 📉 Lower signal-to-noise ratio
- ⚡ Reduced inference efficiency
CtxLean aims to solve this at the context layer — before a single token reaches the model.
Instead of blindly sending everything available straight to the LLM, CtxLean runs an optimization pipeline first:
Everything available
│
▼
┌────────────────────┐
│ Context Analyzer │
├────────────────────┤
│ Deduplication │
├────────────────────┤
│ Relevance Filter │
├────────────────────┤
│ Compression │
├────────────────────┤
│ Token Budget │
└─────────┬──────────┘
▼
Optimized Context
│
▼
LLM
(Target behavior — see Roadmap for build status of each.)
Understand exactly where your tokens are going.
CtxLean Analysis
────────────────────────────
System Prompt 2,143 tokens
Conversation 8,421 tokens
Repository 31,220 tokens
Retrieved Context 12,840 tokens
Tool Results 4,231 tokens
User Prompt 42 tokens
────────────────────────────
Total 58,897 tokens
Detect repeated content — both exact duplicates and content duplicates — before it's sent to the model.
README.md
README.md
README.md
becomes
README.md
Long conversations and documents get compressed into structured summaries instead of being sent verbatim.
Conversation Summary
Objective: Fix authentication middleware.
Important decisions:
- JWT authentication is used.
- Redis stores sessions.
- OAuth is not currently enabled.
Current issue: Token expiration is not handled correctly.
Relevant files: auth.py, middleware.py, redis.py
Not every file in a repository is relevant to every request. For "Fix the JWT authentication bug", CtxLean can select auth.py, middleware.py, jwt.py, config.py, tests/test_auth.py — instead of the entire repository.
CtxLean fingerprints reusable context (system prompt, coding rules, repo summary) via content hashing:
Context Fingerprint
SHA256: a7f8c92...
Unchanged content → Cache HIT. Changed content → Cache MISS. This lets CtxLean reuse previously processed context wherever the underlying infrastructure supports it.
Define a budget:
budget:
max_input_tokens: 32000
reserve_output_tokens: 4000When context exceeds budget, CtxLean works through: remove duplicates → remove low-relevance content → compress history → compress documents → recalculate tokens.
Request #1842
Before Optimization After Optimization
──────────────────── ───────────────────
Input Tokens: 74,231 Input Tokens: 31,842
Output Budget: 4,096
Reduction: 57.1%
Estimated Savings: $0.18 / request
CtxLean is designed to work independently of the model provider. The goal is to optimize the context, not lock users into a particular model.
| Provider | Support |
|---|---|
| OpenAI | 🟡 Planned |
| Google Gemini | 🟡 Planned |
| Anthropic Claude | 🟡 Planned |
| Ollama | 🟡 Planned |
| vLLM | 🟡 Planned |
| SGLang | 🟡 Planned |
| llama.cpp | 🟡 Planned |
| Custom LLM APIs | 🟡 Planned |
(Table will update to 🟢 as each integration lands in Phase 5 — see Roadmap.)
┌─────────────────┐
│ AI Application │
└────────┬────────┘
│
▼
┌─────────────────────┐
│ CtxLean SDK │
└──────────┬──────────┘
│
┌─────────────┴─────────────┐
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Context Manager │ │ Token Analyzer │
└────────┬────────┘ └────────┬────────┘
│ │
▼ ▼
┌─────────────────┐ ┌─────────────────┐
│ Deduplication │ │ Budget Manager │
└────────┬────────┘ └────────┬────────┘
│ │
└─────────────┬─────────────┘
▼
┌─────────────────────┐
│ Optimization Engine │
└──────────┬──────────┘
│
▼
┌─────────────────────┐
│ Cache / Retrieval │
└──────────┬──────────┘
│
▼
┌─────────────────┐
│ LLM Provider │
└─────────────────┘
Not yet published. This is the target install path once Phase 1 ships — track progress in the Roadmap.
pip install ctxleanOr with uv:
uv add ctxleanTo try the current development state:
git clone https://github.com/NotHarshhaa/ctxlean.git
cd ctxlean
uv syncRequires Python 3.10+.
(Target API — not functional yet.)
from ctxlean import CtxLean
optimizer = CtxLean()
result = optimizer.optimize(
system_prompt=system_prompt,
history=conversation,
context=documents,
user_prompt=user_prompt,
)
print(result.optimized_context)
print(result.original_tokens)
print(result.optimized_tokens)
print(result.reduction_percentage)Example output:
Original Tokens: 48,421
Optimized Tokens: 21,842
Reduction: 54.89%
from ctxlean import CtxLean
from openai import OpenAI
optimizer = CtxLean()
client = OpenAI()
result = optimizer.optimize(
system_prompt=system_prompt,
history=history,
context=context,
user_prompt=user_prompt,
)
response = client.chat.completions.create(
model="gpt-5",
messages=result.messages,
)CtxLean will not require replacing your existing LLM provider client — it sits in front of the request.
(Planned config shape.)
ctxlean:
max_context_tokens: 32000
optimization:
deduplication: true
compression: true
semantic_selection: true
cache:
enabled: true
backend: sqlite
metrics:
enabled: true(Planned — see Phase 1 in Roadmap.)
ctxlean analyze prompt.jsonTarget output:
ctxlean
──────────────────────────────
Input Tokens 82,431
Estimated Cost $0.91
Optimization
──────────────────────────────
Deduplication -8,421
Compression -12,832
Relevance Filter -17,120
Final Tokens 44,058
Reduction 46.54%
Each stage below is independently configurable (on/off, thresholds, order):
Input → Context Analysis → Token Counting → Deduplication →
Relevance Filtering → Context Compression → Budget Enforcement →
Final Context → LLM
CtxLean will include reproducible benchmarks once the optimization pipeline (Phase 2) is functional. Planned dataset: repository QA.
Dataset: Repository QA
Before After
Tokens 82,421 39,842
Latency 4.21s 2.37s
Context Reduction 0% 51.66%
Metrics tracked: token reduction, latency, cost, context relevance, answer quality, cache hit rate, compression ratio. Optimization should never blindly sacrifice answer quality — benchmarks will report quality deltas alongside token savings, not just the savings alone.
- Provider Agnostic — don't depend on one model provider.
- Optimization Before Inference — reduce unnecessary context before it reaches the model.
- Quality First — token reduction is meaningless if response quality collapses.
- Observable — every optimization decision should be measurable.
- Configurable — developers control optimization policies.
- Local First — sensitive enterprise context can stay local.
- Open Source — core optimization logic stays transparent and auditable.
CtxLean is designed to keep analysis, cache, and metrics local by default — it will not require sending application context to a third-party optimization service.
Application → CtxLean → Local analysis / Local cache / Local metrics
This matters most for enterprise repositories, source code, internal documentation, customer data, and proprietary AI applications.
⚠️ Scope note: CtxLean does not modify or intercept proprietary services such as GitHub Copilot. It's an independent, provider-agnostic layer for pipelines you control.
Coding Assistants — trim a full repository down to relevant files before sending to the model.
RAG Applications — optimize retrieved documents (e.g. top 100 vector search hits) before they hit the LLM.
AI Agents — compact the growing pile of tool-call results that agents accumulate over a session.
MCP Applications — sit between MCP tool results and the model to keep context lean.
Enterprise AI Platforms — apply token governance, cost controls, context policies, metrics, and caching centrally, in front of an LLM gateway.
| Capability | Traditional LLM Gateway | CtxLean |
|---|---|---|
| Request routing | ✅ | 🔜 |
| Authentication | ✅ | 🔜 |
| Rate limiting | ✅ | 🔜 |
| Token counting | 🟡 | ✅ |
| Context analysis | ❌ | ✅ |
| Deduplication | ❌ | ✅ |
| Context compression | ❌ | ✅ |
| Semantic selection | ❌ | ✅ |
| Prompt caching | 🟡 | ✅ |
| Token budgets | 🟡 | ✅ |
| Context optimization | ❌ | ✅ |
| Optimization analytics | 🟡 | ✅ |
(✅ = built once Phase 1–3 land; 🔜 = not planned near-term — CtxLean focuses on the context layer, not routing/auth.)
CtxLean focuses specifically on the context layer, not on replacing your existing gateway.
Language Python
Package Management uv
Configuration Pydantic
CLI Typer
Testing Pytest
Caching SQLite, Redis
Retrieval Embeddings, Vector Database
Observability OpenTelemetry, Prometheus
API FastAPI
Future infrastructure: Docker, Kubernetes, Helm, Prometheus, Grafana, OpenTelemetry.
CtxLean can evolve from a Python SDK into a full context control plane sitting in front of any LLM gateway or local model:
Applications → CtxLean (Optimization, Governance, Retrieval,
Caching, Cost Mgmt, Observability)
→ LLM Gateway / Local Models → OpenAI / Gemini / Claude / etc.
Make every token count.
Contributions are welcome — CtxLean is early enough that architectural feedback matters as much as code.
git clone https://github.com/NotHarshhaa/ctxlean.git
cd ctxlean
uv sync
uv run pytestgit checkout -b feature/context-optimizerMake your changes, add tests, and open a pull request. See CONTRIBUTING.md for guidelines.
Released under the Apache License 2.0.
If CtxLean looks useful, consider starring the repo — it helps gauge interest while the project is still pre-alpha.
CtxLean — Optimize the context. Protect the budget. Accelerate AI.