OptiMoa

Smart LLM proxy that saves money and improves quality.

Point any AI agent's base_url to OptiMoa — it routes simple questions to cheap models, opens multi-model committees for high-risk questions, and learns from every decision to get smarter over time.

Why OptiMoa? Not every question needs GPT-4o. "What's 2+2?" and "Design a distributed system" shouldn't cost the same. OptiMoa automatically picks the right model for each question — cheap for simple, strong for complex, committee for high-stakes.

Quick Start

One-line Install

curl -fsSL https://raw.githubusercontent.com/haichen1985/opti-moa/main/install.sh | bash

From Source

git clone https://github.com/haichen1985/opti-moa.git
cd opti-moa && npm install && npm run build
node dist/index.js   # First run opens setup wizard

Docker

docker build -t opti-moa .
docker run -d -p 8080:8080 -v ~/.opti-moa:/root/.opti-moa opti-moa

Open http://localhost:8080 → fill in your API keys → done.

Connect Your Agent

Change one line — your agent's base_url:

# LangGraph / LangChain / CrewAI
llm = ChatOpenAI(base_url="http://127.0.0.1:8080/v1", api_key="none", model="auto")

# Claude Code
export ANTHROPIC_BASE_URL=http://127.0.0.1:8080

# Cursor: Settings → Models → OpenAI Base URL
# Base URL: http://127.0.0.1:8080/v1
# API Key: none
# Model: auto

# AutoGen
config_list = [{"base_url": "http://127.0.0.1:8080/v1", "api_key": "none", "model": "auto"}]

# Hermes (~/.hermes/config.yaml)
model:
  default: auto
  provider: custom
custom_providers:
  - name: opti-moa
    base_url: http://127.0.0.1:8080/v1
    api_key: "none"
    model: auto

# curl

curl http://127.0.0.1:8080/v1/chat/completions \
  -d '{"messages":[{"role":"user","content":"Hello"}]}'

Your agent works exactly as before — but now it's smarter and cheaper.

What It Does

Feature	Effect
Smart routing	Simple questions → cheap model, saves 60-80% cost
LLM-enhanced scoring	Borderline cases → keyword + LLM fusion scoring
Conditional committee	High-risk → quorum committee (2/3 members, no waiting for slowest)
Async Judge scoring	Background quality evaluation, zero added latency
Experience learning	Records quality scores, reuses best strategy per task type
Context compression	Trim tool logs + summarize old messages, save tokens
Semantic memory	Embedding-based recall of past conversations
Reasoning model support	Auto-detects reasoning_content (mimo, deepseek-reasoner)
Dynamic committee	Learns best model combo per task type from experience
Flywheel data	Export decision data for offline ML training
Cost control	Daily budget + MOA limit + auto-degrade

vs Alternatives

	OptiMoa	LangChain Router	LiteLLM	OpenRouter
Smart routing	✅ 4-tier + risk scoring	✅	❌	❌
Multi-model committee	✅ Conditional MOA	❌	❌	❌
Quality learning	✅ Judge + experience	❌	❌	❌
Zero-LLM scoring	✅ Keywords, <1ms	❌	❌	❌
Self-hosted	✅ SQLite, no cloud	✅	✅	❌
Setup effort	1 command	Heavy config	Medium	Easy

How It Works

Agent request
  ↓
① Keyword scoring (<1ms, zero LLM cost)
  ↓ borderline? → LLM fusion scoring (30% keyword + 70% LLM)
② Recall relevant memory (embedding similarity)
  ↓
③ Compress context (trim tool logs, summarize old turns)
  ↓
④ Check experience (has this task type been solved well before?)
  ├── Yes → reuse best strategy → return
  └── No → score-based routing
       ├── High risk → quorum committee (2/3 members respond → aggregate)
       └── Normal → single model
            → Judge checks quality (background, async)
            ├── Confident → return
            └── Uncertain → escalate to committee
  ↓
⑤ Record experience (SQLite)
⑥ Store memory (embeddings + SQLite)
  ↓
Return OpenAI-compatible response

Configuration

Config is auto-generated by the setup wizard. For manual editing:

# ~/.opti-moa/config.yaml
models:
  cheap:
    base_url: https://api.deepseek.com/v1
    api_key: ${DEEPSEEK_API_KEY}
    model: deepseek-chat
  strong:
    base_url: https://api.openai.com/v1
    api_key: ${OPENAI_API_KEY}
    model: gpt-4o

tiers:
  C0: cheap    # simple chat
  C1: cheap    # routine coding
  C2: strong   # complex analysis
  C3: strong   # high-risk (architecture/legal/medical)

committee:
  members: [cheap, strong]
  aggregator: strong
  trigger_threshold: 0.7

Any OpenAI-compatible endpoint works: DeepSeek, OpenAI, Anthropic, Groq, SiliconFlow, DashScope, Ollama, vLLM, etc.

API Endpoints

Endpoint	Description
POST /v1/chat/completions	OpenAI-compatible proxy (main entry)
GET /v1/models	List available models
GET /health	Health check
GET /stats	Experience engine stats
GET /memory	Semantic memory stats
GET /flywheel	Flywheel data stats + ML readiness
GET /flywheel/export	Export all decisions as JSONL
GET /	Web setup page (if not configured)

Project Structure

opti-moa/
├── src/
│   ├── index.ts          # Entry point
│   ├── config.ts         # Config loading + env var resolution
│   ├── scorer.ts         # 4-dimension keyword scoring (<1ms, zero LLM cost)
│   ├── llmScorer.ts      # LLM-enhanced scoring for borderline cases
│   ├── llmClient.ts      # LLM API client + reasoning model support
│   ├── judge.ts          # Answer quality judge
│   ├── experience.ts     # Experience learning engine (SQLite)
│   ├── moa.ts            # Quorum committee (2/3 members, no waiting)
│   ├── memory.ts         # Semantic memory (embeddings + SQLite)
│   ├── compressor.ts     # Context compression
│   ├── cacheControl.ts   # Anthropic prompt cache injection
│   ├── setup.ts          # Interactive CLI setup wizard
│   ├── server.ts         # Hono HTTP server + OpenAI proxy
│   └── web/setup.html    # Web configuration page
├── extras/               # Optional modules
│   ├── mcpServer.ts      # MCP tool server endpoints
│   ├── flywheel.ts       # Data flywheel export (standalone)
│   └── adapters/         # Agent framework adapters (Glue Layer)
│       ├── interface.ts  # Unified lifecycle hooks (ARP)
│       ├── hermes.ts     # Hermes framework adapter
│       ├── openclaw.ts   # OpenClaw framework adapter
│       └── index.ts      # Adapter registry + auto-detect
├── test/
│   ├── integration.test.ts  # Unit tests (16 tests, no LLM needed)
│   └── e2e.test.ts          # End-to-end tests (real LLM calls)
├── install.sh            # One-line install script
├── Dockerfile
├── package.json
├── tsconfig.json
├── config.example.yaml
└── README.md

Dependencies

• hono — HTTP server
• @hono/node-server — Node.js adapter for Hono
• better-sqlite3 — SQLite for experience + memory
• js-yaml — Config file parsing
• @inquirer/prompts — Interactive CLI setup

Node 18+ required (uses native fetch).

Testing

# Unit tests (no LLM needed, 16 tests)
npx tsx test/integration.test.ts

# End-to-end tests (requires configured API keys, 9 tests)
npx tsx test/e2e.test.ts

Framework Adapters (Glue Layer)

opti-moa can act as a Glue Layer between agent frameworks. Adapters in extras/adapters/ implement unified lifecycle hooks (ARP — Agent Runtime Protocol) to inject opti-moa's capabilities without modifying framework code.

Hermes

# Add to ~/.hermes/config.yaml
model:
  default: auto
  provider: custom
custom_providers:
  - name: opti-moa
    base_url: http://127.0.0.1:8080/v1
    api_key: "none"
    model: auto

OpenClaw

export OPENAI_BASE_URL=http://127.0.0.1:8080/v1
export OPENAI_API_KEY=none
export OPENAI_MODEL=auto

Any OpenAI-compatible framework

Point base_url to http://127.0.0.1:8080/v1 — routing, memory, compression, and experience learning happen automatically server-side.

License

MIT