One endpoint. Every LLM you need. Zero lock-in.

---

Origin Story: This project is a ground-up restructuring of freellmapi — a brilliant gateway that aggregated free-tier overseas models (GPT-4, Claude, Gemini) behind a single OpenAI-compatible endpoint. We took its core idea (Express proxy + key pool + SQLite + AES encryption) and flipped the thesis: instead of chasing free overseas quota, we optimize for Chinese domestic models with intent × difficulty routing — while keeping the best of overseas (Claude) accessible through the same unified endpoint. Same DNA, evolved philosophy.

	freellmapi	llm-keypool
Target	Overseas free tiers (GPT-4, Claude, Gemini)	Domestic + overseas (DeepSeek, GLM, Qwen, Doubao, MiniMax, Kimi, Claude)
Routing	Round-robin / first-available	Intent × Difficulty matrix (8 intents × 3 levels)
Selection	Cheapest free model that works	Best-fit model for the task (code→coder, reason→R1, chat→flash)
Key mgmt	Free key pooling	Encrypted pool + free-tier-first + 429 auto-fallback
Dashboard	React SPA	React + Vite SPA (4-page admin: Dashboard / Keys / Models / Playground)
Philosophy	Maximize free overseas quota	Match the right model to the right job (domestic-first, global-ready)

---

Why This Exists

The Chinese LLM landscape is fundamentally different from the overseas one:

1. Fragmentation — 12+ providers, 38+ models, each with different APIs, rate limits, and strengths. There's no single "best" model — DeepSeek R1 dominates reasoning, GLM-4-Flash is the fastest free chat, Qwen-Long handles 10M context, Kimi excels at search-augmented tasks, Claude leads in complex reasoning and code, GPT-4o and Gemini 2.5 Pro push the frontier on multimodal tasks, and Agnes AI offers free 1M-context access.
2. Free tiers everywhere — Unlike the overseas market where free quota is scarce, Chinese providers offer generous free tiers. The challenge isn't finding free access — it's picking the right model for the right job.
3. Intent matters — Sending a coding task to a general chat model wastes tokens and gets worse results. Sending a simple "hello" to DeepSeek R1 wastes its reasoning capability and costs more time. You need routing, not just proxying.

You send:  POST /v1/chat/completions  {"model": "auto", "messages": [...]}
We think:  intent=code, difficulty=hard → what's the best coder for hard tasks?
We route:  deepseek-coder via DeepSeek provider (with 429 fallback to qwen-coder)
You get:   OpenAI-compatible response + routing metadata headers

---

✨ Highlights

	Feature	What it means for you
🧭	Intent × Difficulty Routing	8 intents × 3 difficulty levels → 24-cell matrix auto-picks the right model across 12 providers
🔑	Key Pool with AES-256-GCM	Add multiple keys per provider; encrypted at rest, free-tier-first selection
🛡️	Automatic Fallback	Provider A returns 429? Cooldown that key, switch to Provider B — transparently
📡	Streaming Passthrough	SSE streams flow through untouched — no buffering, no content-type mangling
🔌	Drop-in OpenAI Compatible	Change base_url, keep everything else — works with LangChain, Cursor, Continue, Claude Code, etc.
🌏	Domestic-First + Global	DeepSeek, GLM, Qwen, Doubao, MiniMax, Kimi, Agnes (free tiers) + OpenAI, Anthropic, Gemini, OpenRouter (overseas)
🤖	Multi-Protocol Support	Anthropic Claude (Messages API), Google Gemini (Generative AI API), OpenAI-compatible — all transparent to clients
📊	React Admin Dashboard	4-page SPA: real-time monitoring, key management, model routing matrix, interactive playground
🪶	Zero Dependencies Bloat	Express + better-sqlite3 + dotenv. That's it. No Redis, no Docker, no Kubernetes

---

🚀 30-Second Quick Start

git clone https://github.com/xiaopengs/llm-keypool.git
cd llm-keypool
npm install && npm run build

# Build the admin dashboard
cd client && npm install && npm run build && cd ..

# Generate encryption key (required for API key storage)
node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"

# Configure
cp .env.example .env
# Edit .env: paste the key as ENCRYPTION_KEY, set PROXY_TOKEN

# Launch
npm start
# → [keypool] v2.0.0 listening on :3131
# → [keypool] Dashboard: http://localhost:3131/dashboard

# Add your first API key (free tier — most Chinese providers offer one)
curl -X POST http://localhost:3131/api/keys \
  -H "Authorization: Bearer YOUR_PROXY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"provider":"deepseek","api_key":"sk-xxx","tier":"free"}'

# Send a request — watch it auto-route
curl http://localhost:3131/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Write a Python quicksort"}]}'

# Response headers tell you where it went:
# X-Routed-Provider: deepseek
# X-Routed-Model: deepseek-coder
# X-Intent-Detected: code
# X-Difficulty: medium

# Use Claude (Anthropic) — just specify the model
curl http://localhost:3131/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-4","messages":[{"role":"user","content":"Explain quantum computing"}]}'

# Open the React admin dashboard to manage and monitor everything
open http://localhost:3131/dashboard

That's it. One key, one endpoint, zero configuration beyond .env. The routing matrix has sensible defaults — you only customize when you want to.

---

💻 IDE & Tool Integration

llm-keypool speaks pure OpenAI API protocol. Any tool that lets you configure a custom base_url or api_base works out of the box.

Claude Code (Anthropic CLI)

Claude Code supports custom OpenAI-compatible endpoints via environment variables:

# Set in your shell or .bashrc/.zshrc
export OPENAI_API_KEY="anything"  # keypool handles upstream auth
export OPENAI_BASE_URL="http://localhost:3131/v1"

# Now Claude Code routes through keypool
claude "Write a TypeScript HTTP server"
# → Routes to code:medium → qwen-coder (or your configured model)

Or configure per-project in .claude/settings.json:

{
  "env": {
    "OPENAI_API_KEY": "anything",
    "OPENAI_BASE_URL": "http://localhost:3131/v1"
  }
}

Cursor

Settings → Models → OpenAI API Base URL:

API Base URL:  http://localhost:3131/v1
API Key:       anything
Model:         auto

Or use intent-prefixed models for specific tasks:

Model in Cursor	Routes to
auto	Automatic intent + difficulty routing
code:deepseek-coder	Force coding → DeepSeek Coder
reasoning:deepseek-r1	Force reasoning → DeepSeek R1
deepseek-v3	Direct model — no routing

Continue (VS Code / JetBrains)

Edit ~/.continue/config.json:

{
  "models": [
    {
      "title": "llm-keypool (auto)",
      "provider": "openai",
      "model": "auto",
      "apiBase": "http://localhost:3131/v1",
      "apiKey": "anything"
    },
    {
      "title": "llm-keypool (code)",
      "provider": "openai",
      "model": "code:deepseek-coder",
      "apiBase": "http://localhost:3131/v1",
      "apiKey": "anything"
    }
  ]
}

Cline / Roo Code (VS Code)

In extension settings:

API Provider:  OpenAI Compatible
Base URL:      http://localhost:3131/v1
API Key:       anything
Model ID:      auto

Aider

aider --openai-api-base http://localhost:3131/v1 \
      --model openai/auto \
      --openai-api-key anything

LangChain / LlamaIndex / Any Python SDK

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3131/v1",
    api_key="anything"
)

# Auto-route
response = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Explain transformers"}]
)

# Force reasoning
response = client.chat.completions.create(
    model="reasoning:deepseek-r1",
    messages=[{"role": "user", "content": "Prove the halting problem"}]
)

Shell Alias (Quick Setup)

Add to your .bashrc / .zshrc:

# Route all OpenAI calls through keypool
export OPENAI_API_KEY="anything"
export OPENAI_API_BASE="http://localhost:3131/v1"

# Or for specific tools
export DEEPSEEK_API_KEY="anything"
export DEEPSEEK_API_BASE="http://localhost:3131/v1"

Tip: The model field is your routing control plane. "auto" for hands-off, "intent:alias" for explicit control, "alias" for direct selection. See Routing for details.

---

🧭 Routing: How It Works

This is the core innovation over freellmapi's round-robin approach. Every request goes through a 3-stage pipeline:

┌─────────────────────────────────────────────────────────┐
│                    Request Pipeline                      │
│                                                         │
│  1. INTENT         2. DIFFICULTY       3. RESOLVE       │
│  ┌───────────┐    ┌──────────────┐    ┌──────────────┐  │
│  │ Explicit: │    │ msg length   │    │  8 × 3       │  │
│  │  prefix:  │───▶│ tool count   │───▶│  MATRIX      │  │
│  │  header:  │    │ conv depth   │    │  lookup      │  │
│  │ Implicit: │    │ max_tokens   │    │      │       │  │
│  │  keywords │    │ system prompt│    │      ▼       │  │
│  │  patterns │    │      │       │    │  model_alias │  │
│  └───────────┘    └──────────────┘    └──────────────┘  │
│        │                 │                    │          │
│   chat/code/        easy/medium/       deepseek-v3/    │
│   reasoning/...     hard               glm-4-flash/... │
│                         │                    │          │
│                    score: 0-6           key pick +      │
│                                         rate limit +    │
│                                         fallback        │
└─────────────────────────────────────────────────────────┘

Intent Classification

Signal	Example	Confidence
Model prefix	"model": "code:deepseek-v3"	1.0 (explicit)
X-Intent header	X-Intent: reasoning	1.0 (explicit)
Content keywords	"写代码", "debug", "implement"	0.3–1.0 (implicit)
Tool names	code_interpreter, file_edit	0.3 (implicit)
Fallback	none matched	0.5 → defaults to chat

8 intents: chat · code · reasoning · creative · search · vision · translation · summarization

Difficulty Assessment

The gateway evaluates request complexity on a 0–6 scale across multiple signals:

Signal	Easy (+0)	Medium (+1)	Hard (+2)
Message length	<1K chars	1K–4K	>4K
Tool count	0	1–3	>3
Conversation depth	≤5 turns	5–10	>10
System prompt	absent	present	complex
max_tokens hint	≤200	200–1K	>1K

Score ≥3 → hard, ≥1 → medium, else → easy

This is what makes keypool different: freellmapi treated every request the same. We ask "what are you trying to do?" and "how hard is it?" — then route accordingly.

Default Routing Matrix

Intent	Easy	Medium	Hard
chat	glm-4-flash	deepseek-v3	qwen-max
code	glm-4-flash	qwen-coder	deepseek-coder
reasoning	deepseek-v3	deepseek-r1	minimax-m1
creative	doubao-lite	minimax-creative	glm-z1
search	glm-4-flash	kimi	qwen-max
vision	glm-4v	qwen-vl	kimi-vl
translation	qwen-turbo	deepseek-v3	glm-4
summarization	glm-4-flash	qwen-long	glm-4-long

Override any cell via PUT /api/routing — no code changes, no restarts.

---

🔌 Providers

Provider	Base URL	Auth	Notes
DeepSeek	api.deepseek.com/v1	Bearer token	Most popular free tier; R1 is best-in-class reasoning
智谱 GLM	open.bigmodel.cn/api/paas/v4	Bearer token	GLM-4-Flash: 100 RPM free; GLM-4-Long: 1M context
通义 Qwen	dashscope.aliyuncs.com/compatible-mode/v1	Bearer token	Qwen-Long: 10M context; best multi-variant lineup
豆包 Doubao	ark.cn-beijing.volces.com/api/v3	Bearer token	Volcengine/ByteDance; uses endpoint_id mapping
MiniMax	api.minimax.chat/v1	Bearer token	M1: 1M context reasoning; abab6.5s for creative
Kimi/Moonshot	api.moonshot.cn/v1	Bearer token	Best search-augmented chat; VL for multimodal
OpenAI	api.openai.com/v1	Bearer token	GPT-4o, GPT-4o Mini, o1, o3-mini; official API
Anthropic	api.anthropic.com/v1	x-api-key header	Claude Sonnet 4, Opus 4, Haiku 3.5; protocol adaptation
Google Gemini	generativelanguage.googleapis.com/v1beta	API key (query param)	Gemini 2.5 Pro/Flash, 2.0 Flash; Generative AI API
OpenRouter	openrouter.ai/api/v1	Bearer token	Unified access to 200+ models via OpenAI-compatible API
Agnes AI	apihub.agnes-ai.com/v1	Bearer token	无限期免费的全模态 AI；Agnes-2.0-Flash: 1M 上下文
Custom	User-configured	Bearer token	Any OpenAI-compatible endpoint (vLLM, Ollama, LiteLLM, etc.)

Getting Free API Keys

Most Chinese providers offer free tiers with no credit card required:

Provider	Sign-up URL	Free Tier
DeepSeek	platform.deepseek.com	50 RPM, 50K TPM
智谱 GLM	open.bigmodel.cn	GLM-4-Flash: 100 RPM free
通义 Qwen	dashscope.console.aliyun.com	Free quota on all models
豆包 Doubao	console.volcengine.com/ark	Free tier available
MiniMax	platform.minimax.chat	Free quota
Kimi/Moonshot	platform.moonshot.cn	Free tier available
OpenAI	platform.openai.com	API usage-based pricing
Anthropic	console.anthropic.com	API usage-based pricing
Google Gemini	aistudio.google.com	Free tier available
OpenRouter	openrouter.ai	Unified billing for 200+ models
Agnes AI	agnes-ai.com	无限期免费，无需信用卡
Custom	N/A	Depends on your endpoint

Doubao Special Handling

Doubao uses Volcengine Ark's endpoint_id system instead of model names. The gateway handles this transparently:

# You use the stable alias
curl ... -d '{"model":"doubao-pro",...}'

# We translate to the endpoint_id under the hood
# → {"model": "ep-xxxx", ...}  (configured in DB)

Anthropic Protocol Adaptation

Anthropic's Messages API differs significantly from OpenAI's Chat Completions API. The gateway handles all protocol differences transparently:

Aspect	OpenAI Format	Anthropic Format	Gateway Action
Endpoint	/v1/chat/completions	/v1/messages	Auto-redirect
Auth	Authorization: Bearer	x-api-key header	Auto-convert
System msg	In messages array	Top-level system field	Auto-extract
Max tokens	Optional	Required (default 4096)	Auto-inject
Response	choices[0].message.content	content[0].text	Auto-transform

# You send OpenAI-compatible format
curl http://localhost:3131/v1/chat/completions \
  -d '{"model":"claude-sonnet-4","messages":[{"role":"user","content":"Hello!"}]}'

# We convert to Anthropic format, call their API,
# then transform the response back to OpenAI format

Streaming: SSE streams pass through with X-Upstream-Format: anthropic header. Clients should handle Anthropic's SSE event format for streaming Claude requests.

---

🗂️ Supported Models (38)

DeepSeek (3 models)

Alias	API Model	Context	Intent	Difficulty
deepseek-v3	deepseek-chat	64K	chat, code	medium
deepseek-r1	deepseek-reasoner	64K	reasoning	hard
deepseek-coder	deepseek-chat	128K	code	medium

智谱 GLM (5 models)

Alias	API Model	Context	Intent	Difficulty
glm-4	glm-4	128K	chat, reasoning	medium
glm-4-flash	glm-4-flash	128K	chat, search, translation	easy
glm-z1	glm-z1-air	128K	reasoning	hard
glm-4v	glm-4v	128K	vision	medium
glm-4-long	glm-4-long	1M	chat, summarization	medium

通义 Qwen (6 models)

Alias	API Model	Context	Intent	Difficulty
qwen-max	qwen-max	32K	chat, reasoning	hard
qwen-plus	qwen-plus	131K	chat, code	medium
qwen-turbo	qwen-turbo	131K	chat, search, translation	easy
qwen-coder	qwen-coder-plus	131K	code	medium
qwen-vl	qwen-vl-max	32K	vision	medium
qwen-long	qwen-long	10M	summarization	medium

豆包 Doubao (3 models)

Alias	API Model	Context	Intent	Difficulty
doubao-pro	doubao-pro-32k	32K	chat, creative	medium
doubao-lite	doubao-lite-32k	32K	chat	easy
doubao-pro-128k	doubao-pro-128k	128K	chat, summarization	medium

MiniMax (3 models)

Alias	API Model	Context	Intent	Difficulty
minimax-m1	MiniMax-M1	1M	chat, reasoning	hard
minimax-text	abab6.5s-chat	245K	chat	medium
minimax-creative	abab6.5s-chat	245K	creative	medium

Kimi / Moonshot (4 models)

Alias	API Model	Context	Intent	Difficulty
kimi	moonshot-v1-auto	128K	chat, search	medium
kimi-8k	moonshot-v1-8k	8K	chat	easy
kimi-32k	moonshot-v1-32k	32K	chat, summarization	medium
kimi-vl	kimi-vl-audio	128K	vision	medium

Anthropic Claude (3 models)

Alias	API Model	Context	Intent	Difficulty
claude-sonnet-4	claude-sonnet-4-20250514	200K	chat, code, reasoning, creative	hard
claude-opus-4	claude-opus-4-20250514	200K	reasoning, code, creative	hard
claude-haiku-3.5	claude-3-5-haiku-20241022	200K	chat, search, translation	easy

OpenAI (4 models)

Alias	API Model	Context	Intent	Difficulty
gpt-4o	gpt-4o	128K	chat, code, reasoning, creative	hard
gpt-4o-mini	gpt-4o-mini	128K	chat, search, translation	easy
o1	o1	200K	reasoning	hard
o3-mini	o3-mini	200K	reasoning, code	medium

Google Gemini (3 models)

Alias	API Model	Context	Intent	Difficulty
gemini-2.5-pro	gemini-2.5-pro-preview-06-05	1M	reasoning, code, creative	hard
gemini-2.5-flash	gemini-2.5-flash-preview-05-20	1M	chat, code, search	medium
gemini-2.0-flash	gemini-2.0-flash	1M	chat, search, translation	easy

OpenRouter (4 models)

OpenRouter provides unified access to 200+ models. These are pre-configured aliases for popular models:

Alias	API Model	Context	Intent	Difficulty
openrouter:claude-sonnet-4	anthropic/claude-sonnet-4	200K	chat, code, reasoning, creative	hard
openrouter:gpt-4o	openai/gpt-4o	128K	chat, code, reasoning, creative	hard
openrouter:gemini-2.5-pro	google/gemini-2.5-pro-preview	1M	reasoning, code, creative	hard
openrouter:deepseek-r1	deepseek/deepseek-r1	64K	reasoning	hard

Agnes AI (1 model)

Agnes AI provides free multimodal AI services. Currently only the text model is supported (image/video generation requires different API endpoints):

Alias	API Model	Context	Intent	Difficulty
agnes-2.0-flash	agnes-2.0-flash	1M	chat, code, reasoning, creative	medium

Note: Agnes AI also offers image generation (Agnes-Image-2.1-Flash) and video generation (Agnes-2.0-Video) models, but these require different API endpoints and are not yet supported by llm-keypool.

Custom Endpoint (user-configured)

The custom provider supports any OpenAI-compatible API endpoint (vLLM, Ollama, LiteLLM, etc.). No models are pre-seeded — you define them via the admin API:

# Add a custom endpoint
curl -X POST http://localhost:3131/api/keys \
  -H "Authorization: Bearer YOUR_PROXY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"provider":"custom","api_key":"your-key","label":"My vLLM","base_url":"http://localhost:8000/v1"}'

---

📊 Dashboard

llm-keypool ships with a React + Vite admin dashboard at http://localhost:3131/dashboard — a full-featured SPA for managing your gateway.

Pages

Page	Route	Description
Dashboard	/dashboard/	Real-time monitoring: stats overview, timeline charts, live request table, provider distribution, intent×difficulty heatmap
Keys	/dashboard/keys	API key management: add/remove/toggle keys per provider, free/paid tier, masked display
Models	/dashboard/models	Model registry & routing matrix: 38 models across 12 providers, intent×difficulty assignments, provider badges
Playground	/dashboard/playground	Interactive chat: test any model through the gateway with live streaming responses, routing metadata display

Tech Stack

Layer	Technology
Framework	React 18 + TypeScript
Build	Vite 5 + PostCSS + Tailwind CSS 3
UI	shadcn/ui (Card, Button, Table, Badge, Switch, Select...) + Radix UI
Data	TanStack React Query 5 (server state management)
Charts	Recharts (timeline, heatmap)
Icons	Lucide React
Routing	React Router DOM 6

Authentication

The dashboard requires the same PROXY_TOKEN as the admin API. Enter it in the auth gate on first visit — stored in localStorage for convenience.

Legacy Dashboard

The original single-file HTML dashboard is still available at http://localhost:3131/legacy-dashboard for backward compatibility. It provides the same real-time monitoring without requiring a build step.

Build & Development

# Build the dashboard for production
cd client && npm install && npm run build && cd ..
# Output: client/dist/ → served at /dashboard

# Development with hot reload
cd client && npm run dev
# Vite dev server at http://localhost:5173, proxies API to :3131

---

📖 API Reference

Chat Completion

POST /v1/chat/completions
Content-Type: application/json

Request body — 100% OpenAI-compatible, plus:

Field	Type	Description
model	string	Model alias, "auto", or "intent:alias" (e.g. "code:deepseek-v3")
messages	array	Required. Same format as OpenAI
stream	boolean	true for SSE streaming
temperature, top_p, max_tokens, tools, etc.	—	All pass through

Model field routing modes:

model value	Behavior	Example
"auto"	Full auto: classify intent + assess difficulty → matrix lookup	Chat, code, reasoning — all automatic
"intent:alias"	Explicit intent, auto difficulty	"code:auto" → best code model for this difficulty
"intent:specific"	Explicit intent + explicit model	"code:deepseek-coder" → force DeepSeek Coder
"model-alias"	Direct model, no routing	"deepseek-r1" → go straight to R1

Response headers (routing metadata):

X-Routed-Provider: deepseek
X-Routed-Model: deepseek-coder
X-Route-Latency-Ms: 342
X-Intent-Detected: code
X-Intent-Confidence: 1.0
X-Intent-Source: explicit
X-Difficulty: medium
X-Difficulty-Score: 2.0

List Models

GET /v1/models

Returns OpenAI-format model list with extra fields (display_name, intent_tags, context_window, etc.).

Admin — Key Management

POST   /api/keys          # Add key: {provider, api_key, label?, tier?}
GET    /api/keys          # List all keys (masked)
DELETE /api/keys/:id      # Remove key
PATCH  /api/keys/:id      # Enable/disable: {enabled: true/false}

Admin — Routing

GET    /api/routing       # Full routing matrix (default + overrides)
PUT    /api/routing       # Upsert rule: {intent, difficulty, provider, model_alias, priority?}
DELETE /api/routing/:id   # Delete a rule
POST   /api/routing/reset # Reset to defaults

Admin — System

GET    /api/status        # Health + key status
GET    /api/providers     # Registered providers
POST   /api/seed          # Re-seed default models
GET    /health            # Lightweight health check (no auth)

Analytics — Dashboard Data

GET    /api/analytics/summary      # Overview: total requests, today's requests, avg latency, error rate
GET    /api/analytics/timeline     # Time series: requests/latency/errors by minute (?minutes=60)
GET    /api/analytics/recent       # Recent requests: last N requests (?limit=50)
GET    /api/analytics/providers    # Provider stats: requests, latency, error rate, tokens
GET    /api/analytics/models       # Model stats: same as providers, grouped by model
GET    /api/analytics/intents      # Intent × Difficulty matrix heatmap data

All /api/* routes require Authorization: Bearer <PROXY_TOKEN>

---

⚙️ Configuration

Environment Variables

Variable	Required	Default	Description
ENCRYPTION_KEY	✅	—	64 hex chars (32 bytes). AES-256-GCM key for API key storage
PROXY_TOKEN	✅	—	Bearer token for admin API auth
PORT		3131	Server port
DB_PATH		./data/keypool.db	SQLite database path
LOG_LEVEL		info	debug · info · warn · error

Minimal .env

ENCRYPTION_KEY=<output of: node -e "console.log(require('crypto').randomBytes(32).toString('hex'))">
PROXY_TOKEN=your-secret-token-here
PORT=3131

---

🏗️ Architecture

llm-keypool/
├── src/                          # Backend (Express + TypeScript)
│   ├── index.ts                  # Entry: Express server + static serving + SPA fallback
│   ├── config.ts                 # Environment config
│   ├── aes.ts                    # AES-256-GCM encrypt/decrypt
│   ├── rate-limiter.ts           # Sliding window RPM tracker
│   ├── router.ts                 # Core: route → pick key → forward → fallback
│   ├── router/
│   │   ├── intent.ts             # 8-intent classifier (explicit > implicit > default)
│   │   ├── difficulty.ts         # 3-level difficulty assessor (multi-signal)
│   │   └── model-map.ts          # 8×3 routing matrix + DB overrides
│   ├── providers/
│   │   ├── types.ts              # ProviderAdapter interface + error classifier
│   │   ├── openai-compat.ts      # Shared adapter for OpenAI-compatible providers (DeepSeek, GLM, Qwen, MiniMax, Moonshot, Doubao, OpenAI, OpenRouter, Agnes, Custom)
│   │   ├── doubao.ts             # Doubao adapter (endpoint_id mapping)
│   │   ├── anthropic.ts          # Anthropic adapter (Messages API protocol adaptation)
│   │   └── gemini.ts             # Google Gemini adapter (Generative AI API protocol adaptation)
│   │   └── index.ts              # Provider registry
│   ├── db/
│   │   ├── index.ts              # SQLite WAL init + schema
│   │   ├── keys.ts               # Key CRUD + encrypted storage + cooldown/health
│   │   ├── models.ts             # Model CRUD + 38-model seed data
│   │   └── request-log.ts        # Async batch request logger
│   └── routes/
│       ├── chat.ts               # /v1/* endpoints (OpenAI-compatible)
│       ├── admin.ts              # /api/* endpoints (key/routing/status mgmt)
│       └── analytics.ts          # /api/analytics/* endpoints (dashboard data)
├── client/                       # Frontend (React + Vite admin dashboard)
│   ├── src/
│   │   ├── main.tsx              # React entry point
│   │   ├── App.tsx               # Router + auth gate + layout
│   │   ├── pages/
│   │   │   ├── DashboardPage.tsx # Real-time monitoring & analytics
│   │   │   ├── KeysPage.tsx      # API key management (CRUD + toggle)
│   │   │   ├── ModelsPage.tsx    # Model registry (38 models across 12 providers) & routing matrix
│   │   │   └── PlaygroundPage.tsx# Interactive chat playground
│   │   ├── components/
│   │   │   ├── auth-gate.tsx     # Bearer token authentication gate
│   │   │   ├── page-header.tsx   # Shared page header with nav
│   │   │   └── ui/               # shadcn/ui components (12 components)
│   │   └── lib/
│   │       ├── api.ts            # API client (fetch wrappers for all endpoints)
│   │       └── utils.ts          # Tailwind merge utility
│   ├── index.html                # Vite HTML entry
│   ├── vite.config.ts            # Vite config + dev proxy to :3131
│   ├── package.json              # React 18 + TanStack Query + Recharts + shadcn/ui
│   └── dist/                     # Build output → served at /dashboard
├── dashboard/
│   └── index.html                # Legacy single-file dashboard (→ /legacy-dashboard)
├── docs/
│   ├── index.html                # Project landing page
│   └── assets/                   # Logo SVGs
├── tests/                        # Test suite (110 tests)
│   ├── setup.ts                  # Test environment configuration
│   ├── unit/                     # Unit tests (86 tests)
│   │   ├── crypto.test.ts        # AES-256-GCM encryption/decryption
│   │   ├── difficulty.test.ts    # Difficulty assessment logic
│   │   ├── intent.test.ts        # Intent classification
│   │   ├── model-map.test.ts     # Routing matrix
│   │   ├── providers.test.ts     # Provider adapters
│   │   └── rate-limiter.test.ts  # Rate limiting logic
│   └── integration/              # Integration tests (24 tests)
│       └── api.test.ts           # HTTP endpoints + auth + admin + analytics
├── data/                         # SQLite DB (gitignored)
├── .env                          # Config (gitignored)
├── package.json                  # Backend dependencies
├── vitest.config.ts              # Vitest test configuration
└── tsconfig.json

Design Principles

1. First principles over frameworks — No monorepo, no class inheritance, no Redis. Express + SQLite + config objects.
2. Configuration objects over inheritance — 10 providers share one OpenAICompatProvider instance; only baseUrl differs. Anthropic and Gemini have dedicated adapters for protocol adaptation.
3. Explicit > Implicit — Model prefix (code:xxx) and X-Intent header always win over keyword guessing.
4. Fail soft — Confidence < 0.6? Fall back to chat. 429? Cooldown + switch provider. DB not ready? Use defaults.
5. Domestic-first, global-ready — Default routing matrix uses Chinese providers, but OpenAI, Google Gemini, Anthropic Claude, and OpenRouter are fully supported for overseas model access.
6. Frontend-backend separation — React SPA builds to client/dist/, served by Express at /dashboard. Legacy single-file dashboard preserved at /legacy-dashboard. Zero coupling between build chains.

What We Kept from freellmapi

Concept	freellmapi	llm-keypool
Express proxy pattern	✅	✅
SQLite + WAL mode	✅	✅
AES-256-GCM key encryption	✅	✅
Key pool with free-tier-first	✅	✅ (simplified: no tier accounting)
Provider adapter pattern	✅	✅ (config objects, not class inheritance)
Streaming SSE passthrough	✅	✅ (fixed: no content-type override)
Admin dashboard	✅	✅ (React + Vite SPA with 4 pages)

What We Changed

Aspect	freellmapi	llm-keypool
Target providers	GPT-4, Claude, Gemini (overseas)	DeepSeek, GLM, Qwen, Doubao, MiniMax, Kimi, OpenAI, Gemini, Anthropic, OpenRouter, Agnes, Custom (domestic + global)
Routing	Round-robin / first-available	Intent × Difficulty matrix
Architecture	Monorepo (React + Express)	Separate packages (Express backend + React SPA client)
Provider adapters	Class inheritance	Config objects (10 providers share one OpenAICompat instance + 2 protocol adapters)
Doubao	Not supported	Special adapter for Volcengine endpoint_id
Anthropic	Not adapted	Full protocol adaptation (Messages API ↔ OpenAI format)
Google Gemini	Not adapted	Full protocol adaptation (Generative AI API ↔ OpenAI format)
Rate limiting	Token accounting	Sliding window RPM (simpler, practical)
Auth model	Hardcoded	PROXY_TOKEN env (simple Bearer)
Dashboard	React SPA (embedded)	React + Vite SPA (4 pages: Dashboard, Keys, Models, Playground)

---

🛣️ Roadmap

Phase	Status	Scope
Phase 1 Core	✅ Done	Express + SQLite WAL + AES encryption + 6 providers + streaming + 429 fallback
Phase 2 Routing	✅ Done	Intent/difficulty routing + 8×3 matrix + rate limiter + routing CRUD
Phase 3 Claude + Analytics	✅ Done	Anthropic/Claude provider + analytics API + legacy single-file dashboard
Phase 4 Admin Dashboard	✅ Done	React + Vite SPA: 4 pages (Dashboard, Keys, Models, Playground) + shadcn/ui + TanStack Query + Recharts
Phase 5 Global Providers	✅ Done	OpenAI (4 models) + Google Gemini (3 models) + OpenRouter (4 models) + Agnes AI (1 model) + Custom endpoint support (38 models, 12 providers)
Phase 6 Testing	✅ Done	110 tests: 86 unit tests (Vitest) + 24 integration tests (supertest) + coverage reporting
Phase 7 Advanced	📋 Planned	Sticky sessions + CLI tool + token accounting + cost tracking

---

🧪 Testing

The project includes comprehensive test coverage with 110 tests (86 unit + 24 integration):

# Run all tests
npm test

# Run with coverage
npm test -- --coverage

# Run specific test suite
npx vitest tests/unit/crypto.test.ts
npx vitest tests/integration/api.test.ts

Test Structure:

• Unit tests (tests/unit/): Crypto, difficulty assessment, intent classification, model mapping, provider adapters, rate limiter
• Integration tests (tests/integration/): HTTP endpoints, auth middleware, admin API, analytics API, chat completions validation

Tests use Vitest with supertest for HTTP testing. Environment variables are configured in tests/setup.ts.

---

🤝 Contributing

1. Fork → Branch → PR
2. npm install && npm run build — backend must compile clean
3. npm test — all tests must pass
4. cd client && npm install && npm run build — frontend must compile clean
5. New providers: implement ProviderAdapter interface, register in providers/index.ts
6. New intents: add to router/intent.ts + update model-map.ts DEFAULT_MATRIX
7. New dashboard pages: add to client/src/pages/ + register in App.tsx router

---

📄 License

MIT — use it however you want.

---

🏠 Homepage · 📦 GitHub · 📖 Full Docs · 🇨🇳 中文文档

Inspired by freellmapi · Built with 🧭 by xiaopengs