RouteAI

LLM request router — route prompts to the optimal model based on complexity, cost, and latency.

How It Works

flowchart LR
    A[Prompt] --> B[ComplexityAnalyzer]
    B --> C{RoutingStrategy}
    C -->|cost_optimized| D[Cheapest Model]
    C -->|quality_optimized| E[Best Model]
    C -->|balanced| F[Weighted Blend]
    C -->|latency_optimized| G[Fastest Model]
    D & E & F & G --> H[Selected Model]
    H --> I[RequestLog]

Loading

Features

• Smart routing — analyse prompt complexity and pick the best model automatically.
• Four strategies — cost-optimized, quality-optimized, latency-optimized, and balanced.
• Budget constraints — cap cost-per-1k-tokens to stay within budget.
• Capability filtering — require specific capabilities (chat, code, vision, reasoning).
• Fallback chains — try models in order; gracefully fall back on failure.
• Request logging — track every routing decision for cost analysis and observability.

Quickstart

Install

pip install -e .

Python API

from routeai import ModelRouter, Model, RoutingStrategy

router = ModelRouter(
    models=[
        Model("gpt-4o", "openai", cost_per_1k_tokens=0.005,
              max_tokens=128000, latency_ms=1200,
              capabilities=["chat", "code", "reasoning"], quality_score=0.92),
        Model("claude-haiku-3.5", "anthropic", cost_per_1k_tokens=0.00025,
              max_tokens=200000, latency_ms=400,
              capabilities=["chat", "code"], quality_score=0.75),
    ],
    strategy=RoutingStrategy.BALANCED,
    max_budget_per_1k=0.01,
)

model, log = router.select("Explain quantum computing in simple terms")
print(f"Use {model.name} — est. cost ${log.estimated_cost:.6f}")

CLI

# Route a prompt
python -m routeai route "Explain recursion" --strategy balanced

# List preset models
python -m routeai models

Fallback Chain

from routeai import FallbackChain, Model

chain = FallbackChain([primary_model, fallback_model])
response, used_model, was_fallback = chain.execute(
    "Hello", call_fn=my_llm_call
)

Project Structure

RouteAI/
├── src/routeai/
│   ├── __init__.py      # Public API
│   ├── core.py          # Router, models, strategies, analyser
│   ├── config.py        # Preset models & defaults
│   ├── utils.py         # Serialisation & formatting helpers
│   └── __main__.py      # CLI entry-point
├── tests/
│   └── test_core.py     # Test suite
├── docs/
│   └── ARCHITECTURE.md  # Design documentation
├── pyproject.toml
├── Makefile
└── .github/workflows/ci.yml

License

MIT — see LICENSE.

---

Built by Officethree Technologies | Made with ❤️ and AI