SmartRoute: A Training-Free Semantic Router for LLM Cascades

SmartRoute helps developers reduce LLM costs by 47% by proactively routing simple tasks to cheap models and complex tasks to frontier models.

---

🚀 Quick Start

1. Installation

pip install git+https://github.com/psantanusaha/smartroute.git

2. Setup Environment

Create a .env file with your API keys (see .env.example).

3. Usage (Single Provider Example)

You can try SmartRoute with just one API key (e.g., OpenAI) by using different model tiers.

import os
from smartroute import SmartRouter
from dotenv import load_dotenv

load_dotenv()

# Configure tiers using a single provider
router = SmartRouter(
    cheap_config={
        "provider": "openai", 
        "model": "gpt-4o-mini", 
        "api_key": os.getenv("OPENAI_API_KEY")
    },
    expensive_config={
        "provider": "openai", 
        "model": "gpt-4o", 
        "api_key": os.getenv("OPENAI_API_KEY")
    },
    verbose=True # See routing decisions in real-time
)

# Routes to 'cheap' tier
res1 = router.generate("What is 2+2?")

# Routes to 'expensive' tier
res2 = router.generate("Design a sharded database architecture for 10M users.")

---

📊 Default Skill Taxonomy

SmartRoute classifies every prompt into one of these 12 categories to decide the routing tier:

Skill Category	Default Tier	Description
factual_qa	Cheap	Simple lookups, general knowledge
summarization	Cheap	Condensing text, key points
basic_code	Cheap	Single-function scripts, syntax
creative_simple	Cheap	Short emails, social posts
multi_step_reasoning	Mid	Math puzzles, word problems
complex_code	Mid	System design, architecture
data_analysis	Mid	Statistical reasoning, SQL
nuanced_creative	Mid	Voice, tone, long-form narrative
multi_constraint	Mid	Optimization, scheduling
formal_reasoning	Expensive	Mathematical proofs, logic
agentic	Expensive	Multi-tool workflows, automation
ambiguous_open	Mid	Subjective/Philosophical

---

🛠️ Observability

Trust is built through transparency. Use verbose=True to see exactly how SmartRoute handles your traffic:

[SmartRoute] Classifying prompt intent...
[SmartRoute] Detected skill: complex_code
[SmartRoute] Routing to MID tier (Model: llama-3.3-70b-versatile)

---

📂 Research & Benchmarks

For detailed data proving the 47% savings and 100% failure recall, visit the SmartRoute-Cascade Research Hub.

📜 License

MIT