Model Router

Model Router

The Model Router enables multi-LLM support by analyzing GitHub issue complexity and routing tasks to the appropriate model. Simple tasks go to lightweight models (8B parameters), complex tasks go to heavyweight models (30B+).

How It Works

GitHub Issue
    ↓
Complexity Analysis (heuristic scoring 0-10)
    ↓
Score < threshold → "light" tier → 8B model
Score >= threshold → "heavy" tier → 30B model
    ↓
Selected model passed to DockerManager
    ↓
Minion spawned with the routed model

Complexity Scoring

Issues are scored from 0 to 10 based on these signals:

Signal	Score Impact	Details
Simple labels	-2	docs, typo, easy, good-first-issue, minor, documentation
Complex labels	+2	refactor, architecture, performance, security, breaking-change, multi-file
Long body (>2000 chars)	+2	Indicates detailed/complex requirements
Medium body (>800 chars)	+1	Moderate complexity
Checklist items (>=5)	+2	Many subtasks
Checklist items (>=2)	+1	Some subtasks
Code blocks (>=3)	+1	Multiple code examples
File references (>=5)	+2	Multi-file changes
File references (>=2)	+1	Some file changes
Complex title keywords	+2	"refactor", "redesign", "migrate", "rewrite", "overhaul"
Simple title keywords	-1	"fix typo", "update readme", "bump version", "rename"

The final score is clamped to the range 0-10.

Examples

Simple Issue (Score: 0, Tier: light)

Title: "Fix typo in README"
Labels: [docs, typo]
Body: "Line 42 has a spelling error"

Signals: simple labels (-2), simple title keyword (-1) → clamped to 0

Complex Issue (Score: 7, Tier: heavy)

Title: "Refactor authentication system"
Labels: [refactor, security]
Body: (2500 chars with 6 checklist items, 3 code blocks, 5 file refs)

Signals: complex labels (+2), long body (+2), checklist (+2), code blocks (+1), file refs (+2), complex title (+2) → clamped to 10, but realistic score ~7-9

Configuration

Enable Routing

ROUTING_ENABLED=true
ROUTING_COMPLEXITY_THRESHOLD=5    # Score threshold for heavy tier
ROUTING_DEFAULT_TIER=heavy        # Fallback when routing fails

Define Model Profiles

Model profiles are configured via a JSON environment variable:

ROUTING_MODELS='{
  "light": {
    "name": "llama-3.1-8b",
    "base_url": "http://localhost:5000/v1",
    "timeout": 300
  },
  "heavy": {
    "name": "qwen3-coder-30b",
    "base_url": "http://gpu-server:8080/v1",
    "timeout": 600
  }
}'

ModelProfile Fields

Field	Type	Default	Description
name	string	required	Model name
tier	string	required	"light" or "heavy"
base_url	string	""	LLM server URL (inherits default if empty)
timeout	int	600	Request timeout in seconds
streaming	bool	false	Enable streaming

Routing Logic

When routing is enabled:

1. Overlord calls router.select_model(title, body, labels)
2. Router computes complexity score
3. Score determines tier ("light" or "heavy")
4. Router looks up the ModelProfile for that tier
5. If no profile found, falls back to default_tier
6. If still no profile, returns None (uses default LLM config)
7. Selected model is passed to DockerManager.spawn_minion()

When routing is disabled (default):

• select_model() returns None
• DockerManager uses the global LLMConfig for all minions

Integration Points

The router is used at two dispatch points in the Overlord:

1. Slack command handler: Fetches issue details via GitHubQueue.get_issue_details(), then routes
2. Cron queue sweep: Uses the QueuedIssue.body field already fetched during scanning

Slack notifications include the selected model name:

🚀 Spawning minion `minion-abc123` to work on owner/repo#42 (model: `llama-3.1-8b`)

Architecture

nebulus_swarm/
├── config.py                  # ModelProfile, RoutingConfig dataclasses
└── overlord/
    ├── model_router.py        # ModelRouter, ComplexityScore
    ├── main.py                # Integration at both dispatch points
    └── docker_manager.py      # spawn_minion() accepts model_override

Related Pages

• Nebulus Swarm - System overview
• Swarm Overlord - Where routing decisions happen
• Configuration - Environment variables