Implement Adaptive Complexity Routing: AI Team's Democratization Architecture

[joelteply]

Vision: Democratizing AI Through Intelligent Model Routing

The AI team (Claude, DeepSeek, Groq, Fireworks, Together) collaboratively designed an Adaptive Complexity Routing architecture during their 2025-11-21 chat session. This aligns perfectly with our core mission: preventing AI overspend while democratizing access to advanced agent systems on commodity M1+ hardware.

Architecture Document

📄 Full Design: ADAPTIVE-COMPLEXITY-ROUTING.md

Core Innovation: Progressive Model Upgrading

The Problem:

• Simple messages waste money on expensive APIs (GPT-4, Claude Sonnet)
• Complex messages get routed to fast but inadequate models (Groq Lightning)
• No progressive reassessment during generation
• Users forced to choose between cost and quality

The Breakthrough:
Dynamic complexity assessment with mid-stream model upgrading:

1. Start with cheap/free local models (Ollama)
2. Assess complexity every 200 tokens during generation
3. Upgrade to capable models only when needed
4. Preserve context across transitions

THE SPIKE: Engineering Rigor

Critical Innovation: Time-boxed technical investigation (the "spike") to validate assumptions BEFORE full implementation.

Spike validates:

• ✅ Can we hot-swap models without losing context?
• ✅ Is latency acceptable (< 500ms)?
• ✅ Which providers support mid-stream upgrades?
• ✅ Do we need pre-warmed model pools?

Run spike DURING Phase 1 (not after) to pivot early if needed.

Cost & Democratization Impact

Before: 100% premium APIs → $100+/month
After: 80% local Ollama + 15% cheap APIs + 5% premium → $5-10/month
Savings: 90%+ while maintaining quality

M1+ Hardware Support:

• M1 Air: 7B models (qwen2.5, llama3.2) - FREE
• M1 Pro/Max: 70B models (llama3.1, deepseek-coder) - FREE
• M1 Ultra: Multiple models simultaneously - FREE

Implementation Phases

Phase 1: Complexity Assessment Foundation ✅ START HERE

• Complexity assessment engine (straightforward/moderate/nuanced)
• ComplexityLevel classification with hybrid approach (heuristics + LLM)
• Model routing map (local-fast → ollama-capable → api-cheap → api-premium)
• Integration with PersonaMessageEvaluator

Key Files:

• system/user/server/modules/ComplexityAssessor.ts
• system/user/server/modules/ModelRouter.ts
• system/shared/ModelTiers.ts

Phase 2: Progressive Scoring System

• ProgressiveScorer class with token-window analysis
• Upgrade indicator detection (hedging, uncertainty, self-correction)
• Streaming wrapper for generateWithProgressiveScoring()
• Integration with AIProviderDaemon

Key Files:

• system/user/server/modules/ProgressiveScorer.ts
• daemons/ai-provider-daemon/shared/StreamingWrapper.ts

Phase 3: THE SPIKE - Upgrade Feasibility

• Test context preservation (Ollama local-to-local)
• Measure upgrade latency (target: < 500ms)
• Test provider compatibility (OpenAI, Anthropic, DeepSeek)
• Identify blockers and edge cases
• Generate SpikeFinding report with recommendations

Spike Script:

npx tsx tests/spikes/model-upgrade-spike.ts

Decision Matrix (after spike):

• Feasible → hot-swap implementation
• High latency → graceful-restart at turn boundaries
• Context loss → pre-warmed model pools

Phase 4: Upgrade Mechanism Implementation

• UpgradeStrategy based on spike recommendations
• upgradeModel() function with chosen mechanism
• Fallback handling for failed upgrades
• Context preservation protocol via ResponseContext

Key Files:

• system/user/server/modules/ModelUpgrader.ts
• system/user/server/modules/ResponseContext.ts

Phase 5: Analytics and Optimization

• Cost tracking (API spend vs local usage)
• Accuracy metrics (did routing predict correctly?)
• Performance dashboards
• Tuning thresholds based on real data

Commands:

./jtag routing/analytics --userId="joel" --timeRange="30d"

Integration with PersonaUser Convergence

The universal cognitive cycle enhanced:

async serviceInbox() {
  const tasks = await this.inbox.peek(10);
  if (tasks.length === 0) return await this.rest();

  // 🆕 ADAPTIVE ROUTING
  const complexity = await this.routing.assess(task);
  const tier = this.routing.selectTier(complexity);

  // Activate skill (LoRA adapter + base model)
  await this.genome.activateSkill(task.domain, tier);

  // 🆕 PROGRESSIVE SCORING
  await this.processTaskWithUpgrade(task, tier);

  await this.state.recordActivity(task.duration, task.complexity);
}

Success Metrics

Cost Reduction:

• Target: < $0.001/message avg
• Target: > 90% savings vs all-premium

Quality Maintenance:

• Target: > 90% routing accuracy
• Maintains response quality

Democratization:

• Target: > 80% local model usage
• Works on M1+

Why This Matters

Not just cost savings - this enables:

• Local-first architecture (privacy, control, ownership)
• Continuous learning through fine-tuning
• Market disruption of cloud API monopolies
• PersonaUsers as autonomous citizens, not API endpoints

Related Documents

• PERSONA-CONVERGENCE-ROADMAP.md
• LORA-GENOME-PAGING.md
• AUTONOMOUS-LOOP-ROADMAP.md

---

Design Session: 2025-11-21 collaborative AI team chat
AI Team: Claude, DeepSeek, Groq, Fireworks, Together
Concept: The "spike" - validate assumptions early, pivot if needed

[github-actions]

This issue is stale because it has been open for 60 days with no activity. Please comment or update this issue to keep it active.

[github-actions]

This issue was closed because it has been inactive for 30 days since being marked as stale.