Skip to content

Initial version of model router#1

Merged
makosblade merged 1 commit into
mainfrom
drew/initial-push
May 4, 2026
Merged

Initial version of model router#1
makosblade merged 1 commit into
mainfrom
drew/initial-push

Conversation

@makosblade
Copy link
Copy Markdown
Contributor

@makosblade makosblade commented May 4, 2026

Add model router: cluster routing with multi-provider dispatch

Full Go service implementing intelligent LLM routing across Anthropic,
OpenAI, and Google providers. Routes requests per-prompt using a cluster scorer with a deterministic heuristic fallback.

Key components:

  • CLEAN-architecture layered service (auth, proxy, router, providers,
    translate) with strict inward-only imports
  • Cluster router with 9 versioned artifact bundles (v0.1–v0.9),
    multiversion serving, and eval-allowlisted per-request version pinning
  • Bidirectional wire-format translation (Anthropic <-> OpenAI SSE streams)
  • API-key auth with SHA-256 hashing, LRU caching, and soft-delete
  • Anthropic Messages + OpenAI Chat Completions inbound surfaces
  • Eval harness (Python/Modal) for comparing router strategies on
    RouterArena, LiveCodeBench, MMLU, GPQA, SWE-bench, and others
  • Training pipeline (train_cluster_router.py) for centroid clustering
    and α-blended quality/cost rankings
  • OpenTelemetry instrumentation, Postgres via SQLC, Docker Compose stack

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com
Co-Authored-By: Drew Bailey drew@workweave.ai
Co-Authored-By: steventohme steven@workweave.ai

Copy link
Copy Markdown
Contributor Author

This stack of pull requests is managed by Graphite. Learn more about stacking.

@makosblade makosblade requested a review from steventohme May 4, 2026 18:50
Comment thread assets/eval/README.md Outdated
Comment thread docs/architecture/ARCHITECTURE.md Outdated
Comment thread eval/_routerarena_official/__init__.py Outdated
Comment thread scripts/ingest/__init__.py Outdated
Comment thread .DS_Store Outdated
Comment thread AGENTS.md
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

keep internal? unsure to me tbh

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should probably keep it, that way if anyone contributes they'll follow the rest of the codebase better than they otherwise would have

@makosblade makosblade force-pushed the drew/initial-push branch from ec0b03d to 5f0995b Compare May 4, 2026 19:06
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Drew Bailey <drew@workweave.ai>
Co-Authored-By: steventohme <steven@workweave.ai>
@makosblade makosblade force-pushed the drew/initial-push branch from 5f0995b to 66c2ce7 Compare May 4, 2026 19:36
@makosblade makosblade marked this pull request as ready for review May 4, 2026 19:42
@makosblade makosblade merged commit 9bf0317 into main May 4, 2026
1 check passed
@steventohme steventohme deleted the drew/initial-push branch May 15, 2026 06:18
steventohme added a commit that referenced this pull request May 19, 2026
Adds a badge + callout noting Weave Router ranks #1 on the RouterArena
leaderboard (arXiv:2510.00202), and cites the paper in the same footnote
style as the existing Avengers-Pro reference.
steventohme added a commit that referenced this pull request May 19, 2026
Adds a badge + callout noting Weave Router ranks #1 on the RouterArena
leaderboard (arXiv:2510.00202), and cites the paper in the same footnote
style as the existing Avengers-Pro reference.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants