feat(routing): add opt-in knn task-level model routing backend#163
Merged
Conversation
Extend RoutingConfig with a backend switch (ecoclaw | knn) and knn-specific fields (models, memory_path, k, lambda_cost, embedding_endpoint), plus a ModelEndpoint submodel. Additive with defaults; existing fields and the default ecoclaw backend keep current behavior. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
PerModelProvider dispatches each call to a per-model CustomProvider by model name; unknown models fall back to a wrapped provider unchanged. Used by the knn routing backend where routable models live on different OpenAI-compatible endpoints. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
KNNModelRouter embeds the incoming task, retrieves the k nearest training tasks from a prebuilt memory, and picks the model with the best expected value (reward - lambda_cost * cost) over those neighbours. Same select_model_chain interface as the ecoclaw ModelRouter; any failure returns (None, []) so the caller uses the default model. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
Extract build_model_routing() and pick the router by config.routing.backend: knn builds a KNNModelRouter and wraps the provider in PerModelProvider; ecoclaw keeps the PinchBench ModelRouter. Routing stays opt-in and off by default; the agent loop is unchanged. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
Leave the configured default model only with enough evidence, so an off-distribution query (e.g. casual chat) or a cold-start memory keeps the default instead of switching on a weak match. Score the pick over the similar neighbours (cosine >= min_similarity) rather than a fixed top-k. Gates: memory size, count of similar neighbours, and margin over the default model's score. Defaults keep routing safe out of the box (k=30, min_similarity=0.6, min_similar_neighbors=4, min_memory_size=10); KNNModelRouter takes the agent default model so the caller's model is the fallback. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
Drive AgentLoop._process_message with a real KNNModelRouter + PerModelProvider over two OpenRouter models: the simple task stays on the cheap default, the hard task switches to the strong model, and the routed model returns a real answer. Self-skips unless an embedding endpoint and OPENROUTER_API_KEY are available; sandbox/MCP bring-up is stubbed so a text turn runs without a VM. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
Contributor
|
LGTM overall — CI is green and I ran the unit suite locally One non-blocking nit before merge: |
Memory entries can store 'text' instead of a precomputed 'embedding'; the router embeds each at load via the configured endpoint (batch=1, cached under ~/.raven), so the file stays small and its vectors stay consistent with live queries. Precomputed embeddings are still honoured. Also score reward and cost over the same neighbours, and wrap the KNN path so any routing error degrades to the default model instead of failing the turn. Ships a small example memory (ScienceWorld + AppWorld). Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
raven/routing/knn_router.py imports numpy directly; it was only present transitively. Declare it so the dependency is explicit and the lockfile stays honest. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
Add the trailing newline end-of-file-fixer expects on the shipped example memory (was failing the pre-commit CI check), and note why numpy is a direct dependency, mirroring the tiktoken comment, per review. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
Bound the direct numpy dependency instead of leaving it unversioned, matching the repo convention where every dependency carries a range. The lower bound clears both lancedb (>=1.24.0) and everalgo-clustering (>=1.26); the <3.0 cap surfaces a future numpy 3.x bump loudly rather than silently pulling an untested major. Resolved numpy stays at 2.4.4. Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
0xKT
approved these changes
Jul 21, 2026
0xKT
added a commit
that referenced
this pull request
Jul 21, 2026
## Summary Bump version from 0.1.6 to 0.1.7 for the next release. Touches `pyproject.toml` and `uv.lock` only. Release-scope changes since v0.1.6: - feat(agent): offer deep-vs-regular choice for deep_research (#168) - feat(routing): add opt-in knn task-level model routing backend (#163) - fix(tracing): show model-call reasoning above the answer, not below (#154) - fix(tui): announce log path only on abnormal tui exit (#166) - docs: update readme / chinese readme for recent features (#159, #160) ## Type - [ ] Fix - [ ] Feature - [ ] Docs - [ ] CI / tooling - [ ] Refactor - [x] Other ## Verification - [x] Relevant lint / type checks pass locally ``` uv lock # resolved, raven 0.1.6 -> 0.1.7 uv run python scripts/check_commit_messages.py origin/main..HEAD # exit 0 ``` ## Risk - [x] Backward compatibility considered - [x] Rollback path is clear for risky changes Version-only bump; no code changes. ## Related Issues N/A Co-authored-by: Claude (claude-opus-4-8) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional task-level KNN model-routing backend alongside the existing EcoClaw router. At the start of each turn the router embeds the task, retrieves the nearest training tasks from a prebuilt memory, and picks the model with the best expected value (reward - lambda_cost * cost). Routing is opt-in and off by default (routing.enabled = false); when disabled the agent loop is unchanged.
Commit by commit:
Type
Verification
Commands and results:
Offline routing reproduction against a prebuilt memory (the algorithm makes the live decision; the routed model's outcome is looked up from stored per-model trajectories):
ScienceWorld: KNN 58.47 vs Pure 27B 57.99 at ~13% lower cost.
AppWorld: KNN 55.3% pass vs Pure 27b 50.9%.
Relevant tests pass locally
Relevant lint / type checks pass locally
User-facing docs or screenshots are updated when needed
Risk
Off by default: with routing.enabled = false, build_model_routing returns the provider unchanged and the loop's routing block is skipped, so behaviour is identical to before. The new provider and router classes are only imported when the knn backend is enabled. Rollback is to disable routing in config or revert the branch.
Related Issues
N/A