Adding a chat model plugin hook for model routing based on complexity#18845
Closed
rjmorales13 wants to merge 5 commits intoanomalyco:devfrom
Closed
Adding a chat model plugin hook for model routing based on complexity#18845rjmorales13 wants to merge 5 commits intoanomalyco:devfrom
rjmorales13 wants to merge 5 commits intoanomalyco:devfrom
Conversation
… savings Adds a new 'chat.model' plugin hook that allows plugins to dynamically route messages to different models based on content/complexity. When a plugin sets output.model, the routed model is persisted in session state so future turns and subagents (via Task tool) inherit it. Changes: - packages/plugin/src/index.ts: Add chat.model hook type definition - packages/opencode/src/session/prompt.ts: Fire chat.model hook before model resolution, persist routed model for subagent inheritance - packages/opencode/src/tool/task.ts: Check for routed model when resolving subagent model (fixes subagent model inheritance) Related: anomalyco#18644, anomalyco#17870, anomalyco#6928
Convert plain strings from plugin hook to branded types using ProviderID.make() and ModelID.make(). Cast branded types to plain strings when sending to the hook interface. Fixes TS2345/TS2322 typecheck errors.
Allows plugins to make content-based routing decisions by receiving the user's message text in the chat.model hook. Previously plugins could only route by agent name. Now plugins can analyze the actual prompt to route simple tasks to free models and complex tasks to premium models.
Contributor
|
Hey! Your PR title Please update it to start with one of:
Where See CONTRIBUTING.md for details. |
github-actions
bot
added
the
needs:compliance
Contributor
|
This PR doesn't fully meet our contributing guidelines and PR template. What needs to be fixed:
Please edit this PR description to address the above within 2 hours, or it will be automatically closed. If you believe this was flagged incorrectly, please let a maintainer know. |
Contributor
|
The following comment was made by an LLM, it may be inaccurate: Based on my search, the only PR that appears in all results is PR #18845 itself (the current PR). The searches returned various feature and provider-related PRs, but none of them appear to be duplicates of the "Feat/plugin chat model hook" PR. No duplicate PRs found |
rjmorales13
changed the title
Core changes (3 files, 55 lines): - packages/plugin/src/index.ts: chat.model hook type with message field - packages/opencode/src/session/prompt.ts: fire hook, persist routed model - packages/opencode/src/tool/task.ts: subagent model inheritance Reference plugin (1 file): - packages/opencode/examples/plugins/token-optimizer.ts: cascading confidence router Plugin features: - Stage 1: Zero-cost heuristics (catches 60% - greetings, short queries) - Stage 2: Complexity estimation with caching (catches 35%) - Stage 3: Config-based routing with threshold (remaining 5%) - Per-prompt overrides: !premium, !free, !think - De-escalation: simple tasks on premium → route to free - Escalation: complex tasks on free → route to premium - Heartbeat fallback: Ollama slow → Zen/OpenRouter free - Config: TOKEN_OPTIMIZER_FREE_MODEL, TOKEN_OPTIMIZER_PREMIUM_MODEL Tested: - De-escalation: hello on Claude → routed to Ollama - Escalation: implement JWT auth → routed to Claude via OpenRouter - Override: !premium skips routing - Subagent inheritance: Task tool inherits routed model Related: anomalyco#18844, anomalyco#18667, anomalyco#18666, anomalyco#18644, anomalyco#17870, anomalyco#6928
Documents implementation status, testing results, and pending items. Testing for new changes is pending — will continue tomorrow.
Contributor
|
This pull request has been automatically closed because it was not updated to meet our contributing guidelines within the 2-hour window. Feel free to open a new pull request that follows our guidelines. |
github-actions
bot
removed
the
needs:compliance
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR: #18844
Closes #
Type of change
What does this PR do?
Please provide a description of the issue, the changes you made to fix it, and why they work. It is expected that you understand why your changes work and if you do not understand why at least say as much so a maintainer knows how much to value the PR.
If you paste a large clearly AI generated description here your PR may be IGNORED or CLOSED!
How did you verify your code works?
Screenshots / recordings
If this is a UI change, please include a screenshot or recording.
Checklist
If you do not follow this template your PR will be automatically rejected.