Skip to content

Replace hardcoded Auto model with real model routing #570

Description

@urjitc

Summary

Make the Auto model picker option actually route requests instead of being a single hardcoded backend model.

Current repo state

  • DEFAULT_WORKSPACE_AI_CHAT_MODEL_ID is "auto".
  • src/features/workspaces/ai/models.ts defines Auto as a normal picker entry with gatewayModel: "moonshotai/kimi-k2.6".
  • The inline comment says Auto is Kimi K2.6 under the hood until ThinkEx builds or adopts a real router.
  • getWorkspaceAiChatModel(modelId) currently resolves every picker choice, including auto, to one concrete gateway model string.
  • Runtime paths such as beforeTurn, compaction, telemetry, and provider options all receive or derive a resolved model id, but there is no task-aware selection step for Auto.
  • The model picker describes Auto as "Picks a good fit for you", which overstates the current behavior.

Desired direction

When users select Auto, ThinkEx should choose a model based on the actual request, workspace context, tool needs, cost/performance policy, and user/account limits instead of always calling the same model.

Scope to design/build

  • Define the first routing policy for Auto:
    • simple/short tasks -> fast/cheap model,
    • harder reasoning/research/writing -> stronger model,
    • long-context document/PDF tasks -> model with suitable context and reliability,
    • tool-heavy or code/data tasks -> model proven to follow tool contracts well.
  • Decide whether routing happens before the turn in AIThread.beforeTurn, inside getWorkspaceAiLanguageModel, or through provider/gateway routing options.
  • Keep the user-facing model id as auto, but record the actual selected gateway model for telemetry, billing, inspector, and debugging.
  • Add a policy surface that is easy to tune without rewriting chat runtime code.
  • Consider plan/usage limits: Auto should not silently route standard users into premium-priced models unless access/billing rules allow it.
  • Update picker copy so Auto accurately describes what it does.
  • Decide fallback behavior when the preferred routed model is unavailable.

Open questions

  • Should the first version be rule-based heuristics, gateway/provider routing, or a small classifier prompt?
  • What input features should the router use: prompt length, attached files, visible workspace context, selected item types, requested mode, tools enabled, or prior failures?
  • Should users see which model Auto selected after the response?
  • How should routed model choice affect billing tier checks and usage accounting?
  • How do we test and observe routing decisions without leaking prompt/content into logs unnecessarily?

Acceptance criteria

  • Auto no longer resolves to a single static gatewayModel for every chat turn.
  • The actual selected gateway model is captured in telemetry/inspector and billing paths.
  • Access checks run against the selected model or a clearly documented Auto policy.
  • Picker copy matches the implemented behavior.
  • The router has tests or fixtures covering at least simple, long-context, tool-heavy, and high-reasoning examples.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    In progress

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions