Skip to content

design provider multi api

sinrimin edited this page Oct 28, 2026 · 4 revisions

Provider multi-protocol model + user default models (#95)

Issue: #95 · Builds on: #92 (delegated approvals — this reworks fields #92 introduced while #92 is still unmerged; branch stacks on feat/92). Status: implemented (see Verification).

Motivation (trial feedback on #92)

  1. No platform-level default decision model existed. #92 wired approvals machine-scoped only (chatApprovals.providerId/modelId), so nothing answered "which decision model do my sessions use by default" — the provider page is where the user looked for it.
  2. One provider speaks several protocols. Bailian serves openai-chat, openai-responses, anthropic, embedding AND systemone (decision) off the same credential; baseUrl and key are identical, only the path differs. The single api field forced duplicate rows per wire.
  3. Advanced-field gating hung off the provider's api. With per-model wires, presets/warnings belong to the MODEL's apiKind — and irrelevant advanced fields should be HIDDEN (not just warned) on decision/embedding models.

Data model

type ProviderApiKind = 'openai-chat' | 'openai-responses' | 'anthropic'
                     | 'systemone' | 'openai-embedding';   // NEW: embedding

interface ProviderApiEndpoint { kind: ProviderApiKind; path: string | null }
interface LlmProvider { …; apis: ProviderApiEndpoint[]; baseUrl: string | null; … }
interface ProviderModelConfig { id: string; apiKind: ProviderApiKind; …advanced }
  • The row-level api is GONE. apis is the enabled-wire list (≥1, kinds unique); a model's apiKind must be one of them (400 MODEL_KIND_NOT_ENABLED, enforced in the service, prevented in the dialog).
  • path semantics: null/'' = the kind's default path; a /-prefixed string is appended to the shared baseUrl; a FULL http(s) URL escapes the baseUrl (Bailian's systemone lives off the compatible-mode root).
  • Default paths (PROVIDER_DEFAULT_PATHS) match what the SDKs appended pre-#95, so a path: null row behaves byte-identically to an old row: openai-chat /chat/completions · openai-responses /responses · anthropic /v1/messages · systemone /systemone (base carries /v1) · openai-embedding /embeddings. providerEndpointUrl(kind, {baseUrl, path}) (shared, pure) resolves; null = unresolvable (systemone with neither base nor full-URL path).
  • systemone among apis still requires an explicit baseUrl (SYSTEMONE_BASE_URL_REQUIRED kept).
  • The old reserved type: 'embedding' model slot became the openai-embedding KIND — a row with embedding models gains the wire entry.

sqlite migration v32

llm_providers: api TEXT → apis JSON (a row with embedding models gains the openai-embedding entry); every model's type rewritten in-place to apiKind (chat → the row's old api, embedding → openai-embedding, decision → systemone) via JSON1; the dead api column is dropped. users: new default_models JSON column. Unit-tested against a hand-built v31 schema (test/migrations-95.test.ts).

Consumers (the filter moved from row to model)

  • Runtime push (W3): a target's candidate models are those with apiKind ∈ PROVIDER_API_SUPPORT[target]; the spec flavor derives from the selected MODEL (providerApiToSpecApi(model.apiKind)). One Bailian row now serves claude (anthropic model), dsh/opencode (openai-chat), codex (openai-responses).
  • Platform assistant + notes-ai: candidate set = isChatApiKind(m.apiKind) (new shared type guard). buildGatewayModel takes the MODEL's kind.
  • #92 approvals gate: the decision model is the machine override pair when set, ELSE the user's default slot; requires model.apiKind === 'systemone'; the endpoint is providerEndpointUrl('systemone', …) — callSystemOne now takes the resolved URL (was baseUrl + /systemone).
  • Model discovery (query-models): stored rows derive the listing arm (openai arm if any openai-ish wire, else anthropic, else none for systemone-only rows → empty list, #92 behavior).
  • Presets/warnings (#56): SERVER_TOOL_PRESETS/EXTRA_BODY_PRESETS and providerModelConfigWarnings(models) key on each MODEL's apiKind.

User default models

  • User.defaultModels { embedding?, rerank?, decision? }, each slot {providerId, modelId}. embedding/rerank are reserved (schema accepts, UI disables; rerank saves are refused DEFAULT_MODELS_RERANK_RESERVED).
  • REST: GET/PATCH /api/users/me/default-models (self-scoped, full replace; {} clears). The decision slot validates provider visibility + a systemone model (DEFAULT_MODEL_PROVIDER_NOT_FOUND / DEFAULT_MODEL_KIND_MISMATCH).
  • Entry: the provider list page's Default models dialog (decision active, the two reserved slots visibly disabled).

Approvals wiring (the discoverability fix)

  • Machine chatApprovals.providerId/modelId are now OPTIONAL — omitting both = Follow platform default (follow the user's default decision model). Threshold / neverAllow / denyNote stay machine-scoped; when the machine row is absent entirely the gate falls back to threshold .9 / no floor / note on.
  • Resolution chain (server, realtime/chat.ts): machine override pair → user defaultModels.decision; both must still point at a visible provider with a live systemone model, else the session reads manual.
  • open() stays opt-in: a machine row without responder: 'delegated' opens manual even when a default exists. The session switch (chat:approvals.set) re-resolves on every flip. The axis is ALWAYS advertised (manual + no model included); with no resolvable model the switch renders DISABLED with a provider-defaults link, and a flip to delegated is refused CHAT_APPROVALS_NO_PROVIDER.

Web

  • Provider dialog: base section (name/baseUrl/credential) + an API endpoint editor (kind select of un-enabled kinds + path input with the default-path placeholder + resolved-URL preview + remove); models carry an apiKind select limited to enabled kinds; non-chat models HIDE the chat-only advanced + metadata fields (id + displayName remain). List: kind chips + a supports-kind filter.
  • Provider list header: Default models dialog (three slots, decision active).
  • Machine approvals card: provider select gains Follow platform default; the resolved default shows as a hint.
  • Runtime push card: model-level filtering (above).

Out of scope

rerank kind (slot reserved), per-kind credentials/baseUrls, OpenAI Decisions API adapter, org-level floors.

Verification (2026-10-27, all green)

  • Quartet on the Node 22.19 toolchain: pnpm -r build + typecheck + test green. server 498 tests (multi-wire service rules, default-models service, gate follow-default + no-machine-config, full-URL systemone path, sqlite v32 transform against a hand-built v31 schema); shared 139.
  • ui-snap: only the /llm-providers desktop shots differ (the redesigned list/dialog — intentional); baselines refreshed and synced back. The rig's provider seeds updated to the apis shape (they 400 otherwise now).
  • agent-browser E2E (throwaway memory-server rig, worktree build): multi-wire create dialog (3 wires + 3 per-kind models), kind-chip list row, Default models dialog (embedding/rerank visibly Reserved, decision slot round-trips), machine approvals card showing Follow platform default + the resolved default hint, and the claude-code runtime card correctly NOT offering the Bailian row (no anthropic-wire model — model-level filter). Live-session delegation itself was E2E-verified under #92; #95's delta is the resolution chain, covered by unit tests + the REST round-trip.
  • docker verification stack redeployed from the branch: the real sqlite ran the v32 migration (9 provider rows transformed in place, api column dropped, users.default_models added; server healthy, both containers carry the new code).
  • E2E ergonomics note (not fixed — trial feedback): in the create dialog, clicking a wire/model Select that sits below the fold can dismiss the whole dialog (the off-view coordinate lands on Radix's overlay); newly added rows should scrollIntoView on mount.

Trial-fix round (2026-10-28, commit c183dc1)

  • The switch was unreachable (trial: "switch invisible"): two layers. (a) open() advertised the slice only when a model resolved, and the web's only source was the daemon-synthesized session_config — alpha.8 never sent it. The server now advertises UNCONDITIONALLY and pushes the slice itself at ready/reattach/flip (4838fe7). (b) The daemon's history resync rebuilds the fold and re-emits session_config WITHOUT approvals on alpha.8, wiping the pushed slice; the fold now carries the config across the rebuild (880194b), and the stable listener buffers the reattach push. Verified on the deployed alpha.8 session: Permissions control visible, switch enabled with bailian-decision/decision-model-preview.
  • Global vertical scrollbar: mermaid parks a .mermaidTooltip div on <body> at ~100vh at render time; in the SPA it survives route changes and overflows the document on every page/skin. Pruned after bind + on unmount, with a base-CSS kill-switch (#mermaidTooltip, .mermaidTooltip { display: none }).
  • Rig lesson: restarting the API server mid-test orphaned daemon-side adapters that jammed the per-target prewarm pool — later opens stalled pre-ready until the zombies were reaped. Not a product bug, but the daemon could grow a stuck-adapter timeout (follow-up).

Clone this wiki locally