Repository navigation
Replies: 2 comments
|
Alignment with #1658 (owner identity seam, now on
Browser-stored provider keys and model choices would move into the per-owner settings store in this RFC's P2, once encrypted storage (P1) exists. Course data is migrated earlier, silently in the background, as part of the persistence work. |
|
Resolution for the Browserless APIs use the same slot configuration as the UI. The Request fields are removed outright, with no deprecation window. This covers:
The request carries only generation input. Correspondingly, the line under "Compatibility and migration" that keeps Capability discovery for API callers comes from slot resolution. It no longer comes from scanning provider lists as The |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Summary
This RFC reduces how OpenMAIC connects to models to three concepts:
Whether a capability is available follows from whether its slot resolves to an assignment. A token plan is an ordinary provider that happens to cover many slots.
All configuration lives on the server, and each source has one job: operators write
openmaic.yml(server defaults, plus what they choose to lock), users edit in the web UI and changes are stored in the database, and.envonly holds secrets and boot parameters. Changes to the yml take effect on restart. The browser no longer stores keys, and requests no longer carry keys or provider choices.The goal is a Docker deployment that runs from a single command: zero configuration goes through a first-run wizard, and a real deployment mounts one yml file. Single-user and multi-user deployments use the same mechanism and differ by one policy switch.
Background and problems
OpenMAIC started as a client-side, single-user tool: the person using it, paying for it and configuring it was the same person, and settings lived in the browser. Server persistence is now the only durable store for courses, media and folders (#1658, landed in #1710), so these roles separate. Settings did not move with them: every provider key still lives in the browser and travels with each request. There are six concrete problems today:
server-providers.ymlis the base, environment variables override it field by field, and the browser overrides again per request through headers such asx-api-key,x-modelandx-image-provider. No single place decides which model a call uses or where that choice came from.DEFAULT_MODELand a JSON string inMODEL_ROUTES..env.examplealready has 131 variables, and an operator'sMODEL_ROUTESsilently overrides the user's per-stage choices.apply-token-plan.ts, 766 lines). It also only exists in the browser, so operators cannot use it.imageGenerationEnabled; server flags<CAP>_<VENDOR>_ENABLED=falsethat can only turn off one vendor; andNEXT_PUBLIC_*values baked in at build time, which a prebuilt image cannot change.Scope
This RFC answers one question: when the system calls an AI capability, which vendor, which model, and whose key. The following topics are out of scope and have their own homes:
asrslot)Moving each capability's execution dispatch from
switchstatements to a registry is orthogonal engineering work and can proceed in parallel.Core model
There are only three concepts; everything else follows from them.
Provider: an account that can be called, made of
{id, preset, baseUrl, apiKey}. Which capabilities and models it offers is described by a preset in the built-in registry, not stored in user data. Direct vendors, aggregator gateways, token plans and custom OpenAI-compatible endpoints are all providers; they only differ in how many capabilities they cover.Capability slot: a product use that needs AI. Slots form a forest with one tree per capability type, and each root is the default for that capability. A dotted child slot's parent is the slot with its last segment removed; top-level LLM use slots have
llmas their parent.llmcourse.researchllmweb-search-query-rewritecourse.outlinellmscene-outlines-streamcourse.agentsllmagent-profilescourse.contentllmscene-contentcourse.content.slide/.interactive/.quiz/.pblcourse.contentscene-content, split by scene typecourse.actionsllmscene-actionsclassroomllmchat-adapter,quiz-grade,pbl-chat,pbl-v2-runtime*agentllmmaic-agent,maic-agent-driveragent.titleagentconversation-title(Pro session titles); yml only, not shown in the UItts/asr/image/video/webSearch/documentThe course-related LLM slots map one-to-one to the stations in the Course Model settings (web research, outline, agents, scene content, scene actions, classroom), so the UI, the yml and the database all use the same names.
The test for whether a use gets its own slot is: would a user switch models for this use on its own? If yes, it gets a slot. Purely technical splits (such as the four PBL runtime endpoints, or title generation) are folded into a slot in code. The 21 existing stage keys become an internal "stage → slot" mapping in code, where each stage belongs to exactly one slot; they no longer appear in configuration or the UI. This also replaces the current "station → stage keys" alignment table in the settings UI (
station-stage-keys.ts) and the class of bugs where a missed key silently falls back to the main model.Assignment: slot →
providerId:modelId(optionally with call parameters such as thinking, and afallbackmodel), or an explicitnull. A provider without models to pick (most search and document services) is referenced by its id alone (webSearch: tavily), which means its default model; chat slots always name a model. The fallback is the model to retry once on a retryable failure, as introduced in #1614; it belongs to the assignment and is inherited with it.Inheritance rules
nullstops there and makes the whole subtree unavailable.agentrequires tool calling;agent.titledoes not, so it can use a cheap model.)locknames nodes, and a locked node fixes its whole subtree. Writing a node underslotsdoes not lock it; it sets the server default.Three consequences:
*Enabledflags. An operator turns a capability off by default by writingnullon its node, and keeps it off for everyone by also listing it inlock. Capability switches live on the cards in the Course Model settings; the generation composer no longer has them.course.content.slide). "One-click connect" means the first-run wizard adds the provider and fills the slots it covers that are still unassigned; after that it is an ordinary provider. Enrollment markers, seed fingerprints, yielding and restore-on-disconnect all go away.Configuration sources
Each source has one job, and they no longer override each other field by field.
openmaic.yml.env/ SecretThere are two rules: what the yml writes under
slotsis the server default, which users may change in the UI; what it names underlockis fixed together with its subtree, shown as "fixed by the administrator" and read-only. Everything a user changes is stored in the database..envis only referenced from the yml through${VAR}and no longer overrides any field directly.slotshas no server default and is left to the UI.locknames slots written underslots(orlock: all); a locked slot whose subtree resolves to nothing is refused at startup. A locked node also fixes its children: a user cannot set a child of a locked slot.${VAR}fails startup with the exact location, instead of being skipped silently.Resolution and security
Every AI call goes through one server-side resolver:
resolveSlot(workspace, slot) → {provider, model, params, source}.scene-contentwith scene type slide →course.content.slide.Otherwise the user's choice wins anywhere on the lineage: walk up the tree through the workspace's assignments first; only if none is found, walk up through the server defaults (the yml, or translated legacy variables). Return on the first assignment; an explicit
nullstops either walk. Server values are only defaults: whatever a user changes wins, and a step that follows its parent follows the user's choice on that parent.sourcerecords where the value came from (server default, workspace, or fixed by the administrator), and the settings UI uses it to label each card.Requests no longer carry keys or provider choices:
x-api-key,x-base-url,x-model,x-model-routes,x-image-provider,x-video-providerand the equivalent body fields are removed. Generation requests only carry generation options. Background jobs can reach credentials after the browser is closed.Key storage and display:
OPENMAIC_SECRET_KEY. If unset, one is generated on first start and written to the data volume; if it is lost, stored keys must be re-entered.sk-…abcd) and connectivity status.Browserless APIs (such as
/api/generate-classroom) resolve against the caller's workspace and behave the same as the UI.Frontend and settings UI
The frontend holds no configuration. It is a viewer and editor for the server's slot tree. The first purpose of the UI is to make routing readable: after connecting a token plan, you can see at a glance which model each step uses. Changing it comes second, and most users never will.
API
What the settings show: only what the user can change
Users never see slots, locks or policies as concepts. The settings take one of three shapes, derived from the configuration:
allowUserKeysis on (the default)allowUserKeys: falselock: all)One rule decides every control: show it only if using it can change something. Adding a service for a capability appears only when that capability still has a slot the user may set; Token Plan appears only when keys are allowed and at least one slot the plan recommends is not locked. A locked card collapses to one read-only line with a lock and "fixed by the administrator"; a card on a server default says so and offers "reset to server default" once the user has changed it.
Course Model settings: a zoomable flow diagram, read-first
llmroot, where the default model is set. Every LLM step links to it. This replaces the main-model control currently at the top of the panel.null: the line shows "off" and the flow arrow turns dashed. If no provider offers a capability at all, there is no switch, only "not configured". The generation composer no longer has capability switches.asr) is not part of the generation pipeline and does not follow the default chat model. It is a separate card in the top-right corner with no inheritance line; its switch and model live on that card. The classroom card only configures its chat model.agent.title) and follow the Pro agent by default, matching the current code, which reuses the agent's connection; they are not shown separately in the UI.Where current controls go
llmroot)null"llmroot; writes the same value as the settings UI, with no "this generation only" mode. Shown only whenllmcan be changed: hidden when it is lockedRemoved from the frontend as a result: capability switches in the composer, keys and the seven provider-state slices in the browser, request headers such as
x-model/x-model-routes/x-api-key, global*Enabledflags, thestation-stage-keys.tsalignment table, and token plan enrollment / fingerprint / yielding / restore logic.Deployment journeys
The primary target is Docker. Both ways of deploying land on the same model:
docker compose upopenmaic.ymlwith providers andslotsopenmaic.ymlwith providers,slots,allowUserKeys: false, optionally somelockopenmaic.ymlwithlock: allSingle user
docker compose up. Since feat(persistence)!: server-backed persistence only, with a one-way legacy browser import #1710 this starts a bundled PostgreSQL and runs in single-user owner mode: every request resolves to one fixed owner, so the configuration survives browser changes.llmassignment, the first-run wizard opens: pick a token plan and enter its key, or pick a vendor / custom endpoint. After the connectivity check passes, the slots it covers are filled automatically.Only the first-run wizard fills slots from a preset. Slots in the yml are written out explicitly; they are defaults unless
locknames them. A public deployment can setACCESS_CODEto keep strangers out.Multi-user
allowUserKeys: false, lock what must not change, and configure identity per [RFC] Owner / identity seam and server-only persistence (tracking) #1658.allowUserKeys: true, users can also add providers in their own workspace, effective only for themselves.A team sharing one workspace behaves like a single user: members share one set of web settings.
Compatibility and migration
Deployments that only configure providers and a default model upgrade without changing their configuration; the old forms are kept for two minor versions and then removed. Per-stage routing is the exception: it has to be rewritten as slots.
MINIMAX_API_KEY,TTS_MINIMAX_API_KEY,DEFAULT_MODEL,MODEL_FALLBACK, …): translated at startup into the equivalent yml structure (providers,DEFAULT_MODELas thellmdefault,MODEL_FALLBACKas its fallback; nothing translated is locked), with a deprecation notice.<CAP>_<VENDOR>_ENABLED=falsehas no equivalent and is reported. Whenopenmaic.ymlexists, the old variables are ignored with a notice.MODEL_ROUTES: not translated. Per-stage routes do not map one to one onto slots (several stages share a slot, the agent driver and retries follow rules of their own), so a deployment that setsMODEL_ROUTESwithoutopenmaic.ymlfails at startup with a message asking for the per-stage models as slots.server-providers.yml: translated the same way, with a prompt to migrate toopenmaic.yml.llmassignment (per-stage routes are not imported, for the same reason asMODEL_ROUTES; they are set again on the model diagram), and token plan enrollment becomes an ordinary provider. Keys are cleared from the browser after import.x-api-keyand friends are kept for one version, only for browserless API compatibility, and responses mark them deprecated.NEXT_PUBLIC_*to runtime reads is out of scope here, but it is a prerequisite for prebuilt Docker images and should land in the same timeframe.stageRoutesin token plan presets are translated through the "stage → slot" mapping, since the presets are maintained in this repository and can be adjusted where stages merge.Phases
Work happens on the
integration/provider-configbranch (tracking: #1725), in four phases that can each be reviewed on their own.openmaic.ymlschema and the "stage → slot" mapping; implementresolveSlot; translate old provider variables,server-providers.yml,DEFAULT_MODELandMODEL_FALLBACKinto the new structureresolveSlot; request headers enter deprecationserver-providers.ymland request headersIn parallel: move each capability's execution dispatch from
switchto a registry (TTS and web search first), with no dependency on the phases above.Open questions and non-goals
Open questions
generate-classroomanddirector) needs to be checked so that every stage has exactly one slot, and we need to confirm thatagent(withagent.title) is enough outside the pipeline.Resolved
slotsin the yml are defaults, and operators fix what must not change withlock(Oct 3 revision).Non-goals
Feedback on the slot tree, the precedence rule and the phases is especially welcome.
Revision history
Oct 3: configuration modes, for 1.2.0 (before
openmaic.ymlis first released). Writing a slot in the yml no longer locks it:slotsare server defaults users may change, and locking is explicit withlock: [...]orlock: all, which fixes the whole subtree of each named slot (a user can no longer override a child of a locked slot).policy.allowWorkspaceProvidersbecomesallowUserKeys. The settings take one of three shapes derived from the configuration (set it up yourself / choose a model / configured by the administrator, all on the same course model diagram) and show a control only when using it can change something, so a fully locked deployment no longer offers Token Plan, adding services or the home page model picker. Translated legacy variables become defaults, asDEFAULT_MODELalready was. Outside locked subtrees a user's choice anywhere up the tree outranks every server default. Found while hand-testing a deployment whose yml wrote every root: every control in the settings was visible and none could change anything.Sep 30: P1 and P2 landed on
integration/provider-config(feat(persistence): workspace model configuration with keys encrypted at rest #1733–refactor(settings)!: server-side model settings only; import browser settings once #1744, fix(config): a policy without workspace providers also stops using them #1750). Decisions made along the way: a workspace provider reaches media, search and document services only at its preset's own endpoint (custom endpoints and self-hosted media presets are deployment-only); the settings API serves the preset catalogue, so no client-safe registry split was needed; fallbacks apply to language-model slots only;policy.allowWorkspaceProviders: falsealso stops using providers workspaces added before; provideroptions(non-secret, deployment-only) carry settings such as the VoxCPM backend; the browser import carries providers, keys and the model choice, not per-stage routes or per-browser capability toggles; the browser's own speech recognition stays available whileasris unassigned.Sep 29: updated after feat(persistence)!: server-backed persistence only, with a one-way legacy browser import #1710 landed. Background now reflects that settings and keys are still in the browser; added the caller-supplied endpoint attack surface as a motivation; assignments carry the feat(llm): retry once on a fallback model for retryable failures #1614 fallback; the Docker journey uses
docker compose upwith single-user owner mode, which resolves the first open question; the browser settings import reuses the feat(persistence)!: server-backed persistence only, with a one-way legacy browser import #1710 importer; development moved tointegration/provider-config, tracked in [RFC] Providers, capability slots and server-side model configuration (tracking) #1725.Sep 30: a provider-only reference (
webSearch: tavily) names the provider's default model (feat(config): resolve media and tool capabilities through their slots #1737); retries follow the slot's fallback (feat(config)!: resolve every LLM call through its capability slot #1735).Sep 30:
MODEL_ROUTESand per-stage routes in browsers are no longer translated: emulating them needed special rules for shared slots, the agent driver and retries and still relied on easy-to-miss notices. Only providers,DEFAULT_MODELandMODEL_FALLBACKcarry over;MODEL_ROUTESwithoutopenmaic.ymlfails at startup (refactor(config)!: translate only providers and the default model #1732).Sep 29: requirements are no longer inherited by child slots, found while mapping stage keys to slots (feat(config): capability slot registry and stage-to-slot mapping #1726): inherited,
agent.titlewould have needed tool calling.All reactions