feat(runtime-host): refresh the models.dev catalog at Host startup - #4467
Conversation
e763324 to
01c3117
Compare
jackwener
left a comment
jackwener
left a comment
There was a problem hiding this comment.
I reviewed this at 01c31179ddb031578a9edefaca46b7e4af314287. No P0, P1, P2, or P3.
The Host now fetches models.dev once at startup through the same outbound admission as WebFetch. Privacy mode and a missing proxy credential skip the fetch and leave the bundled snapshot. Offline, timeout, a non-200, an oversized body, or a catalog the projector refuses all keep that snapshot and announce nothing. A catalog that projects whole replaces the table, then the Host publishes connection.catalog.changed so clients re-query. That frame is not configuration.changed, so Desktop does not raise a settings-changed-elsewhere notice. The decoder is exact on kind and revision. Epoch 88 against current main 87 covers both the new frame kind and video/pdf modalities.
Which models a connection has still comes from stored rows, fallbacks, and enabled ids. models.dev only answers what is true about a model. Desktop and TUI both re-read the catalog on the frame.
#4460 is also sitting on epoch 88. The first of these to merge is fine; the other must re-bump after main moves. This is a feature; I am not merging it.
简体中文
我审的是 01c31179ddb031578a9edefaca46b7e4af314287。没有 P0/P1/P2/P3。
Host 启动时经 WebFetch 同一套出站许可拉一次 models.dev。隐私模式和缺代理凭据会跳过。失败整表保持打包快照，不发通知。成功则整表替换，再发 connection.catalog.changed，不是 configuration.changed。epoch 88（main 87）。连接有哪些模型仍来自存储行和 fallback。#4460 也占着 88，后合的那个要再加。这是功能，我不合入。
Automated review notice: This comment was posted by an automated review agent operated by WAWQAQ. It is not an independent human review and does not replace one.
hqhq1025
left a comment
hqhq1025
left a comment
There was a problem hiding this comment.
No P0 or P1 findings remain on this exact head. I found two non-blocking P2 issues: the advertised response bound is enforced only after the full body is buffered, and the TUI can miss the one-shot catalog refresh between its initial query and late listener registration. The shared projection, whole-table install, protocol epoch/frame handling, Desktop invalidation path, privacy/proxy admission, and current-main compatibility otherwise held under review.
Local verification passed: clean install, build:test, full workspace typecheck, lint/format/ASF/diff checks, 418 affected tests, and focused Host refresh/change-feed/WebFetch/reconnect/policy tests. A streaming probe confirmed that a 40 MiB no-content-length response was read in full before the 32 MiB error. Hosted checks were still running when this review was submitted, so this approval is a code-review result rather than a statement that the merge gate is complete.
Review notice: This review was prepared by an automated review agent operated by hqhq1025 and is published at the direction of AstroHan, who has read these findings and is the human accountable for them.
| if (!response.ok) { | ||
| throw new Error(`models.dev responded ${response.status}`); | ||
| } | ||
| const body = await response.text(); |
There was a problem hiding this comment.
P2 — Enforce the response limit while streaming, before buffering. response.text() consumes the entire response before line 106 checks its length, so the stated 32 MiB bound does not constrain network input or peak Host memory. A no-content-length probe that supplied forty 1 MiB chunks pulled all 40 chunks before this path reported the size error; a fast upstream or configured proxy can therefore make the Host buffer arbitrarily more than the intended limit within the timeout. Please read response.body incrementally, count Uint8Array.byteLength, cancel as soon as the limit is crossed, and optionally reject an oversized Content-Length up front (the repository's readBoundedBytes helper in scripts/release-cli-publication.mjs demonstrates that shape).
| model: selectedTarget.model, | ||
| ...(modelContextWindow === undefined ? {} : { modelContextWindow }), | ||
| modelChoices, | ||
| subscribeModelCatalogChanges: (listener) => |
There was a problem hiding this comment.
P2 — Close the initial-query/subscription gap. The TUI captures connected.catalog at line 142, performs several awaited setup/recovery operations, returns this subscription closure, and only invokes it later in runMakaPiTui (pi-tui-runner.ts:1252). The Host publishes this refresh once, and RuntimeHostConnection does not retain/replay the revision when its local listener set is empty. If the startup fetch completes in that interval, the frame is discarded and this TUI keeps the bundled metadata for its whole lifetime even though the Host installed the live table. Subscribe before accepting the initial snapshot, or subscribe first and immediately re-read/reconcile so a refresh in the gap cannot be lost.
a086d3c to
9e39240
Compare
The Runtime Host is about to turn a models.dev response into `ModelMetadata` at startup, and the generator already does exactly that. Two copies of that projection would let one model mean one thing in the committed snapshot and another when the Host reads it live, which is the class of split this issue exists to close. `packages/core/src/models-dev-projection.ts` now owns provider selection and the per-model metadata projection. The generator keeps what only a build has: pricing, provider facts, model-provider overrides, snapshot IO, and code generation. Those stay out of the runtime parser deliberately — the Host installs metadata alone, so a malformed `cost` field it never reads must not be able to veto a metadata refresh. `prepare` runs the generator before any workspace builds, so the script loads the TypeScript source through its existing `loadTypeScriptModule` rather than a dist build that does not exist yet. That path erases type-only imports but cannot resolve a relative value import, which is why the new module keeps all of its relative imports type-only. The modality vocabulary is now a `Record<ModelModality, true>` derived from `ModelInfo`, so a modality added to the type cannot reach the wire without a decision in the projection. Behaviour is unchanged and verified as such: refreshing the same models.dev response through the previous and current generator yields a byte-identical projection and byte-identical generated modules. Refs #4398 Generated-by: Claude Code
The models.dev layer was a module constant, so the only way to change it was to ship a new build. The Runtime Host is about to fetch the catalog at startup and needs somewhere to put the answer. The seam replaces the whole table rather than merging model by model. A merge would keep describing a model upstream has delisted, and the committed snapshot is not a second opinion about a model upstream still publishes — it is what we fall back to when we have no answer at all. The four tables that used to be derived at module init (Claude subscription, OpenAI OAuth, SiliconFlow overrides, Ollama Cloud thinking) now read the active table, and the static patch layer is memoized on that table's identity so a lookup after an install pays the rebuild once. Refs #4398 Generated-by: Claude Code
The Host owns the model catalog, so a Maka installed months ago described every model by the snapshot compiled into that build. It now asks models.dev once at startup and, on a whole catalog it can project, makes that the metadata every connection resolves against — no reinstall, and no client needing to be on a matching version. The refresh is announced as a fifth `HostChangeFeed` frame, `connection.catalog.changed`, rather than reusing `configuration.changed`: the user's settings did not move, and a client showing a settings-changed-elsewhere notice must not show it here. Desktop re-emits its connection list on the frame; the TUI re-reads and adopts the Host's projection wholesale. The fetch goes through the same admission as the WebFetch tool — privacy mode refuses it, a configured proxy carries it — which is why that operation is now named for what it admits rather than for its first caller. Any failure at all, offline through an upstream shape the projection refuses, leaves the committed snapshot in place; there is no partial install. Mid-pagination the swap is safe: which models a connection has comes from its stored rows, the registry's fallback list and the ids the user enabled, never from models.dev, so `catalogEntryCount` and every cursor survive. Only field values can differ across pages, and the frame makes the client re-read. Refs #4398 Generated-by: Claude Code
Two hand-written entries existed because the snapshot they were written against did not carry the fact: openrouter's deepseek-v4-pro effort set, and the GPT-OSS branch of the Ollama Cloud thinking pin. Both now restate what models.dev publishes, verbatim, in the committed snapshot and in the live catalog alike — verified by resolving each id both ways before and after the deletion. The groq comment claimed models.dev misapplies an effort set to qwen/qwen3-32b; models.dev no longer lists that model at all, so the entry is now the sole source rather than a correction, and says so. Refs #4398 Generated-by: Claude Code
WebFetch bounded its response body with a private helper. The models.dev refresh needed the same bound and grew a second one that counted `String.length` — a UTF-16 unit count, which lets a multi-byte body pass a limit it exceeds by up to three times, and only after the whole body was already buffered. Lift WebFetch's implementation to `@maka/core/bounded-response` and have both call it. The bound stays enforced twice, before and during the read, in the one place either caller can change it. Generated-by: Claude Code
A models.dev refresh is four steps: fetch it under a byte bound, project it, account for what upstream stopped carrying, then decide what to do with the result. Only the projection was shared. The Host reimplemented the fetch bound and skipped the accounting entirely, so it silently dropped models the generator would have stopped a human over — and the two disagreed about which responses are too large. Move `collectProjectionRemovals` beside the projection it describes and add `fetchModelsDevProjection`, which owns all three shared steps. The generator and the Host now differ only in the fourth: the generator refuses a shrinking refresh until a human acknowledges it, the Host records the removals and adopts them, because nothing it installs is committed or redistributed. Drops the Host's `onSkipped` callback, which no caller ever passed. Generated-by: Claude Code
The refresh starts partway through composing the Host. A later step that throws unwinds every other resource it had already built, but the refresh was held in a `try`-scoped binding the catch block could not see, so its in-flight fetch and proxy transport outlived the composition that started them. Hoist the binding beside the other unwound resources and close it first. Generated-by: Claude Code
The TUI kept its own copies of the selected model's context window and thinking levels, resynchronized at three sites. A fourth was now needed: the Host republishes its catalog mid-session, and a copy taken before that keeps sizing the context bar and gating `/thinking` by the model's old properties. Read both from the current choice at the moment they are used. There is nothing left to resynchronize, and the compensation branch that cleared a stale context window on a model switch goes with it. Generated-by: Claude Code
The codec caps a model's display name at 512 characters and its description at 2048. Nothing upstream of it did. A refreshed models.dev row longer than either built a catalog entry the Host could encode but no client could decode, and the whole page was rejected — every model gone, not the one long string. Name the two limits where the catalog is built, truncate there, and have the codec read them. A producer that cannot exceed the limit is why the decoder never has to reject a page. Generated-by: Claude Code
9e39240 to
cba0090
Compare
apache#4496) The connection detail asked `hasModelMetadata()` — a question about the renderer's own bundled models.dev snapshot — to decide whether an enabled model needs a hand-written capability declaration. Since apache#4411 clients read Host-resolved `catalogEntries` rather than resolve a catalog themselves, and since apache#4467 the Host refreshes that catalog at startup; so a model the Host learned about after this build was cut is described everywhere except this one renderer, which still showed it a spurious capability-declaration row. The Host already owns the answer. `ModelCatalogEntry` now carries `describedByMetadata`, set by `makeEntry` from the same metadata lookup `hasModelMetadata` reads, and the renderer asks the entry it already has instead of its stale table. Inferring coverage from whether some optional field happens to be present would put a second, weaker copy of the rule in the renderer — the split this line of work exists to close — so the entry states it directly. That makes it a wire field: it is required on `ModelCatalogEntry`, decoded by `model-catalog-entry-codec`, and the compatibility epoch moves to 90 so a Host and client that disagree about its presence are refused at the handshake. The `apache#1584` case (a user-typed id no inventory describes) still reports false, so that model keeps its declaration row. `provider-endpoint-presentation.ts` also reads the bundled table, but only to look up build-time `generatedModelProviderOverrides` a Host refresh never installs, so it stays consistent and is left alone. Fixes apache#4496 Generated-by: Claude Code
3 participants
Summary
A Maka installed months ago describes every model by the models.dev snapshot compiled into that build. Since #4411 the Runtime Host is the sole authority for the model catalog, so that staleness is now one process's problem to fix: the Host fetches
https://models.dev/api.jsononce at startup and, on a catalog it can project whole, makes it the metadata every connection resolves against. Clients keep reading Host-resolved catalog entries, so a Desktop or TUI on any version sees the refreshed facts without being updated itself.The refresh reaches clients as a fifth
HostChangeFeedframe,connection.catalog.changed, and they re-query. It is deliberately notconfiguration.changed: the user's settings did not move, and Desktop's handler for that frame raises a settings-changed-elsewhere notice which must not fire here.Three things had to be true before a runtime refresh could exist at all, and they are the first three commits:
videoandpdfmodalities; the validator rejected them, so 324 models failed andrefresh:model-metadatawrote nothing. Widening the vocabulary takes rejects to zero (1871 → 1906 models, +66 / −31). This is a wire vocabulary change, hence the compatibility epoch bump to 88.ModelMetadatamapping. It now lives inpackages/core/src/models-dev-projection.tsand both the script and the Host read it, so a model cannot mean one thing in the committed snapshot and another when the Host reads it live. Verified behaviour-preserving by running the pre- and post-refactor generators against the same live input: byte-identical projection, identical generated module.Replacing the whole table is safe for the model list: which models a connection has comes from its stored rows, the registry's fallback list and the ids the user enabled — models.dev feeds only what is true about a model, never which models exist. That is also why a metadata swap mid-pagination is safe:
catalogEntryCountand every catalog cursor are invariant under it, so only field values can differ across pages, and the frame makes the client re-read.Failure is whole and quiet: offline, timeout, a non-200, an oversized body, or an upstream shape the projection refuses all leave the build's committed snapshot in place. There is no partial install and no per-model tolerance — the Host runs exactly the fail-loud projection the generator does.
The generator and the Host share the whole refresh, not just the projection.
fetchModelsDevProjectioninpackages/core/src/models-dev-refresh.tsowns the three steps neither caller gets to decide alone — fetch under a byte bound, project, account for what upstream stopped carrying — and they differ only in the fourth. That difference is deliberate: the generator refuses a shrinking refresh until a human passes--accept-upstream-removals, because it is about to commit and redistribute the result; the Host records the removals to stderr and adopts them, because nothing it installs is committed and the next process start asks again. Both name a removal by the same path, so an operator reading the Host log and a maintainer reading the generator's refusal are looking at the same thing.The byte bound itself is shared one level lower:
readBoundedResponseTextinpackages/core/src/bounded-response.tsis the one implementation, used here and by the WebFetch tool. It refuses on the declaredcontent-lengthbefore a byte arrives and again on the running byte count, because a body that never declared a length — or lied — is the case a single check misses.The fetch goes through the same admission as the WebFetch tool: privacy mode refuses it outright and a configured proxy carries it. That operation is now named
resolveHostOutboundExecutionfor what it admits rather than for its first caller.Finally, the refresh retires static overrides that only existed because the snapshot they were written against lacked the fact: openrouter's
deepseek-v4-proeffort set and the GPT-OSS branch of the Ollama Cloud thinking pin. Both now restate models.dev verbatim, in the committed snapshot and in the live catalog alike.Deliberately excluded, per the issue: TTL, on-disk cache, conditional GET, and an opt-out toggle. Also not refreshed here, and unchanged by this PR: pricing (
model-pricing.generated.ts), provider routing overrides, and provider facts. The connection editor's draft view still resolves locally once the draft diverges from what the Host committed, so an editor mid-edit can briefly show pre-refresh facts; that is the existing draft contract, not a regression.Two things a live catalog makes reachable that a compiled snapshot never did are fixed here rather than deferred. The catalog's display names and descriptions are now truncated where the entry is built, to the same limits the wire codec enforces — before, an upstream string longer than either produced a page the Host could encode and no client could decode, losing every model on it rather than one long string. And the TUI now derives the selected model's context window and thinking levels from the live choice list instead of keeping mirrored copies, which a mid-session catalog refresh would otherwise leave stale.
One commit on this branch is unrelated to #4398:
scripts/qualify-released-cli-state-root.test.mjsasserted POSIX-separator paths and failed thepackagejob on Windows. The bug arrived onmainwith #4427, not with this PR; it is fixed here only because it was blocking this branch's CI.Snapshot-vs-upstream drift check, a scheduled refresh job, and removing
docsUrlfromModelMetadatabelong to the parallel PR onci/4398-model-metadata-drift-check.Refs #4398
Verification
Run locally against this branch, all green:
npm run buildnpm run typechecknpm run format:checknpm run lintnpm run check:asf-headersnpm run check:model-metadataAffected suites only (
node --test, concurrency ≤ 4), 0 failures:@maka/runtime-host:model-metadata-refresh(new, 6 tests),host-change-feed,web-fetch-tool,connection-session,handshake-compatibility,host-kernel,project-catalog-protocol,runtime-policy-protocol,reconnecting-connection,catalog-reader,runtime-policy-coordinator@maka/core:models-dev-refresh(new, 4 tests),model-metadata,model-catalog,model-facts,model-thinking,llm-connections,provider-catalog-contract,runtime-policy-codec@maka/runtime:local-web-fetch,web-fetch-tool,model-factory-thinking,provider-conformancescripts/sync-model-metadata.test.mjs@maka/storage:runtime-policy-stores@maka/cli:pi-tui-runner,runtime-host-cli-context,runtime-host-onboarding,runtime-host-tui-context,tui-mcp-remote-publicationmodel-catalog-choices,relay-thinking-bulk,relay-profile-draftNot run locally: the repo-wide suite, Windows and Electron packaging jobs. Left to CI.
Measurements backing the claims above, taken against live
models.dev:refresh:model-metadatawrites nothing. After: 0 rejected, 1871 → 1906 models (+66 / −31).openrouter/deepseek/deepseek-v4-proandollama-cloud/gpt-oss:{120b,20b}resolve to the samethinkingOptionsbefore and after the deletion, under both the committed snapshot and the live catalog.openrouter/openai/gpt-5.6-solwas checked and kept — it pins atogglemodels.dev omits.Compatibility
RUNTIME_HOST_COMPATIBILITY_EPOCH87 → 88, for two reasons in one epoch: a connection model can now carryvideoorpdfin either modality direction, which an older peer's strict codec rejects, and the Host emits aconnection.catalog.changedframe whose kind an older client's frame decoder does not know. Mismatched peers are refused at the handshake, as intended.AI use
Tool(s) and scope: Claude Code wrote the implementation, tests and documentation edits in this PR under human direction and review. The affected commits carry
Generated-by: Claude Codetrailers; please keep the trailer on the squash commit.Checklist
Does this PR entail a change in behavior?