Provider model lists go stale, and the only cure is editing node_modules

[ayiejosh]

Provider model lists go stale, and the only cure is editing node_modules

Package: @earendil-works/pi-ai (vendored by dsh-llm-pi-ai, pinned ^0.85.1)
Observed: dsh 0.1.7-rc.2 → 0.2.0-rc.1, pi-ai 0.85.1 → 0.87.1

What happens

The model picker shows a list that does not match the provider. New models are absent with no
error, so it looks authoritative and is not.

The only way to correct it is to edit the catalog inside the dsh package's own node_modules
(@earendil-works/pi-ai/dist/providers/data/opencode-go.json). That file belongs to npm, not
the profile, so a dsh update replaces the tree and the correction is silently reverted.

Not profile-configurable

The route sets only apiKeyEnv. It cannot express the list either: resolveEntry resolves
api = request.api ?? base?.api ?? routeApi, so a route-level api overrides the catalog for
every model — and this provider is mixed-protocol. A profile models: block breaks the
other protocols. Today: edit node_modules (erased each update) or ~/.no-auto (forfeits all
updates).

Measured today

route	live	bundled 0.87.1	missing	stale
opencode-go	30	30	0	0
openrouter	460	386	80	6

opencode-go is in sync at 0.87.1 by luck: it was 27 at 0.85.1 against a live 30,
including a missing deepseek-v4.1-flash. openrouter is clearer — 80 live models are
unpickable, 6 bundled ones are dead, and its endpoint needs no key:

curl -s https://openrouter.ai/api/v1/models | jq '.data | length'   # 460

The part that makes this a bug

pi-ai already has the machinery for a live catalog, and it is unused.

dist/models.js implements the whole refresh path: createProvider({ refreshModels, filterModels }) (~468), runProviderRefreshPhase() calling provider.refreshModels({ credential, stored, publish, allowNetwork, force, signal }) (~125), refresh() selecting
every provider with that hook (~141), and getAvailable() applying
provider.filterModels?.(models, credential) (~271).

But no shipped provider defines either hook — grep -rln 'fetchModels:' dist/providers/*.js
returns nothing — and the harness never calls refresh(): dsh-llm-pi-ai/lib/index.js imports
only builtinModels, builtinProviders, getBuiltinModels, getBuiltinProviders. Plumbing built
on one side, never connected on the other.

What I would suggest

Wire it up. Give catalog providers a fetchModels and call refresh():

GET https://openrouter.ai/api/v1/models       # public
GET https://opencode.ai/zen/go/v1/models      # Bearer, or public

filterModels earns its keep on opencode-go: public vs key-filtered differ — 42 public vs
35 with the org key — and listed-but-unentitled models answer 400 Model is unavailable.

Two gaps I cannot settle from outside:

• Metadata. /models returns only { id, object, created, owned_by }; context window,
  cost and reasoning options live in pi-ai's bundle or models.dev/api.json. A refresh needs a
  merge policy for live models with no metadata anywhere — guessing limits is worse than
  omitting a model.
• Wire protocol. The endpoint does not say whether a model is anthropic-messages,
  openai-completions or openai-responses. I derived it by calibrating against the 27
  shipped models; it held, but the provider should say so.

Or, if a live refresh is out of scope: track pi-ai releases more eagerly (0.87.1 has 30
models to 0.85.1's 27 — no new code needed), and at minimum surface the model count and
catalog date in the picker or dsh doctor, so "this model does not exist" is distinguishable
from "my list is old". Today they are indistinguishable.

Why I am reporting it

dsh auto-updated and reverted opencode-go from a hand-corrected 35 to 27, removing the
model the session was configured to use. Reinstalling pi-ai 0.87.1 fixed it; the next update
will revert that too. I do not expect the profile to carry a model list — only that the
catalog not be a build artifacts that only a `node_modules edit can correct.

[argszero]

Your resolveEntry reading is exactly right, and it is worth naming the second half of it: the
per-model field list carries no api, so a models: entry cannot declare its own protocol at all
(api = request.api ?? base?.api ?? routeApi). That is why a mixed-protocol route has no
profile-side fix, and it is the structural reason your models: block breaks the other protocols
rather than a quirk of this provider.

One thing I would correct, because it changes how urgent the gap is. "Absent from the picker" is
the milder of the two failures. Naming an unlisted id does not reach the provider and get a 404 —
the adapter resolves the model out of its installed snapshot at the top of the stream call
(modelOf, before any network I/O) and throws:

throw new LlmError(`pi-ai provider "${provider}" has no configured model "${model}"`, 'UNKNOWN_MODEL')

So an id that is not in the bundle is not unpickable-and-maybe-usable, it is unusable: a session
configured for it cannot succeed on any endpoint, and no retry, timeout or live refresh of an
unrelated route changes that. That is precisely the shape of your last paragraph — a hand-corrected
node_modules that the next dsh update reverts leaves a session holding a model id that can only
fail before dispatch.

Of your three suggestions, fetchModels/refresh() wiring is the one that cannot be a plugin: the
hooks live inside the pinned @earendil-works/pi-ai dist and the adapter's import list, so a plugin
has no seam to add a hook no shipped provider defines. Your third — making "this model does not
exist" distinguishable from "my list is old" — is plugin-mountable, so I built it:

@argszero/cordis-plugin-model-catalog-gap@0.1.0 — https://github.com/argszero/cordis-plugin-model-catalog-gap

npm install --save-dev @argszero/cordis-plugin-model-catalog-gap

then mount model-catalog-gap from its cordis.patch.yml and call the tool
model_catalog_gap with { "provider": "openrouter", "baseURL": "https://openrouter.ai/api/v1" }
(the endpoint is contacted only when you hand it one). It prints the two counts, names the ids on
the wrong side in both directions, places one model: id explicitly, and states the consequence of
each standing. Two details of how it reads the provider that are worth knowing:

• The provider's list comes from the same service's model-discovery seam, asked with the endpoint
  and without the route name. That omission is what makes the comparison possible at all —
  naming the route makes the adapter answer from its own registry (on purpose: that answer carries
  capacities no listing endpoint reports), so only an unnamed request is the provider talking.
• The settings namespace is read from listConfigurableProviders() rather than assumed to be
  llm-pi-ai, so a deployment mounted under another namespace still gets served.

It changes nothing — no configuration written, no file under node_modules touched — because the
cure you describe is an adapter change. What it does add is the one configuration shape that is
expressible today, which your route's mixed protocols make non-obvious: where a route speaks one
protocol, the ids go in the route's models:; where it speaks several, the shape that works is a
sibling route that declares api for the ids it carries (same baseURL, same apiKeyEnv,
different route key), one sibling per protocol. It refuses to guess which of the two applies or
which protocol each new id speaks, and says so — no listing endpoint discloses the protocol, and a
wrong one fails at request time, so the split has to be yours. Your filterModels observation is
the same gap from the other side: 42 public vs 35 with the org key means the listing endpoint
alone is not a sufficient source for the route's model set either, so a refresh would still need
your entitlement filter before it could be authoritative.