Skip to content

Pin the client provider mirrors against their server originals instead of trusting "keep in lockstep" comments #5673

Description

@atomantic

Problemclient/src/utils/providers.js (1777 lines) is a hand-mirror of three server modules (providerModels.js, localModelHeuristics.js, providerGateways.js) plus the toolkit's aiToolkit/providers.js. The repo already owns exactly the right mechanism for this — server/lib/mirrorParity.js#compareDeclaration, which reads the two files as TEXT (no import, so no client deps leak into the server CI job) and fails when a named declaration diverges. It is applied to precisely one declaration set (stageRunner.mirror.test.js, the context-window table) and to the client/src/lib catalog (mirrorCoverage.test.js). Everything in client/src/utils/providers.js is unguarded: the effort ladders, the capability regexes, and the gateway registry are pinned by prose comments and by per-side hardcoded expectations, which is the exact hole stageRunner.mirror.test.js was written to close ("each side had its own hardcoded expectations … a model bump that added a row to only one copy left BOTH suites green"). The consequence is not cosmetic: an effort level the client offers but the server rejects 400s a run, and a tool-use family the server recognises but the client doesn't gets a "⚠ no known tool use" badge on a model that works fine.

Evidence — the mirrors, all comment-pinned only:

client/src/utils/providers.js:237-251 duplicates the six effort ladders and CODEX_ULTRA_MODELS from server/lib/providerModels.js:89-115; client/src/utils/providers.js:496 duplicates EFFORT_RANK from server/lib/providerModels.js:373.

client/src/utils/providers.js:583 inlines a THREE-way mirror and says so:

 * Tool-use (function-calling) capable model detector  mirror of `isToolUseModel`
 * in server/lib/localModelHeuristics.js (and the TOOL_USE_RE inlined in
 * server/lib/aiToolkit/providers.js). Keep all three in lockstep (the server libs
 * can't be imported here).

The client test that claims to cover it never reads the server file — client/src/utils/providers.test.js:755 is describe('isToolUseModel (mirror of server localModelHeuristics)') with a hardcoded id list and no cross-file read, so a family added to server/lib/localModelHeuristics.js:175 only leaves both suites green.

client/src/utils/providers.js:47 still says // Keep in sync with server/lib/stageRunner.js. — that path has not existed since the module moved to server/services/stageRunner.js, which is the mirror comment already rotting.

client/src/utils/providers.js:1208 is a THIRD copy of PROVIDER_GATEWAYS. server/lib/providerGateways.parity.test.js:9 explicitly declines to cover it: "The client copy … is deliberately NOT imported here: a server suite that imports a client module drags the client's deps into the server CI job." That rationale does not apply to compareDeclaration, which reads the file with readFileSync and never imports it.

Plan

  1. Add server/lib/providerModels.mirror.test.js, modelled verbatim on server/services/stageRunner.mirror.test.js: SERVER_PATH = resolve(__dirname, 'providerModels.js'), CLIENT_PATH = resolve(__dirname, '../../client/src/utils/providers.js'), and MIRRORED_NAMES = ['CLAUDE_EFFORT_LEVELS','CODEX_EFFORT_LEVELS','CODEX_ULTRA_EFFORT_LEVELS','ANTIGRAVITY_EFFORT_LEVELS','OPENCODE_LOCAL_EFFORT_LEVELS','CURSOR_EFFORT_LEVELS','CODEX_ULTRA_MODELS','EFFORT_RANK','CONFIGURED_DEFAULT_SENTINELS','ANTIGRAVITY_EFFORT_SUFFIX_RE']. All ten are byte-identical today, verified, so the suite goes green on the first run.
  2. Add server/lib/localModelHeuristics.mirror.test.js. The three capability regexes are spelled as a multi-line new RegExp([...].join('|'), 'i') array on the server and as a single inline literal on the client, so declaration-text comparison cannot be used. Decide: compare COMPILED SOURCE, not text — read both files, extract the server's EMBEDDING_RE / VISION_RE / TOOL_USE_RE and the client's inline literal for isEmbeddingModel / isVisionModel / isToolUseModel with a narrow regex, and assert new RegExp(serverAlternatives.join('|'), 'i').source === clientLiteral.source. Rationale: the two sides must accept the same ids; how the alternation is typeset is irrelevant, and forcing the client to adopt the array form would churn a file with 84 commits in 60 days.
  3. Extend that same suite to the toolkit's third copy: assert server/lib/aiToolkit/providers.js's TOOL_USE_RE array equals server/lib/localModelHeuristics.js's, using compareDeclaration (both are the array form, so text comparison works).
  4. Add the client PROVIDER_GATEWAYS copy to server/lib/providerGateways.parity.test.js as a text-only third leg: read client/src/utils/providers.js, extract its PROVIDER_GATEWAYS rows, and assert the id + label + legacyMarker fields match the server registry row-for-row. Do not assert baseURL / legacyApiKeyField — the client deliberately omits them (it never dials the gateway), and that omission is the reason toEqual cannot be used here. Replace the "deliberately NOT imported" paragraph in that file's docstring with the new mechanism.
  5. Fix the rotted pointer at client/src/utils/providers.js:47: // Keep in sync with server/services/stageRunner.js.

Tests — the three new/extended parity suites ARE the deliverable. Each uniquely catches a divergence class no existing test can: (a) providerModels.mirror.test.js catches an effort level or ultra-model added on one side only; (b) localModelHeuristics.mirror.test.js catches a model family added to the server capability regex but not the client badge (and vice versa) — the failure mode providers.test.js:722/:755 provably cannot see; (c) the gateway leg catches a new gateway added to both server copies but missing from the picker. Do not add per-id unit cases — the existing hardcoded suites already cover behaviour.

Acceptance criteria

  • server/lib/providerModels.mirror.test.js and server/lib/localModelHeuristics.mirror.test.js exist and pass against the current tree.
  • Mutating any one mirrored declaration on one side only makes exactly one parity test fail with a message naming the declaration and stating the server copy is authoritative.
  • server/lib/providerGateways.parity.test.js covers the client copy's id/label/legacyMarker rows.
  • client/src/utils/providers.js:47 names an existing file.
  • cd server && npm test and cd client && npm test both pass.

Out of scope — deduplicating the mirrors themselves (a shared package is a much larger architectural change); the client/src/components/cos/constants.js and client/src/components/meatspace/post/constants.js mirrors; any change to provider behaviour.


Filed by a /do:better --scan-only --issues audit (2026-09-01). Category: dry · Severity: high · Files: client/src/utils/providers.js:47, client/src/utils/providers.js:69, client/src/utils/providers.js:534, client/src/utils/providers.js:554, client/src/utils/providers.js:583, client/src/utils/providers.js:1208, server/lib/providerModels.js:89, server/lib/localModelHeuristics.js:27, server/lib/localModelHeuristics.js:94, server/lib/localModelHeuristics.js:175, server/lib/providerGateways.js:53, server/lib/aiToolkit/providers.js:147, server/lib/mirrorParity.js:1, server/services/stageRunner.mirror.test.js:1

All labels already exist in the repo; do NOT create labels. Never add planner:* labels.

Metadata

Metadata

Assignees

Labels

code-qualityProposed from a module-hygiene auditdryDRY and YAGNIeffort:mediumEffort: mediummodel:mediumModel size: mediumplanTracked by /do:replanplanner:fable-5.1Plan authored by the Fable 5.1 modelseverity:hightestsTest suite / test infrastructure

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions