fix(opencode-free): replace unhealthy bootstrap model - #2166
Conversation
|
Follow-up hardening pushed in 5314eba: the test no longer treats any HTTP 200 as success or relies on one free model remaining healthy. It validates the completion envelope, probes candidates under a shared 15s budget, and adopts a fallback only for Maka-managed seeds. A live probe rejected the current Nemotron error envelope and succeeded with mimo-v2.5-free in 4.3s. Full npm test passes. |
Astro-Han
left a comment
Astro-Han
left a comment
There was a problem hiding this comment.
Thanks for improving OpenCode Free resilience. Probing multiple candidates within one shared deadline is reasonable, but connection testing should remain observational rather than modify user configuration.
No P0/P1 findings.
P2
isManagedOpenCodeFreeSeedinfers ownership from shape, so a user-selected single-model connection may be mistaken for a managed seed and overwritten.- Customized connections may test successfully through another model while retaining a broken default.
isOpenAIChatCompletionaccepts malformed responses such as{"choices":[{}]}.- The fallback write uses a pre-test snapshot and may overwrite edits made during the network request.
P3
- Please add focused coverage for customized connections, explicit-model tests, malformed responses, timeouts, and fallback persistence.
I suggest keeping this PR smaller: probe candidates and return modelTested, but do not persist a new default. Automatic model switching can be handled separately.
ee7621b to
1e720f0
Compare
|
Thanks, agreed that connection testing must remain observational. Addressed in Changes:
Focused coverage now locks customized connections, explicit-model tests (one model, no fallback), malformed completion envelopes, one shared deadline, and fallback persistence remaining absent. The three focused suites pass 20/20; core passes 781/781. The latest PR run has all regular checks and the Windows baseline green. |
likun666661
left a comment
likun666661
left a comment
There was a problem hiding this comment.
The candidate probing is a useful resilience improvement, but the current head still leaves the original existing-install failure unresolved and introduces two false/hanging connection-test cases. Details are inline.
1e720f0 to
0421ae5
Compare
|
Follow-up
Validation after the final rebase: core/storage/runtime builds passed; desktop typecheck passed; the directly affected core, connection-store, OpenCode Free, and network transport suites passed 54/54. The full core suite passed locally. Full storage remains non-authoritative on this Windows checkout because its existing SQLite cleanup cases fail with |
0421ae5 to
4e951f9
Compare
likun666661
left a comment
likun666661
left a comment
There was a problem hiding this comment.
Re-reviewed the latest head. The three requested changes are addressed: legacy bootstrap migration is narrowly fingerprinted and CAS-protected, valid reasoning/tool-call completions are accepted, and the production response body remains under the shared deadline. I found no new blocking issues; CI, typecheck, E2E, and Windows baseline are green.
Astro-Han
left a comment
Astro-Han
left a comment
There was a problem hiding this comment.
Re-reviewed the latest head. The connection test is now observational, legacy migration is narrowly scoped and concurrency-safe, completion validation covers valid reasoning/tool-call responses, and the shared deadline covers production body reads. No blocking findings.
Regenerate the checked-in catalog from a single fetch of https://models.dev/api.json on 2026-08-20, sha256 da57b8e4281fd65d725ef7f2f73f681d052848c07737c4aaa1c707d91ad26e33. The digest records which payload produced this snapshot. models.dev serves a rolling document and the payload is not archived here, so this is a provenance record, not a reproducibility guarantee. Models 1690 -> 1779, deprecated 82 -> 98. Pricing gains 86 entries, drops 9, and reprices 37 — openrouter:~google/gemini-flash-latest input falls 1.5 -> 0.375, deepinfra:moonshotai/Kimi-K3 rises 2.7 -> 2.85. Three consumers had to follow, and nothing more. Upstream deprecated north-mini-code-free and laguna-s-2.1-free, which provider-registry validates at module load, so importing the registry threw. Removing the two ids is the response that validation exists to force. The four that remain are unchanged from main and are still checked for active and tool-capable on every import. They come from the set #1720 chose by transcribing "active, tool-capable, cost=0" out of the snapshot, pinned by hand only because the snapshot carries no cost field. Individual ids have had operational attention since — #1720 called big-pickle anonymously, #2166 replaced an unhealthy bootstrap model and probed the fallbacks — but the four have never been verified as a set, and this refresh adds none. It also adds no reachable model: it only drops two the upstream retired. gemini-3-pro-preview left the catalog, so the Google thinking test moves to gemini-3.1-pro-preview, whose effort values carry the same no-off shape the test asserts. The snapshot now sits about 2 KB under Biome's 1 MiB default, and this refresh alone added 57 KB, so the next one crosses it. An over-size file is skipped with a warning while format:check still exits 0, so crossing would quietly stop checking the snapshot rather than fail on it. The ceiling is raised for that one file through an override; a global raise would also disarm the guard for hand-written files, which is what it exists for. Generated-by: Claude Code
The committed models.dev snapshot had drifted far enough that it no longer described the catalog it claims to mirror: 1690 models to 1779, deprecations 82 to 98, and pricing rows 1239 to 1316 (+86 new, -9 removed, 37 repriced). The refresh is regenerated from one pinned api.json payload rather than a live fetch, so every number here is reproducible from a fixed input. Three consumers had to follow. - Upstream deprecated two of the four ids pinned for the OpenCode Free offering, `north-mini-code-free` and `laguna-s-2.1-free`. Both are removed. `provider-registry.ts` asserts its pins are active and tool-capable at module load, so leaving them would fail the process at import, not at first use. - `gemini-3-pro-preview` left the catalog, so the Google thinking test moves to `gemini-3.1-pro-preview`, whose effort values carry the same no-`off` shape the test asserts. Two Google fallback ids the snapshot no longer contains, `gemini-2.0-flash` and `gemini-1.5-pro`, are dropped from that list. - The snapshot now sits about 2 KB under Biome's 1 MiB default, and this refresh alone added 57 KB, so the next one crosses it. An over-size file is skipped with a warning while `format:check` still exits 0, so crossing would quietly stop checking the snapshot rather than fail on it. The ceiling is raised for that one file through an override; a global raise would also disarm the guard for hand-written files, which is what it exists for. The refresh also flips 15 ZenMux models from active to deprecated. Two carry a model-level protocol override and are excluded from the fallback source, so 13 reach it. Discovery keeps only the fallback set and the catalog marks what survives available and default-capable, so all 13 would have been offered as usable choices for a provider that had none before. ZenMux now filters deprecated ids at the call site, as six other providers already do. That fix is deliberately narrow. Six providers still ship deprecated ids in their fallback lists — 28 models across `togetherai`, `mistral`, `xiaomi`, `deepinfra`, `nvidia`, and `openai`, the last writing its list by hand rather than through `toolCallingModelIds`. Reverting this refresh changes none of them, so converging them is tracked in #3355, and a new contract test records that set as an explicit boundary which fails both on regression into it and on a listed provider being cleaned up without being removed. Not addressed here: persisted user catalogs keep ids this refresh retires, because nothing reconciles `enabledModelIds` against the current registry and `opencode-free` has no discovery contract that could later repair it. That predates this branch — #2050 removed `hy3-free` the same way and touched no persistence — and reverting this refresh does not fix it, since the ids stay retired upstream. Tracked in #3354. The four surviving OpenCode Free pins are not evidence of anonymous entitlement and are not claimed as such. Individual ids have had operational attention — #1720 called `big-pickle` anonymously, #2166 replaced an unhealthy bootstrap model — but the four have never been verified as a set, and this refresh adds no verification and no reachable model; it only drops two the upstream retired. Reviewed by @hqhq1025 with Codex assistance under the maintainer-approved review workflow; two findings were raised, one fixed here and one filed as #3354. Generated-by: Claude Code
3 participants
Summary
big-picklebootstrap preferenceCloses #2165
Verification
npm testnpm run format:checknemotron-3-ultra-freeresponse, fell back tomimo-v2.5-free, and returnedok: truein 4.3s