fix(loki): vision is a provider chain, not a pinned model that rots - #220
Merged
Conversation
Every screenshot attached to Loki has been failing. GROQ_VISION_MODEL
pinned meta-llama/llama-4-scout-17b-16e-instruct, which is DECOMMISSIONED
— 404 — and Groq now serves no vision model at all on this account. Found
while probing models for the tool loop, not by anyone using the feature,
which is the part worth fixing.
This is the fourth pinned-free-model rot in this fleet, so it gets the
class treatment rather than a new pin:
config/vision-models.ts a CHAIN across providers, tried in order.
Entries are OpenAI-compatible, so adding a
provider is a row, not a client. Groq keeps a
seat with no default model so restoring one is
one string.
lib/vision.ts walks the chain, first real answer wins.
probe-models.ts now probes BOTH halves — chat tool-calling AND
vision — with a real image. `npm run probe:models`.
"Real answer" is load-bearing: nvidia/nemotron-nano-12b-v2-vl returns
HTTP 200 with EMPTY content, which a naive client reports as a successful
analysis of nothing — reading to the user as "nothing notable in your
screenshot" rather than as a failure. Empty is treated as a dead link and
the chain continues.
Verified live against a solid-colour test image:
google/gemma-4-26b-a4b-it:free -> 200, correctly answered "Red" OK
google/gemma-4-31b-it:free -> 429 rate limited fallback
nvidia/nemotron-nano-12b-v2-vl -> 200 but empty excluded
FleetCrown had no vision-capable credential at all; it now shares
orangecat's OPENROUTER_API_KEY (added to /opt/fleetcrown/app/.env on the
box, which deploy.sh bootstraps into shared/ on the next release).
Retires callGroqVision + GROQ_VISION_MODEL — one caller, now on the chain,
and leaving a dead constant is how the next person re-pins a dead model.
The analysis block now names the model that read the image: it is the one
part of a Loki turn NOT grounded in FleetCrown's records, and saying so
keeps it from blending into the cited answer around it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
HTTP_TIMEOUT_XL_MS lost its only user when the Groq vision path was retired. Caught by CI, not locally, because I linted the files I had touched instead of the repo — and the orphan is by definition in a file the change did not add a line to. Run `npm run lint`, not eslint on a path list. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Every screenshot attached to Loki has been failing.
GROQ_VISION_MODELpinnedmeta-llama/llama-4-scout-17b-16e-instruct, which is decommissioned — 404 — and Groq now serves no vision model at all on this account.Found while probing models for the tool loop (#214), not by anyone using the feature. That's the part worth fixing.
Why a chain, not a new pin
This is the fourth pinned-free-model rot in this fleet. A single pinned free model is a scheduled outage, so this gets the class treatment:
config/vision-models.tslib/vision.tsscripts/probe-models.tsnpm run probe:models."Real answer" is load-bearing.
nvidia/nemotron-nano-12b-v2-vlreturns HTTP 200 with empty content, which a naive client reports as a successful analysis of nothing — reading to the user as "nothing notable in your screenshot" rather than as a failure. Empty is treated as a dead link and the chain continues.Verified live
Against a solid-colour test image:
End-to-end through the new chain:
Credential
FleetCrown had no vision-capable credential at all. It now shares orangecat's
OPENROUTER_API_KEY, appended to/opt/fleetcrown/app/.envon the box (backed up first).deploy.shbootstraps that file intoshared/on the next release, so it survives.Cleanup
Retires
callGroqVision+GROQ_VISION_MODEL— one caller, now on the chain. Leaving a dead constant behind is how the next person re-pins a dead model.The analysis block now names the model that read the image: it's the one part of a Loki turn not grounded in FleetCrown's records, and saying so keeps it from blending into the cited answer around it.
65/65 unit files pass, tsc clean, 0 lint errors.
🤖 Generated with Claude Code