Skip to content

fix(loki): vision is a provider chain, not a pinned model that rots - #220

Merged
github-actions[bot] merged 2 commits into
mainfrom
fix/vision-provider-chain
Aug 13, 2026
Merged

fix(loki): vision is a provider chain, not a pinned model that rots#220
github-actions[bot] merged 2 commits into
mainfrom
fix/vision-provider-chain

Conversation

@catomean

Copy link
Copy Markdown
Collaborator

The bug

Every screenshot attached to Loki has been failing. GROQ_VISION_MODEL pinned meta-llama/llama-4-scout-17b-16e-instruct, which is decommissioned — 404 — and Groq now serves no vision model at all on this account.

Found while probing models for the tool loop (#214), not by anyone using the feature. That's the part worth fixing.

Why a chain, not a new pin

This is the fourth pinned-free-model rot in this fleet. A single pinned free model is a scheduled outage, so this gets the class treatment:

File Role
config/vision-models.ts A chain across providers, tried in order. Entries are OpenAI-compatible, so adding a provider is a row in a table, not a new client. Groq keeps a seat with no default model so restoring one later is one string.
lib/vision.ts Walks the chain; first real answer wins.
scripts/probe-models.ts Now probes both halves — chat tool-calling AND vision, with a real image. npm run probe:models.

"Real answer" is load-bearing. nvidia/nemotron-nano-12b-v2-vl returns HTTP 200 with empty content, which a naive client reports as a successful analysis of nothing — reading to the user as "nothing notable in your screenshot" rather than as a failure. Empty is treated as a dead link and the chain continues.

Verified live

Against a solid-colour test image:

✓ google/gemma-4-26b-a4b-it:free   200, correctly answered "Red"
⚠ google/gemma-4-31b-it:free       429 rate limited  → kept as fallback
✗ nvidia/nemotron-nano-12b-v2-vl   200 but empty     → excluded

End-to-end through the new chain:

Vision chain (2 usable link(s)):
  ✓ openrouter/google/gemma-4-26b-a4b-it:free → answered: Red

Credential

FleetCrown had no vision-capable credential at all. It now shares orangecat's OPENROUTER_API_KEY, appended to /opt/fleetcrown/app/.env on the box (backed up first). deploy.sh bootstraps that file into shared/ on the next release, so it survives.

Cleanup

Retires callGroqVision + GROQ_VISION_MODEL — one caller, now on the chain. Leaving a dead constant behind is how the next person re-pins a dead model.

The analysis block now names the model that read the image: it's the one part of a Loki turn not grounded in FleetCrown's records, and saying so keeps it from blending into the cited answer around it.

65/65 unit files pass, tsc clean, 0 lint errors.

🤖 Generated with Claude Code

test and others added 2 commits August 13, 2026 14:50
Every screenshot attached to Loki has been failing. GROQ_VISION_MODEL
pinned meta-llama/llama-4-scout-17b-16e-instruct, which is DECOMMISSIONED
— 404 — and Groq now serves no vision model at all on this account. Found
while probing models for the tool loop, not by anyone using the feature,
which is the part worth fixing.

This is the fourth pinned-free-model rot in this fleet, so it gets the
class treatment rather than a new pin:

  config/vision-models.ts  a CHAIN across providers, tried in order.
                           Entries are OpenAI-compatible, so adding a
                           provider is a row, not a client. Groq keeps a
                           seat with no default model so restoring one is
                           one string.
  lib/vision.ts            walks the chain, first real answer wins.
  probe-models.ts          now probes BOTH halves — chat tool-calling AND
                           vision — with a real image. `npm run probe:models`.

"Real answer" is load-bearing: nvidia/nemotron-nano-12b-v2-vl returns
HTTP 200 with EMPTY content, which a naive client reports as a successful
analysis of nothing — reading to the user as "nothing notable in your
screenshot" rather than as a failure. Empty is treated as a dead link and
the chain continues.

Verified live against a solid-colour test image:
  google/gemma-4-26b-a4b-it:free  -> 200, correctly answered "Red"   OK
  google/gemma-4-31b-it:free      -> 429 rate limited                fallback
  nvidia/nemotron-nano-12b-v2-vl  -> 200 but empty                   excluded

FleetCrown had no vision-capable credential at all; it now shares
orangecat's OPENROUTER_API_KEY (added to /opt/fleetcrown/app/.env on the
box, which deploy.sh bootstraps into shared/ on the next release).

Retires callGroqVision + GROQ_VISION_MODEL — one caller, now on the chain,
and leaving a dead constant is how the next person re-pins a dead model.

The analysis block now names the model that read the image: it is the one
part of a Loki turn NOT grounded in FleetCrown's records, and saying so
keeps it from blending into the cited answer around it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
HTTP_TIMEOUT_XL_MS lost its only user when the Groq vision path was
retired. Caught by CI, not locally, because I linted the files I had
touched instead of the repo — and the orphan is by definition in a file
the change did not add a line to. Run `npm run lint`, not eslint on a
path list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions
github-actions Bot merged commit 0314e31 into main Aug 13, 2026
1 check passed
@github-actions
github-actions Bot deleted the fix/vision-provider-chain branch August 13, 2026 13:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant