Skip to content

fix voice auto-install chain — use canonical lmstudio-community GGUF ids and respect non-lmstudio providers#519

Merged
atomantic merged 2 commits into
mainfrom
worktree-fix-voice-model-install-chain
May 29, 2026
Merged

fix voice auto-install chain — use canonical lmstudio-community GGUF ids and respect non-lmstudio providers#519
atomantic merged 2 commits into
mainfrom
worktree-fix-voice-model-install-chain

Conversation

@atomantic

Copy link
Copy Markdown
Owner

Summary

The voice agent's auto-install chain in server/services/voice/bootstrap.js was firing on startup with model ids lms get can't fetch, producing:

🎙️  voice: qwen/qwen3.5-9b unavailable () — trying next
🎙️  voice: lmstudio-community/Qwen2.5-7B-Instruct-GGUF unavailable () — trying next
🎙️  voice: meta-llama/Llama-3.1-8B-Instruct unavailable () — trying next
🎙️  voice: exhausted install chain ... — set voice.llm.model explicitly in Settings

Two root causes:

  1. Wrong / non-existent ids. qwen/qwen3.5-9b was never released (Qwen line went 2.5 → 3 → 3.6); meta-llama/Llama-3.1-8B-Instruct is a gated HF repo that lms get can't pull without auth; qwen/qwen3-4b-2507 is the MLX-only slug. Each attempt failed in ~100 ms with empty stderr, which the regex-filtered warning rendered as the literal ().
  2. Ran for the wrong backend. ensureToolCapableModel / preloadModel always invoked lms get / lms load regardless of the configured voice provider, so an Ollama user saw lms failures for a CLI they never asked for.

Changes

  • Replace the chain with the un-gated lmstudio-community/...-GGUF ids that lms get actually accepts — and that match the curated localLlmCatalog.js entries, so the "recommended installs" UI and the voice auto-installer agree on the same model set:
    • lmstudio-community/Qwen3-4B-Instruct-2507-GGUF (~2.6 GB, current non-thinking Qwen3, fast TTFT)
    • lmstudio-community/Llama-3.2-3B-Instruct-GGUF (~2 GB, smaller fallback)
    • lmstudio-community/Qwen2.5-7B-Instruct-GGUF (~4.7 GB, classic workhorse)
    • lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF (~4.7 GB, Llama fallback)
  • Gate ensureToolCapableModel / preloadModel on the effective voice provider — mirrors resolveLlmEndpoint's fallback (deleted / CLI-type / endpoint-less providers still resolve to LM Studio, so we still provision in those cases). Only skip when the configured provider resolves to a usable non-lmstudio api-type backend (e.g. a working Ollama).
  • Rewrite the per-attempt warning to surface the last non-empty line of stderr (or stdout) instead of the trailing newline — so failures show what went wrong instead of ().
  • Extract isEffectiveLmStudioVoiceProvider helper near LMS_BASE.

Test plan

  • cd server && npm test -- --run services/voice/ → 387 voice tests pass
  • Restart PortOS with voice.llm.provider='lmstudio', voice.llm.tools.enabled=true, voice.llm.model='auto' — verify the install chain attempts a real id and either succeeds or logs an actionable failure reason
  • Restart PortOS with voice.llm.provider='ollama' — verify no lms get log lines appear

atomantic added 2 commits May 28, 2026 21:58
…ids and skip non-lmstudio voice providers

The previous chain referenced model ids that `lms get` can't fetch:
qwen/qwen3.5-9b was never released (Qwen line is 2.5 → 3 → 3.6);
meta-llama/Llama-3.1-8B-Instruct is a gated HF repo; qwen/qwen3-4b-2507
is the MLX-only slug. Each attempt failed in ~100 ms with empty stderr,
which the regex-filtered warning rendered as "unavailable ()". Replace
with the un-gated lmstudio-community/...-GGUF ids that already appear
in the curated localLlmCatalog so the "recommended installs" UI and the
voice auto-installer agree on the same model set.

Also gate ensureToolCapableModel / preloadModel on the voice provider so
Ollama users don't see the install chain run for a CLI they're not using,
and rewrite the per-attempt warning to surface the last non-empty line
of stderr (or stdout) instead of the trailing newline.
…ing LM Studio provisioning

resolveLlmEndpoint in llm.js falls back to LM Studio whenever
voice.llm.provider points to a deleted, CLI-type, or endpoint-less
provider — meaning the chat path still uses LM Studio. The previous gate
short-circuited on the raw provider id string and would skip provisioning
in those fallback cases, leaving voice without a tool-capable model.

Mirror the resolution logic: only skip when the configured provider id
actually resolves to a usable non-lmstudio api-type backend.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes the voice agent's auto-install chain so it (a) targets canonical lmstudio-community/...-GGUF ids that lms get can actually fetch, (b) only runs when the effective voice provider resolves to LM Studio, and (c) surfaces actionable failure reasons instead of empty () warnings.

Changes:

  • Replace the default tool-model chain with un-gated lmstudio-community/*-GGUF ids that match localLlmCatalog.js.
  • Add isEffectiveLmStudioVoiceProvider and gate both ensureToolCapableModel and preloadModel on it, mirroring resolveLlmEndpoint's fallback rules.
  • Improve the per-attempt failure warning to print the last non-empty line of stderr/stdout instead of a trailing newline.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@atomantic atomantic merged commit 8ff7580 into main May 29, 2026
3 checks passed
@atomantic atomantic deleted the worktree-fix-voice-model-install-chain branch May 29, 2026 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants