Skip to content

llama.cpp auto-translate: discover custom GGUFs + open-folder button#11028

Merged
niksedk merged 1 commit into
mainfrom
llamacpp-translate-custom-ggufs
May 18, 2026
Merged

llama.cpp auto-translate: discover custom GGUFs + open-folder button#11028
niksedk merged 1 commit into
mainfrom
llamacpp-translate-custom-ggufs

Conversation

@niksedk
Copy link
Copy Markdown
Member

@niksedk niksedk commented May 18, 2026

Summary

Two tightly-related changes to the llama.cpp auto-translate engine:

  1. Auto-discover custom GGUFs in the models folder. Drop any *.gguf (e.g. Shisa v2.1 8B, a custom TranslateGemma quant, an arbitrary fine-tune) into ~/SpeechToText/LlamaCpp/models/ and it appears in the model dropdown alongside the curated catalog. No code change needed to use a model that isn't in the curated list.
  2. Open-folder button. New folder-open icon button next to the engine's Download / Toggle-server buttons, opens the llama.cpp models folder in Explorer / Finder / xdg-open. Makes the discovery flow obvious — there's a visible button to get to the folder where dropped GGUFs are picked up from.

Changes

File What
Logic/LlamaCpp/LlamaCppServerManager.cs New GetAllTranslateModels() — curated TranslateModels + any non-curated *.gguf in the folder. Skips mmproj-*.gguf (vision sidecars). Custom entries: DisplayName = FileName, human-readable Size from disk, Url = "" (sentinel for "already on disk, no download"). Falls back to curated-only on folder-scan failure.
Features/Translate/LlamaCppDownloadHelper.cs LlamaCppModelDisplay.ToString() renders custom entries as <filename> (custom, <size>). GetInstalledModelPath() and ResolveModel() consult the merged list. EnsureReadyAsync gets a guard: if the chosen model is custom and its file has vanished, show a clear error instead of attempting a download with an empty URL.
Features/Translate/AutoTranslateViewModel.cs Constructor now takes IFolderHelper. New [RelayCommand] OpenLlamaCppModelsFolder. Four PopulateModels(LlamaCppModels, LlamaCppServerManager.TranslateModels, …) call-sites swapped to GetAllTranslateModels().
Features/Translate/AutoTranslateWindow.cs New folder-open icon button (Se.Language.General.OpenContainingFolder tooltip) appended to the llama.cpp settings panel after the Toggle-server button. Visibility bound to the existing LlamaCppButtonsAreVisible flag so it shows only when llama.cpp is the selected engine.

Why this matters

Closes the gap reported by users wanting to try GGUFs that aren't in SE's curated list (e.g. the Shisa V2.1 Japanese-tuned models on mradermacher/shisa-v2.1-*-GGUF). The previous answer was "no, only TranslateGemma works"; the new answer is "drop the GGUF in the folder and pick it from the dropdown".

Test plan

  • No custom files in folder → dropdown shows only curated TranslateGemma entries (unchanged behavior).
  • Drop shisa-v2.1-qwen3-8b-Q4_K_M.gguf (or any other GGUF) into LlamaCpp/models/, reopen Auto-translate, switch engine to llama.cpp → entry appears as shisa-v2.1-qwen3-8b-Q4_K_M.gguf (custom, ~4.7 GB), alphabetically after the curated entries.
  • Select the custom entry, run a short translation → llama-server starts with the custom model, translation proceeds.
  • Se.Settings.AutoTranslate.LlamaCppModel persists the custom path across SE restarts.
  • Drop an mmproj-foo.gguf into the folder → it does NOT appear in the dropdown (those are OCR vision sidecars).
  • Drop a file with the exact same name as a curated entry (e.g. translategemma-4b_Q4_K_M.gguf) → only one entry shows (curated wins; size/install indicator reflects the on-disk file).
  • Click the new folder-open icon → Explorer / Finder / xdg-open opens the models folder.
  • Switch engine away from llama.cpp → all four engine-specific buttons (Download / Toggle server / Open folder) hide together; switch back → they reappear.
  • Edge case: select a custom model, delete its .gguf from disk, run translation → clear "custom model file '…' was not found" error instead of a download crash.

🤖 Generated with Claude Code

Two related changes for the llama.cpp auto-translate engine.

1. Auto-discover custom GGUFs in the models folder. New
   LlamaCppServerManager.GetAllTranslateModels() returns the curated
   TranslateModels list plus any other *.gguf the user has dropped in.
   Custom entries carry an empty Url (sentinel for "already on disk,
   no download"), the file name as DisplayName, and a human-readable
   size; mmproj-*.gguf sidecars are skipped because they're vision
   pairs, not standalone translation models. The auto-translate combo
   renders custom rows as "<filename> (custom, <size>)" and the four
   PopulateModels call-sites + GetInstalledModelPath / ResolveModel
   now consult the merged list so a custom selection is recognised
   end-to-end. EnsureReadyAsync gets a defensive branch: a custom
   model whose file has vanished surfaces as a clear error instead of
   a cryptic download failure against an empty URL.

   Closes the "drop a Shisa / any GGUF in and have it appear" gap
   without growing the curated catalog.

2. Open-folder button next to the llama.cpp engine controls. Adds an
   IFolderHelper-backed [RelayCommand] OpenLlamaCppModelsFolder and a
   matching folder-open icon button to AutoTranslateWindow's settings
   panel. Visibility is bound to the existing LlamaCppButtonsAreVisible
   so it only appears alongside the engine-specific Download / Toggle
   server buttons. Tooltip uses the existing
   Se.Language.General.OpenContainingFolder localized string.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@niksedk niksedk merged commit 6761708 into main May 18, 2026
1 of 3 checks passed
@niksedk niksedk deleted the llamacpp-translate-custom-ggufs branch May 18, 2026 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant