llama.cpp auto-translate: discover custom GGUFs + open-folder button#11028
Merged
Conversation
Two related changes for the llama.cpp auto-translate engine. 1. Auto-discover custom GGUFs in the models folder. New LlamaCppServerManager.GetAllTranslateModels() returns the curated TranslateModels list plus any other *.gguf the user has dropped in. Custom entries carry an empty Url (sentinel for "already on disk, no download"), the file name as DisplayName, and a human-readable size; mmproj-*.gguf sidecars are skipped because they're vision pairs, not standalone translation models. The auto-translate combo renders custom rows as "<filename> (custom, <size>)" and the four PopulateModels call-sites + GetInstalledModelPath / ResolveModel now consult the merged list so a custom selection is recognised end-to-end. EnsureReadyAsync gets a defensive branch: a custom model whose file has vanished surfaces as a clear error instead of a cryptic download failure against an empty URL. Closes the "drop a Shisa / any GGUF in and have it appear" gap without growing the curated catalog. 2. Open-folder button next to the llama.cpp engine controls. Adds an IFolderHelper-backed [RelayCommand] OpenLlamaCppModelsFolder and a matching folder-open icon button to AutoTranslateWindow's settings panel. Visibility is bound to the existing LlamaCppButtonsAreVisible so it only appears alongside the engine-specific Download / Toggle server buttons. Tooltip uses the existing Se.Language.General.OpenContainingFolder localized string. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two tightly-related changes to the llama.cpp auto-translate engine:
*.gguf(e.g. Shisa v2.1 8B, a custom TranslateGemma quant, an arbitrary fine-tune) into~/SpeechToText/LlamaCpp/models/and it appears in the model dropdown alongside the curated catalog. No code change needed to use a model that isn't in the curated list.Changes
Logic/LlamaCpp/LlamaCppServerManager.csGetAllTranslateModels()— curatedTranslateModels+ any non-curated*.ggufin the folder. Skipsmmproj-*.gguf(vision sidecars). Custom entries:DisplayName = FileName, human-readableSizefrom disk,Url = ""(sentinel for "already on disk, no download"). Falls back to curated-only on folder-scan failure.Features/Translate/LlamaCppDownloadHelper.csLlamaCppModelDisplay.ToString()renders custom entries as<filename> (custom, <size>).GetInstalledModelPath()andResolveModel()consult the merged list.EnsureReadyAsyncgets a guard: if the chosen model is custom and its file has vanished, show a clear error instead of attempting a download with an empty URL.Features/Translate/AutoTranslateViewModel.csIFolderHelper. New[RelayCommand] OpenLlamaCppModelsFolder. FourPopulateModels(LlamaCppModels, LlamaCppServerManager.TranslateModels, …)call-sites swapped toGetAllTranslateModels().Features/Translate/AutoTranslateWindow.csSe.Language.General.OpenContainingFoldertooltip) appended to the llama.cpp settings panel after the Toggle-server button. Visibility bound to the existingLlamaCppButtonsAreVisibleflag so it shows only when llama.cpp is the selected engine.Why this matters
Closes the gap reported by users wanting to try GGUFs that aren't in SE's curated list (e.g. the Shisa V2.1 Japanese-tuned models on
mradermacher/shisa-v2.1-*-GGUF). The previous answer was "no, only TranslateGemma works"; the new answer is "drop the GGUF in the folder and pick it from the dropdown".Test plan
shisa-v2.1-qwen3-8b-Q4_K_M.gguf(or any other GGUF) intoLlamaCpp/models/, reopen Auto-translate, switch engine to llama.cpp → entry appears asshisa-v2.1-qwen3-8b-Q4_K_M.gguf (custom, ~4.7 GB), alphabetically after the curated entries.llama-serverstarts with the custom model, translation proceeds.Se.Settings.AutoTranslate.LlamaCppModelpersists the custom path across SE restarts.mmproj-foo.ggufinto the folder → it does NOT appear in the dropdown (those are OCR vision sidecars).translategemma-4b_Q4_K_M.gguf) → only one entry shows (curated wins; size/install indicator reflects the on-disk file)..gguffrom disk, run translation → clear "custom model file '…' was not found" error instead of a download crash.🤖 Generated with Claude Code