v4.7.0 #10821
mudler
announced in
Announcements
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
🎉 LocalAI 4.7.0 Release! 🚀
LocalAI 4.7.0 is out!
This release widens what LocalAI can generate and how you drive it: a UI-managed voice cloning library, local video and audio-driven avatar generation, and interleaved reasoning that travels with tool calls. It also lands new audio engines (one-pass diarized transcription, F5-TTS, true streaming TTS) and a batch of reliability fixes across auth, transcription, and the model gallery.
Highlights:
localai://voice-profiles/<id>URI. No more hand-edited YAML or copying audio into model folders.longcat-videobackend brings text-to-video, image-to-video, and audio-driven talking-avatar generation, wired into the Studio UI with audio and reference-image controls.reasoningandtool_callstogether and keep the reasoning across the tool-result loop, with areasoning_contentinbound alias and Anthropicthinkingblock support.moss-transcribe-cpp), F5-TTS voice cloning in CrispASR, and true streaming TTS in vibevoice-cpp (time-to-first-audio 2.38s vs 39.96s on CPU).context_size: -1runs a model at its full trained context window, read from GGUF metadata per-model, with a VRAM-fit warning.devices:), and a new model-load cooldown stops a deterministically-failing model from respawning its backend on every poll and leaking VRAM.Plus DFlash speculative-decoding gallery models, an OIDC fix for EC/PS/EdDSA-signed tokens, transcription language/translate settings that reach the backend, and the usual set of dependency updates.
📌 TL;DR
localai://voice-profiles/<id>. NewGET/POST/DELETE /api/voice-profiles, MCP tools (list/create/delete_voice_profile), and a typedtts.voice_cloningconfig. Cloning declared as a capability across 12+ TTS backends (self-discovered, no hardcoded backend names).longcat-videoPython backend: text-to-video, image-to-video, and LongCat-Video-Avatar 1.5 audio-driven avatars. Gallery entrieslongcat-videoandlongcat-video-avatar-1.5; newknown_input_modalities/known_output_modalitiesconfig fields;/videoendpoint extended with staged audio. CUDA 12/13 x86_64 + CUDA 13 ARM64 images.reasoning+tool_callstogether and preserve reasoning across the tool loop.reasoning_contentaccepted as an inbound alias; Anthropic Messages local path round-tripsthinkingblocks (gated onthinking: {type: "enabled"}).moss-transcribe-cpp-0.9b. Offline; 1.6-2.2x faster than reference on CPU, bit-exact on CUDA.vv_capi_tts_stream: TTFA 2.38s vs 39.96s batch (~17x) on CPU forVibeVoice-Realtime-0.5B.f5-tts-crispasrgallery model (24kHz, voice cloning viavoice:/voice_text:options).context_size(orLOCALAI_CONTEXT_SIZE=-1) resolves to the model'sn_ctx_trainfrom GGUF metadata, with a GPU-only VRAM-fit warning and a safe clamp so no backend ever sees a negative window.options: [devices:CUDA1,CUDA2]restricts offload to named GPUs (from--list-devices).10s, geometric growth capped at 5m) so polling clients get503 + Retry-Afterinstead of respawning a crashing backend and leaking GPU memory.spec_type:draft-dflash.🚀 New Features & Major Enhancements
🎙️ Managed voice cloning profiles (Voice Library)
Record or upload a consented reference, save it as a reusable profile, and reference it from any cloning-capable TTS backend.
Voice cloning becomes a first-class, UI-driven workflow. Record or upload consented reference audio in the React UI, normalize it to WAV, preview it, and save it as a named profile. Reference the profile from the TTS UI or
/v1/audio/speechwith a stablelocalai://voice-profiles/<id>URI, no YAML editing and no copying audio into model directories.GET /api/voice-profiles,GET /api/voice-profiles/:id/audio,POST /api/voice-profiles(admin),DELETE /api/voice-profiles/:id(admin), surfaced in Swagger,/api/instructions, and the auth capability registry. MCP admin toolslist_voice_profiles,create_voice_profile,delete_voice_profile.tts.voice_cloning(bool) opts custom model names in or out of Voice Library compatibility;falsealso rejects saved profile references with HTTP 400. Request precedence isvoice->tts.voice->tts.audio_path; existing options still work with no breaking change.vllm-omni,vibevoice-cpp,coqui,pocket-tts,qwen-tts,qwen3-tts-cpp,faster-qwen3-tts,fish-speech,neutts,chatterbox,voxcpm,omnivoice-cpp, and model-dependentcrispasr.params.ref_text. E2E verified on qwen3-tts-cpp (4.47s reference produced a non-silent 3.44s 24kHz WAV).🎬 LongCat video and avatar generation
A new
longcat-videoPython backend brings local video generation to LocalAI: text-to-video, image-to-video, and LongCat-Video-Avatar 1.5 for audio-driven talking avatars. It is wired into the React Studio UI with audio and reference-image controls.longcat-video(text+image input, video output) andlongcat-video-avatar-1.5(text+image+audio input, video output). Backend images: CUDA 12 and CUDA 13 x86_64, plus CUDA 13 ARM64 (DGX Spark / NVIDIA ARM64 guidance included).known_input_modalities/known_output_modalitiesconfig fields so generic code discovers what a checkpoint accepts instead of branching on backend or checkpoint names. The HF importer emits matching self-describing recipe metadata./videoendpoint gains staged audio and video-generation parameters, with distributed audio staging (input bounded to 128 MiB). Avatar mode supports multi-segment generation, BF16 or optional INT8 quantization, and distillation settings, documented in a newfeatures/longcat-video.mdpage.🧠 Interleaved thinking with tool calls
An assistant turn can now carry
reasoningandtool_callstogether, and the reasoning survives the tool-result loop across turns. This is uniform, tested, and documented behavior rather than a per-backend accident.reasoning_contentas an inbound alias for the canonicalreasoningfield (vLLM/DeepSeek/cogito-style clients emit it); emission is unchanged and the canonical field wins when both are present.thinkingblocks: inboundthinkingblocks parse into the message reasoning, and athinkingblock is emitted beforetool_useon both non-streaming and streaming responses, gated on the request paramthinking: {type: "enabled"}. The cloud-proxy passthrough path is untouched.lfm2.5-8b-a1bandgemma-4-e2breturnreasoningplus structuredtool_callsin a single turn. A newfeatures/interleaved-thinking.mddoc is cross-linked from the model-configuration, text-generation, and functions guides.🗣️ New audio engines: diarized transcription, F5-TTS, and streaming TTS
Three additions widen the audio surface:
moss-transcribe-cpp-0.9b(defaultq5_kGGUF). The ggml port is byte-exact to the reference PyTorch and 1.6-2.2x faster on CPU, bit-exact on CUDA (verified on Blackwell / Jetson Thor). Builds across the full Linux matrix plus Darwin/Metal.f5-tts-crispasrgallery model. Produces 24kHz mono audio and auto-detects asf5-tts(nobackend:selector). Voice cloning viaoptions: [voice:/path/ref.wav, voice_text:Transcript...]; note F5-TTS runs a 32-step ODE solver, so CPU synthesis is compute-heavy.vv_capi_tts_streamcallback ABI. Time-to-first-audio drops from 39.96s (batch) to 2.38s (streaming), about 17x, on a CPU-only box forVibeVoice-Realtime-0.5B. Scope is the realtime-0.5B model; the non-streaming path is unchanged.🎚️ Auto full context with
context_size: -1A
context_size: -1sentinel (any negative value) now makes a model run at its full trained context, resolved per-model from the GGUFn_ctx_trainmetadata at load. This lets you opt a model into its true maximum window even when a gallery YAML already pins a value, without hardcoding a number. The global equivalentsLOCALAI_CONTEXT_SIZE=-1/--context-size -1make every model resolve to its own trained max, while an explicit per-model value still wins.Three defense layers keep it safe: GGUF resolution degrades to the default with a warning if metadata lacks a usable max; a GPU-only
warnIfContextExceedsVRAMhelper logs (never blocks) when the window likely will not fit; and the backend options layer clamps any residual negative to the default so no backend receives a negativen_ctx. The previously-only path (unsetcontext_size) is unchanged.🖥️ GPU device selection and model-load cooldown
device/devicesoption in the llama.cppoptions:array maps to upstream--device, so you can restrict offload to specific GPUs (for example excluding a display or debug GPU). Example:options: [devices:CUDA1,CUDA2,CUDA3]. Device names come fromllama-server --list-devices.503 + Retry-Afterinstead of respawning the backend on every poll. This stops a deterministically-failing model from leaking GPU/CUDA state (a reporter saw leaked contexts climb to ~58 GB) and, underLOCALAI_SINGLE_ACTIVE_BACKEND, from stealing the active slot from healthy models. Configurable via--model-load-failure-cooldown/LOCALAI_MODEL_LOAD_FAILURE_COOLDOWN(default10s,0disables); the cooldown doubles per consecutive failure, caps at 5m, and resets on a successful load. Coalesced followers of a genuine concurrent burst still get their one retry.🧠 Models
llama-cppbackend, each bundling a full target model plus its small block-diffusion drafter (drafters are not standalone chat models):qwen3-4b-dflash,qwen3.5-9b-dflash,qwen3.6-27b-dflash,qwen3.6-35b-a3b-dflash. Per-entry config isflash_attention: on,draft_model:,use_jinja:true,spec_type:draft-dflash,spec_n_max:15. Requires the pinned llama.cpp with upstream DFlash support; every bundled drafter is verifiedgeneral.architecture = dflash(fork-onlydflash-draftGGUFs are intentionally excluded because they fail to load). GPU recommended.deepseek-v4(auto-generated from unsloth), so LocalAI applies correct generation parameters for the family automatically.🐛 Bug Fixes (recap)
fix(auth): accept EC/PS/EdDSA-signed OIDC ID tokens, not just RS256 (OIDC login was 500ing at callback with EC-signed tokens, e.g. Authentik) - fix(auth): accept EC/PS/EdDSA-signed OIDC ID tokens, not just RS256 #10736fix(transcription): honor model-configparameters.language/parameters.translateand the OpenAIlanguageform field, which were silently ignored for multipart uploads - fix(transcription): honor model-config language/translate + OpenAI language form field #10731fix(backends):opusandlocal-storenow refuse foreign model loads, so an LLM with no explicit backend cannot silently bind to the audio codec or vector store during backend probing - fix(backends): refuse foreign model loads in opus and local-store #10769fix(gallery): backend (re)install is now a clean atomic replace (stage, validate, swap, rollback) instead of an overlay, so stale files from a prior version no longer shadow the new one - fix(gallery): make backend (re)install a clean replace instead of an overlay #10726fix(vram): report the largest single GGUF quant instead of summing every quant in an HF repo, so a 9B model no longer shows as 71 GB / "May not fit" - fix(vram): report largest GGUF quant instead of whole HF repo for gallery size (#10700) #10707fix(logs): capture backend stdout/stderr into the log store by default in single mode, so the Backend Logs page is populated out of the box - fix(logs): capture backend logs by default in single mode #10742fix(vllm): pin the L4T arm64 backend tovllm==0.24.0for GB10 / DGX Spark stability (0.23 crashes deterministically on cold loads and pins GPU memory) - fix(vllm): pin L4T arm64 backend to vllm==0.24.0 for GB10 stability #10725fix(ds4): bundle the full transitive runtime dependency closure so the from-scratch DS4 image no longer exits 127 on a missing gRPC library - fix(ds4): bundle transitive runtime dependencies #10783fix(diffusers,vllm-omni,tinygrad): save generated images as PNG explicitly, fixing anunknown file extension: .tmpcrash right after successful inference - fix(diffusers,vllm-omni,tinygrad): save generated images as PNG explicitly #10729fix(react-ui): preserve uploaded file content when regenerating a non-last answer (attachments were silently dropped after forking and regenerating a file turn) - fix: preserve uploaded file content when regenerating a non-last answer #10819fix(ui): prevent a large data table from breaking the flexbox layout and forcing a page-wide horizontal scrollbar - fix(ui): prevent table container from breaking flexbox layout #10754👒 Dependencies
Submodule and backend bumps this cycle:
ggml-org/llama.cppx7CrispStrobe/CrispASRx7vllm-metal(darwin) x6ServeurpersoCom/qwentts.cppx4ServeurpersoCom/omnivoice.cppx4leejet/stable-diffusion.cppx3ikawrakow/ik_llama.cppx3ggml-org/whisper.cppx2mudler/moss-transcribe.cppx1mudler/locate-anything.cppx1vllm-project/vllm0.24.0 -> 0.25.0 (Python backend) and cu130 wheel to0.25.0Plus
grpcio1.81.1 -> 1.82.1,charset-normalizer>=3.4.9, and GitHub Actions bumps (actions/cache4 -> 6,actions/stale10.3.0 -> 10.4.0).📖 Documentation
🙌 New Contributors
Full Changelog: v4.6.2...v4.7.0
This discussion was created from the release v4.7.0.
Beta Was this translation helpful? Give feedback.
All reactions