v3.0.0 #5689

mudler · 2025-06-19T15:55:11Z

mudler
Jun 19, 2025
Maintainer

🚀 LocalAI 3.0 – A New Era Begins

Say hello to LocalAI 3.0 — our most ambitious release yet!

We’ve taken huge strides toward making LocalAI not just local, but limitless. Whether you're building LLM-powered agents, experimenting with audio pipelines, or deploying multimodal backends at scale — this release is for you.

Let’s walk you through what’s new. (And yes, there’s a lot to love.)

TL;DR – What’s New in LocalAI 3.0.0 🎉

🧩 Backend Gallery: Install/remove backends on the fly, powered by OCI images — fully customizable and API-driven.
🎙️ Audio Support: Upload audio, PDFs, or text in the UI — plus new audio understanding models like Qwen Omni.
🌐 Realtime API: WebSocket support compatible with OpenAI clients, great for chat apps and agents.
🧠 Reasoning UI Boosts: Thinking indicators now show in chat for smart models.
📊 Dynamic VRAM Handling: Smarter GPU usage with automatic offloading.
🦙 Llama.cpp Upgrades: Now with reranking + multimodal via libmtmd.
📦 50+ New Models: Huge model gallery update with fresh LLMs across categories.
🐞 Bug Fixes: Streamed runes, template stability, better backend gallery UX.
❌ Deprecated: Extras images — replaced by the new backend system.

👉 Dive into the full changelog and docs below to explore more!

🧩 Introducing the Backend Gallery — Plug, Play, Power Up

No more hunting for dependencies or custom hacks.

With the new Backend Gallery, you can now:

Install & remove backends at runtime or startup via API or directly from the WebUI
Use custom galleries, just like you do for models
Enjoy zero-config access to the default LocalAI gallery

Backends are standard OCI images — portable, composable, and totally DIY-friendly. Goodbye to "extras images" — hello to full backend modularity, even with Python-based dependencies.

📖 Explore the Backend Gallery Docs

⚠️ Important: Breaking Changes

From this release we will stop pushing -extra images containing python backends. You can now use standard images, and you will have only to pick the ones that are suited for your GPU. Additional backends can be installed via the backend gallery.

Here below some examples, note that the CI is still publishing the images so won't be available until jobs are processed, and the installation scripts will be updated right after images are publicly available.

CPU only image:

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

NVIDIA GPU Images:

# CUDA 12
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

# CUDA 11
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-11

# NVIDIA Jetson (L4T) ARM64
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64

AMD GPU Images (ROCm):

docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas

Intel GPU Images (oneAPI):

# Intel GPU with FP16 support
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-intel-f16

# Intel GPU with FP32 support
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-intel-f32

Vulkan GPU Images:

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan

AIO Images (pre-downloaded models):

# CPU version
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu

# NVIDIA CUDA 12 version
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-12

# NVIDIA CUDA 11 version
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-11

# Intel GPU version
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-gpu-intel-f16

# AMD GPU version
docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-aio-gpu-hipblas

For more information about the AIO images and pre-downloaded models, see Container Documentation.

🧠 Smarter Reasoning, Smoother Chat

Realtime WebSocket API: OpenAI-style streaming support via WebSocket is here. Ideal for agents and chat apps.
"Thinking" Tags: Reasoning models now show a visual "thinking" box during inference in the UI. Intuitive and satisfying.

🧠 Model Power-Up: VRAM Savvy + Multimodal Brains

Dynamic VRAM Estimation: LocalAI now adapts and offloads layers depending on your GPU’s capabilities. Optimal performance, no guesswork.
Llama.cpp upgrades also includes:

reranking
Enhanced multimodal support via libmtmd

🧪 New Models!

More than 50 new models joined the gallery, including:

🧠 skywork-or1-32b, rivermind-lux-12b, qwen3-embedding-*, llama3-24b-mullein, ultravox-v0_5, and more
🧬 Multimodal, reasoning, and domain-specific LLMs for every need
📦 Browse the latest additions in the Model Gallery

🐞 Bugfixes & Polish

Rune streaming is now buttery smooth
Countless fixes across templates, inputs, CI, and realtime session updates
Backend gallery UI is more stable and informative

The Complete Local Stack for Privacy-First AI

With LocalAGI rejoining LocalAI alongside LocalRecall, our ecosystem provides a complete, open-source stack for private, secure, and intelligent AI operations:

LocalAI	The free, Open Source OpenAI alternative. Acts as a drop-in replacement REST API compatible with OpenAI specifications for local AI inferencing. No GPU required. Link: https://github.com/mudler/LocalAI
LocalAGI	A powerful Local AI agent management platform. Serves as a drop-in replacement for OpenAI's Responses API, supercharged with advanced agentic capabilities and a no-code UI. Link: https://github.com/mudler/LocalAGI
LocalRecall	A RESTful API and knowledge base management system providing persistent memory and storage capabilities for AI agents. Designed to work alongside LocalAI and LocalAGI. Link: https://github.com/mudler/LocalRecall

Join the Movement! ❤️

A massive THANK YOU to our incredible community and our sponsors! LocalAI has over 33,300 stars, and LocalAGI has already rocketed past 750+ stars!

As a reminder, LocalAI is real FOSS (Free and Open Source Software) and its sibling projects are community-driven and not backed by VCs or a company. We rely on contributors donating their spare time and our sponsors to provide us the hardware! If you love open-source, privacy-first AI, please consider starring the repos, contributing code, reporting bugs, or spreading the word!

👉 Check out the reborn LocalAGI v2 today: https://github.com/mudler/LocalAGI

LocalAI 3.0.0 is here. What will you build next?

Full changelog 👇

👉 Click to expand 👈

What's Changed

New Contributors

@omahs made their first contribution in fix: typos #5376
@TheDarkTrumpet made their first contribution in updating the documentation on fine tuning and advanced guide. #5420
@halkeye made their first contribution in fix(install.sh): vulkan docker tag #5589
@leopardracer made their first contribution in Improve Comments and Documentation for MixedMode and ParseJSON Functions #5626
@kilavvy made their first contribution in Fix Typos in Comments and Error Messages #5637
@vtjl10 made their first contribution in Minor Documentation Updates: Clarified Comments in Python and Go Files #5641
@zeevick10 made their first contribution in Fix Typos and Improve Documentation Clarity #5648
@maximevtush made their first contribution in Fix Typos in Documentation and Python Comments #5658

Full Changelog: v2.29.0...v3.0.0-alpha1

This discussion was created from the release v3.0.0.

richiejp · 2025-06-19T16:07:53Z

richiejp
Jun 19, 2025
Collaborator

Great! With this new release the realtime voice transcription mode in https://github.com/richiejp/VoxInput will be functional because it uses the realtime API.

Also being able to download backends is a big step forward UX IMO!

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

v3.0.0 #5689

Uh oh!

{{title}}

Uh oh!

LocalAI

LocalAGI

LocalRecall

What's Changed

Breaking Changes 🛠

Bug fixes 🐛

Exciting New Features 🎉

🧠 Models

📖 Documentation and examples

👒 Dependencies

Other Changes

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

v3.0.0 #5689

Uh oh!

mudler Jun 19, 2025 Maintainer

🚀 LocalAI 3.0 – A New Era Begins

TL;DR – What’s New in LocalAI 3.0.0 🎉

🧩 Introducing the Backend Gallery — Plug, Play, Power Up

⚠️ Important: Breaking Changes

CPU only image:

NVIDIA GPU Images:

AMD GPU Images (ROCm):

Intel GPU Images (oneAPI):

Vulkan GPU Images:

AIO Images (pre-downloaded models):

🧠 Smarter Reasoning, Smoother Chat

🧠 Model Power-Up: VRAM Savvy + Multimodal Brains

🧪 New Models!

🐞 Bugfixes & Polish

The Complete Local Stack for Privacy-First AI

LocalAI

LocalAGI

LocalRecall

Join the Movement! ❤️

Full changelog 👇

What's Changed

Breaking Changes 🛠

Bug fixes 🐛

Exciting New Features 🎉

🧠 Models

📖 Documentation and examples

👒 Dependencies

Other Changes

New Contributors

Replies: 1 comment

Uh oh!

richiejp Jun 19, 2025 Collaborator

mudler
Jun 19, 2025
Maintainer

richiejp
Jun 19, 2025
Collaborator