LocalAI is an open-source alternative to commercial AI APIs, offering a drop-in replacement that's fully compatible with OpenAI, Anthropic, and ElevenLabs API specifications. Run AI models locally or on-premises with complete data privacy and control. This fork is tailored towards local community action.
- Universal Compatibility: Works as a direct replacement for OpenAI and other major AI service APIs
- Local Processing: Run LLMs, generate images, and create audio without sending data to external servers
- Hardware Friendly: Operates on consumer-grade hardware without requiring a GPU
- Multiple Model Support: Compatible with various AI model families and architectures
- Privacy-First: Keep your data secure with 100% on-device processing
- Free & Open Source: Maintained by the community, led by Smith Clove
- Private AI deployments for enterprises
- Cost-effective AI development and testing
- Local AI inferencing for privacy-sensitive applications
- On-premises AI solutions for regulated industries
Whether you're looking to reduce AI infrastructure costs, protect sensitive data, or maintain complete control over your AI stack, LocalAI provides the tools you need for local AI inferencing without compromising on capabilities.
Run the installer script:
curl https://localai.io/install.sh | shOr run with docker:
# CPU only image:
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-cpu
# Nvidia GPU:
docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12
# CPU and GPU image (bigger size):
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
# AIO images (it will pre-download a set of models ready for use, see https://localai.io/basics/container/)
docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpuTo load models:
# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)
local-ai run llama-3.2-1b-instruct:q4_k_m
# Start LocalAI with the phi-2 model directly from huggingface
local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
# Install and run a model from the Ollama OCI registry
local-ai run ollama://gemma:2b
# Run a model from a configuration file
local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
# Install and run a model from a standard OCI registry (e.g., Docker Hub)
local-ai run oci://localai/phi-2:latest- Multimodal with vLLM and Video understanding
- Realtime API
- 🔥🔥 Distributed, P2P Global community pools
- WebUI improvements
- Backends v2
- Improving UX v2
- Moderation endpoint
- Vulkan
- Anthropic API
- 💸💸💸
If you want to help and contribute, reach out!
Build and deploy custom containers:
WebUIs:
- https://github.com/Jirubizu/localai-admin
- https://github.com/go-skynet/LocalAI-frontend
- QA-Pilot(An interactive chat project that leverages LocalAI LLMs for rapid understanding and navigation of GitHub code repository) https://github.com/reid41/QA-Pilot
Model galleries
Other:
- Helm chart https://github.com/go-skynet/helm-charts
- VSCode extension https://github.com/badgooooor/localai-vscode-plugin
- Langchain: https://python.langchain.com/docs/integrations/providers/localai/
- Terminal utility https://github.com/djcopley/ShellOracle
- Local Smart assistant https://github.com/mudler/LocalAGI
- Home Assistant https://github.com/sammcj/homeassistant-localai / https://github.com/drndos/hass-openai-custom-conversation / https://github.com/valentinfrlch/ha-gpt4vision
- Discord bot https://github.com/mudler/LocalAGI/tree/main/examples/discord
- Slack bot https://github.com/mudler/LocalAGI/tree/main/examples/slack
- Shell-Pilot(Interact with LLM using LocalAI models via pure shell scripts on your Linux or MacOS system) https://github.com/reid41/shell-pilot
- Telegram bot https://github.com/mudler/LocalAI/tree/master/examples/telegram-bot
- Another Telegram Bot https://github.com/JackBekket/Hellper
- Auto-documentation https://github.com/JackBekket/Reflexia
- Github bot which answer on issues, with code and documentation as context https://github.com/JackBekket/GitHelper
- Github Actions: https://github.com/marketplace/actions/start-localai
- Examples: https://github.com/mudler/LocalAI/tree/master/examples/
- LLM finetuning guide
- How to build locally
- How to install in Kubernetes
- Projects integrating LocalAI
- How tos section (curated by the seed community)
- Run Visual studio code with LocalAI (SUSE)
- 🆕 Run LocalAI on Jetson Nano Devkit
- Run LocalAI on AWS EKS with Pulumi
- Run LocalAI on AWS
- Create a slackbot for teams and OSS projects that answer to documentation
- LocalAI meets k8sgpt
- Question Answering on Documents locally with LangChain, LocalAI, Chroma, and GPT4All
- Tutorial to use k8sgpt with LocalAI
Do you find LocalAI useful?
Support the project by becoming a sponsor.
A huge thank you to our generous sponsors who support this project covering CI expenses.
LocalAI is a community-driven Italian project created by the brilliant Ettore Di Giacinto.
MIT - Author Ettore Di Giacinto mudler@localai.io
LocalAI couldn't have been built without the help of great software already available from the community. Thank you!
- llama.cpp
- https://github.com/tatsu-lab/stanford_alpaca
- https://github.com/cornelk/llama-go for the initial ideas
- https://github.com/antimatter15/alpaca.cpp
- https://github.com/EdVince/Stable-Diffusion-NCNN
- https://github.com/ggerganov/whisper.cpp
- https://github.com/rhasspy/piper
and of course
This is a community project, a special thanks to our contributors! 🤗