-
-
-
-> Note: the DMGs are not signed by Apple and shows quarantined after install. See https://github.com/mudler/LocalAI/issues/6268 for a workaround, fix is tracked here: https://github.com/mudler/LocalAI/issues/6244
-
-### Run with docker
-
-{{% alert icon="💡" %}}
-**Docker Run vs Docker Start**
-
-- `docker run` creates and starts a new container. If a container with the same name already exists, this command will fail.
-- `docker start` starts an existing container that was previously created with `docker run`.
-
-If you've already run LocalAI before and want to start it again, use: `docker start -i local-ai`
-{{% /alert %}}
-
-The following commands will automatically start with a web interface and a Rest API on port `8080`.
-
-#### CPU only image:
-
-```bash
-docker run -ti --name local-ai -p 8080:8080 localai/localai:latest
-```
-
-#### NVIDIA GPU Images:
-
-```bash
-# CUDA 12.0
-docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12
-
-# CUDA 11.7
-docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-11
-
-# NVIDIA Jetson (L4T) ARM64
-# First, you need to have installed the nvidia container toolkit: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installing-with-ap
-docker run -ti --name local-ai -p 8080:8080 --runtime nvidia --gpus all localai/localai:latest-nvidia-l4t-arm64
-```
-
-#### AMD GPU Images (ROCm):
-
-```bash
-docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas
-```
-
-#### Intel GPU Images (oneAPI):
-
-```bash
-docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-intel
-```
-
-#### Vulkan GPU Images:
-
-```bash
-docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan
-```
-
-#### AIO Images (pre-downloaded models):
-
-```bash
-# CPU version
-docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-cpu
-
-# NVIDIA CUDA 12 version
-docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-12
-
-# NVIDIA CUDA 11 version
-docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-aio-gpu-nvidia-cuda-11
-
-# Intel GPU version
-docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-aio-gpu-intel
-
-# AMD GPU version
-docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-aio-gpu-hipblas
-```
-
-### Downloading models on start
-
-When starting LocalAI (either via Docker or via CLI) you can specify as argument a list of models to install automatically before starting the API, for example:
-
-```bash
-# From the model gallery (see available models with `local-ai models list`, in the WebUI from the model tab, or visiting https://models.localai.io)
-local-ai run llama-3.2-1b-instruct:q4_k_m
-# Start LocalAI with the phi-2 model directly from huggingface
-local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf
-# Install and run a model from the Ollama OCI registry
-local-ai run ollama://gemma:2b
-# Run a model from a configuration file
-local-ai run https://gist.githubusercontent.com/.../phi-2.yaml
-# Install and run a model from a standard OCI registry (e.g., Docker Hub)
-local-ai run oci://localai/phi-2:latest
-```
-
-{{% alert icon="⚡" %}}
-**Automatic Backend Detection**: When you install models from the gallery or YAML files, LocalAI automatically detects your system's GPU capabilities (NVIDIA, AMD, Intel) and downloads the appropriate backend. For advanced configuration options, see [GPU Acceleration]({{% relref "docs/features/gpu-acceleration#automatic-backend-detection" %}}).
-{{% /alert %}}
-
-For a full list of options, you can run LocalAI with `--help` or refer to the [Installer Options]({{% relref "docs/advanced/installer" %}}) documentation.
-
-Binaries can also be [manually downloaded]({{% relref "docs/reference/binaries" %}}).
-
-## Using Homebrew on MacOS
-
-{{% alert icon="⚠️" %}}
-The Homebrew formula currently doesn't have the same options than the bash script
-{{% /alert %}}
-
-You can install Homebrew's [LocalAI](https://formulae.brew.sh/formula/localai) with the following command:
-
-```
-brew install localai
-```
-
-
-## Using Container Images or Kubernetes
-
-LocalAI is available as a container image compatible with various container engines such as Docker, Podman, and Kubernetes. Container images are published on [quay.io](https://quay.io/repository/go-skynet/local-ai?tab=tags&tag=latest) and [Docker Hub](https://hub.docker.com/r/localai/localai).
-
-For detailed instructions, see [Using container images]({{% relref "docs/getting-started/container-images" %}}). For Kubernetes deployment, see [Run with Kubernetes]({{% relref "docs/getting-started/kubernetes" %}}).
-
-## Running LocalAI with All-in-One (AIO) Images
-
-> _Already have a model file? Skip to [Run models manually]({{% relref "docs/getting-started/models" %}})_.
-
-LocalAI's All-in-One (AIO) images are pre-configured with a set of models and backends to fully leverage almost all the features of LocalAI. If pre-configured models are not required, you can use the standard [images]({{% relref "docs/getting-started/container-images" %}}).
-
-These images are available for both CPU and GPU environments. AIO images are designed for ease of use and require no additional configuration.
-
-It is recommended to use AIO images if you prefer not to configure the models manually or via the web interface. For running specific models, refer to the [manual method]({{% relref "docs/getting-started/models" %}}).
-
-The AIO images come pre-configured with the following features:
-- Text to Speech (TTS)
-- Speech to Text
-- Function calling
-- Large Language Models (LLM) for text generation
-- Image generation
-- Embedding server
-
-For instructions on using AIO images, see [Using container images]({{% relref "docs/getting-started/container-images#all-in-one-images" %}}).
-
-## Using LocalAI and the full stack with LocalAGI
-
-LocalAI is part of the Local family stack, along with LocalAGI and LocalRecall.
-
-[LocalAGI](https://github.com/mudler/LocalAGI) is a powerful, self-hostable AI Agent platform designed for maximum privacy and flexibility which encompassess and uses all the software stack. It provides a complete drop-in replacement for OpenAI's Responses APIs with advanced agentic capabilities, working entirely locally on consumer-grade hardware (CPU and GPU).
-
-### Quick Start
-
-```bash
-# Clone the repository
-git clone https://github.com/mudler/LocalAGI
-cd LocalAGI
-
-# CPU setup (default)
-docker compose up
-
-# NVIDIA GPU setup
-docker compose -f docker-compose.nvidia.yaml up
-
-# Intel GPU setup (for Intel Arc and integrated GPUs)
-docker compose -f docker-compose.intel.yaml up
-
-# Start with a specific model (see available models in models.localai.io, or localai.io to use any model in huggingface)
-MODEL_NAME=gemma-3-12b-it docker compose up
-
-# NVIDIA GPU setup with custom multimodal and image models
-MODEL_NAME=gemma-3-12b-it \
-MULTIMODAL_MODEL=minicpm-v-4_5 \
-IMAGE_MODEL=flux.1-dev-ggml \
-docker compose -f docker-compose.nvidia.yaml up
-```
-
-### Key Features
-
-- **Privacy-Focused**: All processing happens locally, ensuring your data never leaves your machine
-- **Flexible Deployment**: Supports CPU, NVIDIA GPU, and Intel GPU configurations
-- **Multiple Model Support**: Compatible with various models from Hugging Face and other sources
-- **Web Interface**: User-friendly chat interface for interacting with AI agents
-- **Advanced Capabilities**: Supports multimodal models, image generation, and more
-- **Docker Integration**: Easy deployment using Docker Compose
-
-### Environment Variables
-
-You can customize your LocalAGI setup using the following environment variables:
-
-- `MODEL_NAME`: Specify the model to use (e.g., `gemma-3-12b-it`)
-- `MULTIMODAL_MODEL`: Set a custom multimodal model
-- `IMAGE_MODEL`: Configure an image generation model
-
-For more advanced configuration and API documentation, visit the [LocalAGI GitHub repository](https://github.com/mudler/LocalAGI).
-
-## What's Next?
-
-There is much more to explore with LocalAI! You can run any model from Hugging Face, perform video generation, and also voice cloning. For a comprehensive overview, check out the [features]({{% relref "docs/features" %}}) section.
-
-Explore additional resources and community contributions:
-
-- [Installer Options]({{% relref "docs/advanced/installer" %}})
-- [Run from Container images]({{% relref "docs/getting-started/container-images" %}})
-- [Examples to try from the CLI]({{% relref "docs/getting-started/try-it-out" %}})
-- [Build LocalAI and the container image]({{% relref "docs/getting-started/build" %}})
-- [Run models manually]({{% relref "docs/getting-started/models" %}})
-- [Examples](https://github.com/mudler/LocalAI/tree/master/examples#examples)
diff --git a/docs/content/docs/faq.md b/docs/content/faq.md
similarity index 95%
rename from docs/content/docs/faq.md
rename to docs/content/faq.md
index 475b0efeb6a3..afb6459e3b00 100644
--- a/docs/content/docs/faq.md
+++ b/docs/content/faq.md
@@ -46,7 +46,7 @@ Model sizes vary significantly depending on the model and quantization level:
### Benchmarking LocalAI and llama.cpp shows different results!
-LocalAI applies a set of defaults when loading models with the llama.cpp backend, one of these is mirostat sampling - while it achieves better results, it slows down the inference. You can disable this by setting `mirostat: 0` in the model config file. See also the advanced section ({{%relref "docs/advanced/advanced-usage" %}}) for more information and [this issue](https://github.com/mudler/LocalAI/issues/2780).
+LocalAI applies a set of defaults when loading models with the llama.cpp backend, one of these is mirostat sampling - while it achieves better results, it slows down the inference. You can disable this by setting `mirostat: 0` in the model config file. See also the advanced section ({{%relref "advanced/advanced-usage" %}}) for more information and [this issue](https://github.com/mudler/LocalAI/issues/2780).
### What's the difference with Serge, or XXX?
@@ -66,7 +66,7 @@ Yes! If the client uses OpenAI and supports setting a different base URL to send
### Can this leverage GPUs?
-There is GPU support, see {{%relref "docs/features/GPU-acceleration" %}}.
+There is GPU support, see {{%relref "features/GPU-acceleration" %}}.
### Where is the webUI?
diff --git a/docs/content/docs/features/GPU-acceleration.md b/docs/content/features/GPU-acceleration.md
similarity index 97%
rename from docs/content/docs/features/GPU-acceleration.md
rename to docs/content/features/GPU-acceleration.md
index 00bdfbdae9ee..7c619962b450 100644
--- a/docs/content/docs/features/GPU-acceleration.md
+++ b/docs/content/features/GPU-acceleration.md
@@ -5,15 +5,15 @@ weight = 9
url = "/features/gpu-acceleration/"
+++
-{{% alert context="warning" %}}
+{{% notice context="warning" %}}
Section under construction
-{{% /alert %}}
+ {{% /notice %}}
This section contains instruction on how to use LocalAI with GPU acceleration.
-{{% alert icon="⚡" context="warning" %}}
-For acceleration for AMD or Metal HW is still in development, for additional details see the [build]({{%relref "docs/getting-started/build#Acceleration" %}})
-{{% /alert %}}
+{{% notice icon="⚡" context="warning" %}}
+For acceleration for AMD or Metal HW is still in development, for additional details see the [build]({{%relref "installation/build#Acceleration" %}})
+ {{% /notice %}}
## Automatic Backend Detection
@@ -32,7 +32,6 @@ Depending on the model architecture and backend used, there might be different w
```yaml
name: my-model-name
-# Default model parameters
parameters:
# Relative to the models path
model: llama.cpp-model.ggmlv3.q5_K_M.bin
@@ -124,7 +123,7 @@ llama_init_from_file: kv self size = 512.00 MB
There are a limited number of tested configurations for ROCm systems however most newer deditated GPU consumer grade devices seem to be supported under the current ROCm6 implementation.
-Due to the nature of ROCm it is best to run all implementations in containers as this limits the number of packages required for installation on host system, compatibility and package versions for dependencies across all variations of OS must be tested independently if desired, please refer to the [build]({{%relref "docs/getting-started/build#Acceleration" %}}) documentation.
+Due to the nature of ROCm it is best to run all implementations in containers as this limits the number of packages required for installation on host system, compatibility and package versions for dependencies across all variations of OS must be tested independently if desired, please refer to the [build]({{%relref "installation/build#Acceleration" %}}) documentation.
### Requirements
@@ -181,7 +180,6 @@ The devices in the following list have been tested with `hipblas` images running
The following are examples of the ROCm specific configuration elements required.
```yaml
-# docker-compose.yaml
# For full functionality select a non-'core' image, version locking the image is recommended for debug purposes.
image: quay.io/go-skynet/local-ai:master-aio-gpu-hipblas
environment:
diff --git a/docs/content/features/_index.en.md b/docs/content/features/_index.en.md
new file mode 100644
index 000000000000..98be6d04ab54
--- /dev/null
+++ b/docs/content/features/_index.en.md
@@ -0,0 +1,38 @@
++++
+disableToc = false
+title = "Features"
+weight = 8
+icon = "lightbulb"
+type = "chapter"
+url = "/features/"
++++
+
+LocalAI provides a comprehensive set of features for running AI models locally. This section covers all the capabilities and functionalities available in LocalAI.
+
+## Core Features
+
+- **[Text Generation](text-generation/)** - Generate text with GPT-compatible models using various backends
+- **[Image Generation](image-generation/)** - Create images with Stable Diffusion and other diffusion models
+- **[Audio Processing](audio-to-text/)** - Transcribe audio to text and generate speech from text
+- **[Embeddings](embeddings/)** - Generate vector embeddings for semantic search and RAG applications
+- **[GPT Vision](gpt-vision/)** - Analyze and understand images with vision-language models
+
+## Advanced Features
+
+- **[OpenAI Functions](openai-functions/)** - Use function calling and tools API with local models
+- **[Constrained Grammars](constrained_grammars/)** - Control model output format with BNF grammars
+- **[GPU Acceleration](GPU-acceleration/)** - Optimize performance with GPU support
+- **[Distributed Inference](distributed_inferencing/)** - Scale inference across multiple nodes
+- **[Model Context Protocol (MCP)](mcp/)** - Enable agentic capabilities with MCP integration
+
+## Specialized Features
+
+- **[Object Detection](object-detection/)** - Detect and locate objects in images
+- **[Reranker](reranker/)** - Improve retrieval accuracy with cross-encoder models
+- **[Stores](stores/)** - Vector similarity search for embeddings
+- **[Model Gallery](model-gallery/)** - Browse and install pre-configured models
+- **[Backends](backends/)** - Learn about available backends and how to manage them
+
+## Getting Started
+
+To start using these features, make sure you have [LocalAI installed](/installation/) and have [downloaded some models](/getting-started/models/). Then explore the feature pages above to learn how to use each capability.
diff --git a/docs/content/docs/features/audio-to-text.md b/docs/content/features/audio-to-text.md
similarity index 99%
rename from docs/content/docs/features/audio-to-text.md
rename to docs/content/features/audio-to-text.md
index db8aa1e74091..2b91a8071e66 100644
--- a/docs/content/docs/features/audio-to-text.md
+++ b/docs/content/features/audio-to-text.md
@@ -41,4 +41,4 @@ curl http://localhost:8080/v1/audio/transcriptions -H "Content-Type: multipart/f
## Result
{"text":"My fellow Americans, this day has brought terrible news and great sadness to our country.At nine o'clock this morning, Mission Control in Houston lost contact with our Space ShuttleColumbia.A short time later, debris was seen falling from the skies above Texas.The Columbia's lost.There are no survivors.One board was a crew of seven.Colonel Rick Husband, Lieutenant Colonel Michael Anderson, Commander Laurel Clark, Captain DavidBrown, Commander William McCool, Dr. Kultna Shavla, and Elon Ramon, a colonel in the IsraeliAir Force.These men and women assumed great risk in the service to all humanity.In an age when spaceflight has come to seem almost routine, it is easy to overlook thedangers of travel by rocket and the difficulties of navigating the fierce outer atmosphere ofthe Earth.These astronauts knew the dangers, and they faced them willingly, knowing they had a highand noble purpose in life.Because of their courage and daring and idealism, we will miss them all the more.All Americans today are thinking as well of the families of these men and women who havebeen given this sudden shock and grief.You're not alone.Our entire nation agrees with you, and those you loved will always have the respect andgratitude of this country.The cause in which they died will continue.Mankind has led into the darkness beyond our world by the inspiration of discovery andthe longing to understand.Our journey into space will go on.In the skies today, we saw destruction and tragedy.As farther than we can see, there is comfort and hope.In the words of the prophet Isaiah, \"Lift your eyes and look to the heavens who createdall these, he who brings out the starry hosts one by one and calls them each by name.\"Because of his great power and mighty strength, not one of them is missing.The same creator who names the stars also knows the names of the seven souls we mourntoday.The crew of the shuttle Columbia did not return safely to Earth yet we can pray that all aresafely home.May God bless the grieving families and may God continue to bless America.[BLANK_AUDIO]"}
-```
+```
\ No newline at end of file
diff --git a/docs/content/docs/features/backends.md b/docs/content/features/backends.md
similarity index 98%
rename from docs/content/docs/features/backends.md
rename to docs/content/features/backends.md
index 2c80b91a1080..ef71a87075b8 100644
--- a/docs/content/docs/features/backends.md
+++ b/docs/content/features/backends.md
@@ -1,11 +1,10 @@
---
-title: "Backends"
+title: "⚙️ Backends"
description: "Learn how to use, manage, and develop backends in LocalAI"
weight: 4
url: "/backends/"
---
-# Backends
LocalAI supports a variety of backends that can be used to run different types of AI models. There are core Backends which are included, and there are containerized applications that provide the runtime environment for specific model types, such as LLMs, diffusion models, or text-to-speech models.
@@ -53,7 +52,6 @@ Where URI is the path to an OCI container image.
A backend gallery is a collection of YAML files, each defining a backend. Here's an example structure:
```yaml
-# backends/llm-backend.yaml
name: "llm-backend"
description: "A backend for running LLM models"
uri: "quay.io/username/llm-backend:latest"
diff --git a/docs/content/features/constrained_grammars.md b/docs/content/features/constrained_grammars.md
new file mode 100644
index 000000000000..33d50c900ba5
--- /dev/null
+++ b/docs/content/features/constrained_grammars.md
@@ -0,0 +1,72 @@
++++
+disableToc = false
+title = "✍️ Constrained Grammars"
+weight = 15
+url = "/features/constrained_grammars/"
++++
+
+## Overview
+
+The `chat` endpoint supports the `grammar` parameter, which allows users to specify a grammar in Backus-Naur Form (BNF). This feature enables the Large Language Model (LLM) to generate outputs adhering to a user-defined schema, such as `JSON`, `YAML`, or any other format that can be defined using BNF. For more details about BNF, see [Backus-Naur Form on Wikipedia](https://en.wikipedia.org/wiki/Backus%E2%80%93Naur_form).
+
+{{% notice note %}}
+**Compatibility Notice:** This feature is only supported by models that use the [llama.cpp](https://github.com/ggerganov/llama.cpp) backend. For a complete list of compatible models, refer to the [Model Compatibility]({{%relref "reference/compatibility-table" %}}) page. For technical details, see the related pull requests: [PR #1773](https://github.com/ggerganov/llama.cpp/pull/1773) and [PR #1887](https://github.com/ggerganov/llama.cpp/pull/1887).
+ {{% /notice %}}
+
+## Setup
+
+To use this feature, follow the installation and setup instructions on the [LocalAI Functions]({{%relref "features/openai-functions" %}}) page. Ensure that your local setup meets all the prerequisites specified for the llama.cpp backend.
+
+## 💡 Usage Example
+
+The following example demonstrates how to use the `grammar` parameter to constrain the model's output to either "yes" or "no". This can be particularly useful in scenarios where the response format needs to be strictly controlled.
+
+### Example: Binary Response Constraint
+
+```bash
+curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
+ "model": "gpt-4",
+ "messages": [{"role": "user", "content": "Do you like apples?"}],
+ "grammar": "root ::= (\"yes\" | \"no\")"
+}'
+```
+
+In this example, the `grammar` parameter is set to a simple choice between "yes" and "no", ensuring that the model's response adheres strictly to one of these options regardless of the context.
+
+### Example: JSON Output Constraint
+
+You can also use grammars to enforce JSON output format:
+
+```bash
+curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
+ "model": "gpt-4",
+ "messages": [{"role": "user", "content": "Generate a person object with name and age"}],
+ "grammar": "root ::= \"{\" \"\\\"name\\\":\" string \",\\\"age\\\":\" number \"}\"\nstring ::= \"\\\"\" [a-z]+ \"\\\"\"\nnumber ::= [0-9]+"
+}'
+```
+
+### Example: YAML Output Constraint
+
+Similarly, you can enforce YAML format:
+
+```bash
+curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
+ "model": "gpt-4",
+ "messages": [{"role": "user", "content": "Generate a YAML list of fruits"}],
+ "grammar": "root ::= \"fruits:\" newline (\" - \" string newline)+\nstring ::= [a-z]+\nnewline ::= \"\\n\""
+}'
+```
+
+## Advanced Usage
+
+For more complex grammars, you can define multi-line BNF rules. The grammar parser supports:
+- Alternation (`|`)
+- Repetition (`*`, `+`)
+- Optional elements (`?`)
+- Character classes (`[a-z]`)
+- String literals (`"text"`)
+
+## Related Features
+
+- [OpenAI Functions]({{%relref "features/openai-functions" %}}) - Function calling with structured outputs
+- [Text Generation]({{%relref "features/text-generation" %}}) - General text generation capabilities
\ No newline at end of file
diff --git a/docs/content/docs/features/distributed_inferencing.md b/docs/content/features/distributed_inferencing.md
similarity index 77%
rename from docs/content/docs/features/distributed_inferencing.md
rename to docs/content/features/distributed_inferencing.md
index a3837633154c..9b42dea59d2f 100644
--- a/docs/content/docs/features/distributed_inferencing.md
+++ b/docs/content/features/distributed_inferencing.md
@@ -49,11 +49,11 @@ The instructions are displayed in the "Swarm" section of the WebUI, guiding you
### Workers mode
-{{% alert note %}}
+{{% notice note %}}
This feature is available exclusively with llama-cpp compatible models.
This feature was introduced in [LocalAI pull request #2324](https://github.com/mudler/LocalAI/pull/2324) and is based on the upstream work in [llama.cpp pull request #6829](https://github.com/ggerganov/llama.cpp/pull/6829).
-{{% /alert %}}
+ {{% /notice %}}
To connect multiple workers to a single LocalAI instance, start first a server in p2p mode:
@@ -90,7 +90,6 @@ Use the WebUI to guide you in the process of starting new workers. This example
```bash
./local-ai run --p2p
-# Get the token in the Swarm section of the WebUI
```
Copy the token from the WebUI or via API call (e.g., `curl http://localhost:8000/p2p/token`) and save it for later use.
@@ -101,19 +100,6 @@ To reuse the same token later, restart the server with `--p2ptoken` or `P2P_TOKE
```bash
TOKEN=XXX ./local-ai worker p2p-llama-cpp-rpc --llama-cpp-args="-m
+
+
+## Installation Steps
+
+1. Download the `LocalAI.dmg` file from the link above
+2. Open the downloaded DMG file
+3. Drag the LocalAI application to your Applications folder
+4. Launch LocalAI from your Applications folder
+
+## Known Issues
+
+> **Note**: The DMGs are not signed by Apple and may show as quarantined.
+>
+> **Workaround**: See [this issue](https://github.com/mudler/LocalAI/issues/6268) for details on how to bypass the quarantine.
+>
+> **Fix tracking**: The signing issue is being tracked in [this issue](https://github.com/mudler/LocalAI/issues/6244).
+
+## Next Steps
+
+After installing LocalAI, you can:
+
+- Access the WebUI at `http://localhost:8080`
+- [Try it out with examples](/basics/try/)
+- [Learn about available models](/models/)
+- [Customize your configuration](/advanced/model-configuration/)
diff --git a/docs/content/docs/integrations.md b/docs/content/integrations.md
similarity index 100%
rename from docs/content/docs/integrations.md
rename to docs/content/integrations.md
diff --git a/docs/content/docs/overview.md b/docs/content/overview.md
similarity index 77%
rename from docs/content/docs/overview.md
rename to docs/content/overview.md
index 5e3e6d742fb2..850991b98d91 100644
--- a/docs/content/docs/overview.md
+++ b/docs/content/overview.md
@@ -5,11 +5,11 @@ toc = true
description = "What is LocalAI?"
tags = ["Beginners"]
categories = [""]
+url = "/docs/overview"
author = "Ettore Di Giacinto"
icon = "info"
+++
-# Welcome to LocalAI
LocalAI is your complete AI stack for running AI models locally. It's designed to be simple, efficient, and accessible, providing a drop-in replacement for OpenAI's API while keeping your data private and secure.
@@ -51,34 +51,17 @@ LocalAI is more than just a single tool - it's a complete ecosystem:
## Getting Started
+LocalAI can be installed in several ways. **Docker is the recommended installation method** for most users as it provides the easiest setup and works across all platforms.
-### macOS Download
+### Recommended: Docker Installation
-You can use the DMG application for Mac:
-
-
-
-
-
-> Note: the DMGs are not signed by Apple shows as quarantined. See https://github.com/mudler/LocalAI/issues/6268 for a workaround, fix is tracked here: https://github.com/mudler/LocalAI/issues/6244
-
-## Docker
-
-You can use Docker for a quick start:
+The quickest way to get started with LocalAI is using Docker:
```bash
-docker run -p 8080:8080 --name local-ai -ti localai/localai:latest-aio-cpu
+docker run -p 8080:8080 --name local-ai -ti localai/localai:latest
```
-For more detailed installation options and configurations, see our [Getting Started guide](/basics/getting_started/).
-
-## One-liner
-
-The fastest way to get started is with our one-line installer (Linux):
-
-```bash
-curl https://localai.io/install.sh | sh
-```
+For complete installation instructions including Docker, macOS, Linux, Kubernetes, and building from source, see the [Installation guide](/installation/).
## Key Features
@@ -104,7 +87,7 @@ LocalAI is a community-driven project. You can:
Ready to dive in? Here are some recommended next steps:
-1. [Install LocalAI](/basics/getting_started/)
+1. **[Install LocalAI](/installation/)** - Start with [Docker installation](/installation/docker/) (recommended) or choose another method
2. [Explore available models](https://models.localai.io)
3. [Model compatibility](/model-compatibility/)
4. [Try out examples](https://github.com/mudler/LocalAI-examples)
diff --git a/docs/content/docs/reference/_index.en.md b/docs/content/reference/_index.en.md
similarity index 92%
rename from docs/content/docs/reference/_index.en.md
rename to docs/content/reference/_index.en.md
index d8a8f2a71a06..ea6a150bf1fd 100644
--- a/docs/content/docs/reference/_index.en.md
+++ b/docs/content/reference/_index.en.md
@@ -2,6 +2,7 @@
weight: 23
title: "References"
description: "Reference"
+type: chapter
icon: menu_book
lead: ""
date: 2020-10-06T08:49:15+00:00
diff --git a/docs/content/docs/reference/architecture.md b/docs/content/reference/architecture.md
similarity index 96%
rename from docs/content/docs/reference/architecture.md
rename to docs/content/reference/architecture.md
index 23abe11171cb..9f701bc5d325 100644
--- a/docs/content/docs/reference/architecture.md
+++ b/docs/content/reference/architecture.md
@@ -7,7 +7,7 @@ weight = 25
LocalAI is an API written in Go that serves as an OpenAI shim, enabling software already developed with OpenAI SDKs to seamlessly integrate with LocalAI. It can be effortlessly implemented as a substitute, even on consumer-grade hardware. This capability is achieved by employing various C++ backends, including [ggml](https://github.com/ggerganov/ggml), to perform inference on LLMs using both CPU and, if desired, GPU. Internally LocalAI backends are just gRPC server, indeed you can specify and build your own gRPC server and extend LocalAI in runtime as well. It is possible to specify external gRPC server and/or binaries that LocalAI will manage internally.
-LocalAI uses a mixture of backends written in various languages (C++, Golang, Python, ...). You can check [the model compatibility table]({{%relref "docs/reference/compatibility-table" %}}) to learn about all the components of LocalAI.
+LocalAI uses a mixture of backends written in various languages (C++, Golang, Python, ...). You can check [the model compatibility table]({{%relref "reference/compatibility-table" %}}) to learn about all the components of LocalAI.
diff --git a/docs/content/docs/reference/binaries.md b/docs/content/reference/binaries.md
similarity index 96%
rename from docs/content/docs/reference/binaries.md
rename to docs/content/reference/binaries.md
index aa818ec7468e..224c72685311 100644
--- a/docs/content/docs/reference/binaries.md
+++ b/docs/content/reference/binaries.md
@@ -32,10 +32,10 @@ Otherwise, here are the links to the binaries:
| MacOS (arm64) | [Download](https://github.com/mudler/LocalAI/releases/download/{{< version >}}/local-ai-Darwin-arm64) |
-{{% alert icon="⚡" context="warning" %}}
+{{% notice icon="⚡" context="warning" %}}
Binaries do have limited support compared to container images:
- Python-based backends are not shipped with binaries (e.g. `bark`, `diffusers` or `transformers`)
- MacOS binaries and Linux-arm64 do not ship TTS nor `stablediffusion-cpp` backends
- Linux binaries do not ship `stablediffusion-cpp` backend
-{{% /alert %}}
+ {{% /notice %}}
diff --git a/docs/content/docs/reference/cli-reference.md b/docs/content/reference/cli-reference.md
similarity index 91%
rename from docs/content/docs/reference/cli-reference.md
rename to docs/content/reference/cli-reference.md
index 60569b66b746..356e7a14e045 100644
--- a/docs/content/docs/reference/cli-reference.md
+++ b/docs/content/reference/cli-reference.md
@@ -7,21 +7,18 @@ url = '/reference/cli-reference'
Complete reference for all LocalAI command-line interface (CLI) parameters and environment variables.
-> **Note:** All CLI flags can also be set via environment variables. Environment variables take precedence over CLI flags. See [.env files]({{%relref "docs/advanced/advanced-usage#env-files" %}}) for configuration file support.
+> **Note:** All CLI flags can also be set via environment variables. Environment variables take precedence over CLI flags. See [.env files]({{%relref "advanced/advanced-usage#env-files" %}}) for configuration file support.
## Global Flags
-{{< table "table-responsive" >}}
| Parameter | Default | Description | Environment Variable |
|-----------|---------|-------------|----------------------|
| `-h, --help` | | Show context-sensitive help | |
| `--log-level` | `info` | Set the level of logs to output [error,warn,info,debug,trace] | `$LOCALAI_LOG_LEVEL` |
| `--debug` | `false` | **DEPRECATED** - Use `--log-level=debug` instead. Enable debug logging | `$LOCALAI_DEBUG`, `$DEBUG` |
-{{< /table >}}
## Storage Flags
-{{< table "table-responsive" >}}
| Parameter | Default | Description | Environment Variable |
|-----------|---------|-------------|----------------------|
| `--models-path` | `BASEPATH/models` | Path containing models used for inferencing | `$LOCALAI_MODELS_PATH`, `$MODELS_PATH` |
@@ -30,11 +27,9 @@ Complete reference for all LocalAI command-line interface (CLI) parameters and e
| `--localai-config-dir` | `BASEPATH/configuration` | Directory for dynamic loading of certain configuration files (currently api_keys.json and external_backends.json) | `$LOCALAI_CONFIG_DIR` |
| `--localai-config-dir-poll-interval` | | Time duration to poll the LocalAI Config Dir if your system has broken fsnotify events (example: `1m`) | `$LOCALAI_CONFIG_DIR_POLL_INTERVAL` |
| `--models-config-file` | | YAML file containing a list of model backend configs (alias: `--config-file`) | `$LOCALAI_MODELS_CONFIG_FILE`, `$CONFIG_FILE` |
-{{< /table >}}
## Backend Flags
-{{< table "table-responsive" >}}
| Parameter | Default | Description | Environment Variable |
|-----------|---------|-------------|----------------------|
| `--backends-path` | `BASEPATH/backends` | Path containing backends used for inferencing | `$LOCALAI_BACKENDS_PATH`, `$BACKENDS_PATH` |
@@ -50,13 +45,11 @@ Complete reference for all LocalAI command-line interface (CLI) parameters and e
| `--watchdog-idle-timeout` | `15m` | Threshold beyond which an idle backend should be stopped | `$LOCALAI_WATCHDOG_IDLE_TIMEOUT`, `$WATCHDOG_IDLE_TIMEOUT` |
| `--enable-watchdog-busy` | `false` | Enable watchdog for stopping backends that are busy longer than the watchdog-busy-timeout | `$LOCALAI_WATCHDOG_BUSY`, `$WATCHDOG_BUSY` |
| `--watchdog-busy-timeout` | `5m` | Threshold beyond which a busy backend should be stopped | `$LOCALAI_WATCHDOG_BUSY_TIMEOUT`, `$WATCHDOG_BUSY_TIMEOUT` |
-{{< /table >}}
-For more information on VRAM management, see [VRAM and Memory Management]({{%relref "docs/advanced/vram-management" %}}).
+For more information on VRAM management, see [VRAM and Memory Management]({{%relref "advanced/vram-management" %}}).
## Models Flags
-{{< table "table-responsive" >}}
| Parameter | Default | Description | Environment Variable |
|-----------|---------|-------------|----------------------|
| `--galleries` | | JSON list of galleries | `$LOCALAI_GALLERIES`, `$GALLERIES` |
@@ -65,23 +58,19 @@ For more information on VRAM management, see [VRAM and Memory Management]({{%rel
| `--models` | | A list of model configuration URLs to load | `$LOCALAI_MODELS`, `$MODELS` |
| `--preload-models-config` | | A list of models to apply at startup. Path to a YAML config file | `$LOCALAI_PRELOAD_MODELS_CONFIG`, `$PRELOAD_MODELS_CONFIG` |
| `--load-to-memory` | | A list of models to load into memory at startup | `$LOCALAI_LOAD_TO_MEMORY`, `$LOAD_TO_MEMORY` |
-{{< /table >}}
> **Note:** You can also pass model configuration URLs as positional arguments: `local-ai run MODEL_URL1 MODEL_URL2 ...`
## Performance Flags
-{{< table "table-responsive" >}}
| Parameter | Default | Description | Environment Variable |
|-----------|---------|-------------|----------------------|
| `--f16` | `false` | Enable GPU acceleration | `$LOCALAI_F16`, `$F16` |
| `-t, --threads` | | Number of threads used for parallel computation. Usage of the number of physical cores in the system is suggested | `$LOCALAI_THREADS`, `$THREADS` |
| `--context-size` | | Default context size for models | `$LOCALAI_CONTEXT_SIZE`, `$CONTEXT_SIZE` |
-{{< /table >}}
## API Flags
-{{< table "table-responsive" >}}
| Parameter | Default | Description | Environment Variable |
|-----------|---------|-------------|----------------------|
| `--address` | `:8080` | Bind address for the API server | `$LOCALAI_ADDRESS`, `$ADDRESS` |
@@ -94,11 +83,9 @@ For more information on VRAM management, see [VRAM and Memory Management]({{%rel
| `--disable-gallery-endpoint` | `false` | Disable the gallery endpoints | `$LOCALAI_DISABLE_GALLERY_ENDPOINT`, `$DISABLE_GALLERY_ENDPOINT` |
| `--disable-metrics-endpoint` | `false` | Disable the `/metrics` endpoint | `$LOCALAI_DISABLE_METRICS_ENDPOINT`, `$DISABLE_METRICS_ENDPOINT` |
| `--machine-tag` | | If not empty, add that string to Machine-Tag header in each response. Useful to track response from different machines using multiple P2P federated nodes | `$LOCALAI_MACHINE_TAG`, `$MACHINE_TAG` |
-{{< /table >}}
## Hardening Flags
-{{< table "table-responsive" >}}
| Parameter | Default | Description | Environment Variable |
|-----------|---------|-------------|----------------------|
| `--disable-predownload-scan` | `false` | If true, disables the best-effort security scanner before downloading any files | `$LOCALAI_DISABLE_PREDOWNLOAD_SCAN` |
@@ -106,11 +93,9 @@ For more information on VRAM management, see [VRAM and Memory Management]({{%rel
| `--use-subtle-key-comparison` | `false` | If true, API Key validation comparisons will be performed using constant-time comparisons rather than simple equality. This trades off performance on each request for resilience against timing attacks | `$LOCALAI_SUBTLE_KEY_COMPARISON` |
| `--disable-api-key-requirement-for-http-get` | `false` | If true, a valid API key is not required to issue GET requests to portions of the web UI. This should only be enabled in secure testing environments | `$LOCALAI_DISABLE_API_KEY_REQUIREMENT_FOR_HTTP_GET` |
| `--http-get-exempted-endpoints` | `^/$,^/browse/?$,^/talk/?$,^/p2p/?$,^/chat/?$,^/text2image/?$,^/tts/?$,^/static/.*$,^/swagger.*$` | If `--disable-api-key-requirement-for-http-get` is overridden to true, this is the list of endpoints to exempt. Only adjust this in case of a security incident or as a result of a personal security posture review | `$LOCALAI_HTTP_GET_EXEMPTED_ENDPOINTS` |
-{{< /table >}}
## P2P Flags
-{{< table "table-responsive" >}}
| Parameter | Default | Description | Environment Variable |
|-----------|---------|-------------|----------------------|
| `--p2p` | `false` | Enable P2P mode | `$LOCALAI_P2P`, `$P2P` |
@@ -119,7 +104,6 @@ For more information on VRAM management, see [VRAM and Memory Management]({{%rel
| `--p2ptoken` | | Token for P2P mode (optional) | `$LOCALAI_P2P_TOKEN`, `$P2P_TOKEN`, `$TOKEN` |
| `--p2p-network-id` | | Network ID for P2P mode, can be set arbitrarily by the user for grouping a set of instances | `$LOCALAI_P2P_NETWORK_ID`, `$P2P_NETWORK_ID` |
| `--federated` | `false` | Enable federated instance | `$LOCALAI_FEDERATED`, `$FEDERATED` |
-{{< /table >}}
## Other Commands
@@ -142,20 +126,16 @@ Use `local-ai © 2023-2025 Ettore Di Giacinto
+ diff --git a/docs/themes/hugo-theme-relearn b/docs/themes/hugo-theme-relearn deleted file mode 100644 index 72f933727e11..000000000000 --- a/docs/themes/hugo-theme-relearn +++ /dev/null @@ -1 +0,0 @@ -9a020e7eadb7d8203f5b01b18756c72d94773ec9 \ No newline at end of file diff --git a/docs/themes/hugo-theme-relearn b/docs/themes/hugo-theme-relearn new file mode 160000 index 000000000000..f69a085322cc --- /dev/null +++ b/docs/themes/hugo-theme-relearn @@ -0,0 +1 @@ +Subproject commit f69a085322cc70b3e14577c0823c5bc781ec6bb2 diff --git a/docs/themes/lotusdocs b/docs/themes/lotusdocs deleted file mode 160000 index 975da91e839c..000000000000 --- a/docs/themes/lotusdocs +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 975da91e839cfdb5c20fb66961468e77b8a9f8fd