LibreChat + Ollama Integration

Complete configuration setup for integrating Ollama local LLM models with LibreChat.

Overview

This repository contains the configuration files and documentation for running LibreChat with Ollama as a custom endpoint, enabling you to use local LLM models alongside (or instead of) cloud-based AI providers.

What's Included

• librechat.yaml - Custom endpoint configuration for Ollama
• docker-compose.override.yml - Docker volume mount configuration
• OLLAMA_INTEGRATION_COMPLETE_GUIDE.md - Comprehensive implementation guide (1000+ lines)
• OLLAMA_INTEGRATION_PLAN.md - Step-by-step execution plan

Quick Start

Prerequisites

• LibreChat installed
• Ollama running locally
• Docker and Docker Compose (if using Docker setup)

Installation

1. Clone this repository into your LibreChat directory:

   cd /path/to/LibreChat
   git clone https://github.com/dinkargupta/librechat-ollama-config.git tmp-ollama-config
   cp tmp-ollama-config/librechat.yaml .
   cp tmp-ollama-config/docker-compose.override.yml .
   rm -rf tmp-ollama-config

2. Start Ollama with correct binding:

   OLLAMA_HOST=0.0.0.0:11434 ollama serve

3. Pull some Ollama models:

   ollama pull llama3.1:8b
   ollama pull llama3:latest
   ollama pull qwen3:14b

4. Restart LibreChat:

   # If using Docker:
   docker compose up -d
   
   # If using npm:
   npm run backend

5. Verify in browser:

   • Open http://localhost:3080
   • Look for "Ollama" in the endpoint selector
   • Select a model and start chatting!

Configuration Details

librechat.yaml

The main configuration file that defines Ollama as a custom endpoint:

• API Endpoint: http://host.docker.internal:11434/v1 (Docker) or http://localhost:11434/v1 (local)
• Model Fetching: Dynamic - automatically discovers available Ollama models
• Title Generation: Uses llama3:latest for conversation titles
• Parameter Dropping: Removes unsupported OpenAI parameters to prevent errors

docker-compose.override.yml

Mounts the librechat.yaml configuration into the Docker container:

services:
  api:
    volumes:
      - type: bind
        source: ./librechat.yaml
        target: /app/librechat.yaml

Documentation

Complete Implementation Guide

See OLLAMA_INTEGRATION_COMPLETE_GUIDE.md for:

• Architecture overview and diagrams
• Detailed configuration explanations
• Troubleshooting guides with solutions
• Docker networking setup
• Service binding requirements
• Performance tuning options
• Security considerations
• Complete troubleshooting flowcharts

Execution Plan

See OLLAMA_INTEGRATION_PLAN.md for:

• Step-by-step implementation instructions
• Current state documentation
• Rollback procedures
• Validation tests
• Safety guarantees

Key Features

• Zero Code Changes: Pure configuration approach
• Dynamic Model Discovery: Automatically fetches available Ollama models
• Dual Endpoints: Run both Ollama and cloud providers simultaneously
• Docker Compatible: Proper networking for Docker environments
• Production Ready: Tested and verified working setup
• Safe Rollback: Complete rollback procedures included
• Comprehensive Docs: 1000+ lines of documentation

Troubleshooting

Ollama endpoint not appearing

# Check if config is mounted (Docker):
docker compose exec api ls -la /app/librechat.yaml

# Check server logs:
docker compose logs api | grep -i ollama

Connection refused errors

# Verify Ollama is accessible:
curl http://localhost:11434/api/tags

# Check Ollama binding:
ss -tlnp | grep 11434
# Should show: 0.0.0.0:11434 (not 127.0.0.1:11434)

# Fix: Restart with correct binding:
OLLAMA_HOST=0.0.0.0:11434 ollama serve

No models available

# List available models:
ollama list

# Pull models if needed:
ollama pull llama3:latest
ollama pull mistral

Architecture Notes

Docker Networking

• LibreChat container → Host machine: host.docker.internal
• Ollama must bind to 0.0.0.0 (all interfaces), not 127.0.0.1 (localhost only)
• Port 11434 must be accessible from Docker containers

Configuration Loading

1. LibreChat reads librechat.yaml from /app/librechat.yaml (inside container)
2. docker-compose.override.yml mounts host file into container
3. Custom endpoints are registered at startup
4. Models are fetched dynamically from Ollama API

Verified Setup

This configuration has been tested and verified working with:

• LibreChat: v0.8.2
• Ollama: Latest (February 2026)
• Docker Compose: v2.x
• Models Tested: gpt-oss:20b, llama3.1:8b, llama3:latest, qwen3:14b

Performance

• Response Time: Minimal latency (local connection)
• Streaming: Full support for real-time responses
• Model Switching: Instant switching between models
• Concurrent Endpoints: Seamlessly switch between Ollama and other providers

Contributing

Found an issue or have an improvement? Feel free to:

1. Open an issue describing the problem/suggestion
2. Submit a pull request with improvements
3. Share your custom configurations or models

License

Configuration files and documentation are provided as-is for use with LibreChat.

LibreChat is licensed under the MIT License - see the LibreChat repository for details.

Resources

• LibreChat Documentation
• Ollama Documentation
• LibreChat Custom Endpoints Guide
• Ollama OpenAI Compatibility

Credits

Implementation completed with Claude (Sonnet 4.5) on 2026-02-01.

Support

For LibreChat issues, see: https://github.com/danny-avila/LibreChat/issues For Ollama issues, see: https://github.com/ollama/ollama/issues

---

Status: ✅ Production Ready & Verified Working Last Updated: 2026-02-01