Skip to content

Installation

Anjana Tiha edited this page Mar 28, 2026 · 3 revisions

Installation

Option A — Docker (recommended for most users)

Run the full stack with no Python setup required.

# 1. Clone the repo
git clone https://github.com/anjanatiha/Retrieval-Augmented-Generation-RAG-Agent.git
cd Retrieval-Augmented-Generation-RAG-Agent

# 2. Pull models (one-time, ~2.3 GB)
docker compose run --rm ollama ollama pull hf.co/CompendiumLabs/bge-base-en-v1.5-gguf
docker compose run --rm ollama ollama pull hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF

# 3. Start
docker compose up

Open http://localhost:8501.

NVIDIA GPU:

docker compose -f docker-compose.yml -f docker-compose.gpu.yml up

Option B — Local Python Install

Prerequisites

Requirement Version Notes
Python 3.11 Exactly 3.11 — other versions are not tested
Ollama Latest Required to run local LLM and embedding models
RAM 8 GB minimum 16 GB recommended for comfortable performance
Disk ~5 GB For models and ChromaDB vector store

Step 1 — Install Ollama

Download and install Ollama from ollama.com/download.

Verify the installation:

ollama --version

Step 2 — Pull the Models

RAG Agent uses two models: one for embeddings and one for language generation and reranking.

# Embedding model (BGE base English, ~270 MB)
ollama pull hf.co/CompendiumLabs/bge-base-en-v1.5-gguf

# Language model (LLaMA 3.2 3B Instruct, ~2 GB)
ollama pull hf.co/bartowski/Llama-3.2-3B-Instruct-GGUF

This step only needs to be done once. Models are cached locally by Ollama.


Step 3 — Clone and Set Up the Project

git clone https://github.com/anjanatiha/Retrieval-Augmented-Generation-RAG-Agent.git
cd Retrieval-Augmented-Generation-RAG-Agent

# Create a Python 3.11 virtual environment
python3.11 -m venv rag_env_311

# Activate the virtual environment
source rag_env_311/bin/activate        # macOS / Linux
rag_env_311\Scripts\activate           # Windows

# Install all dependencies
pip install -r requirements.txt

Important: Always activate the virtual environment (source rag_env_311/bin/activate) before running any command. The system Python will not have the required packages.

Optional — development and evaluation tools:

# Pre-commit hooks (auto-lint before every commit)
pip install -e ".[dev]"
pre-commit install

# RAGAS evaluation
pip install -e ".[eval]"

Step 4 — Start Ollama

In a separate terminal window, start the Ollama server:

ollama serve

Leave this terminal open while using RAG Agent.


Step 5 — Run

# Web UI (recommended)
streamlit run app.py

# Terminal chatbot
python main.py

# Agent mode
python main.py --agent

# Benchmark evaluation
python main.py --benchmark

# RAGAS LLM-as-judge evaluation (requires: pip install -e ".[eval]")
python main.py --ragas

Open http://localhost:8501 for the web UI.


Verifying the Installation

If everything is set up correctly:

  1. The Streamlit UI loads without errors at http://localhost:8501
  2. The sidebar shows "0 chunks indexed" (no documents yet)
  3. Uploading a file or pasting a URL triggers indexing with no error messages

Next Steps

Clone this wiki locally