📚 DocuChat

Chat with your PDFs. Locally. No cloud, no API keys, no leaks.

DocuChat is a fully local Retrieval-Augmented Generation (RAG) app that lets you chat with your own PDF documents. It chunks and embeds your files with sentence-transformers, stores the vectors in a local ChromaDB collection, retrieves the most relevant chunks for a given question, and passes them as context to a locally-running llama3.2:3b model via Ollama — so your documents and questions never leave your machine. A Streamlit UI talks to a FastAPI backend that wraps the whole retrieval + generation pipeline behind a single /chat endpoint.

---

🚀 What is this

Your PDFs shouldn't have to leave your laptop just to get asked a question. DocuChat runs the entire RAG pipeline — embeddings, vector search, and generation — on your own machine. Drop some PDFs in a folder, index them, and start asking. Zero third-party API calls, zero data leaving your machine.

✨ Features

🔒 Fully local	Embeddings, vector storage, and LLM inference all run on your machine; nothing is sent to a third-party API.
📥 PDF ingestion	ingest.py reads every .pdf in data/, splits the extracted text into overlapping chunks (1200 chars, 150-char overlap), and embeds them with the all-MiniLM-L6-v2 sentence-transformer model.
🗃️ Persistent vector store	Chunks and embeddings are stored in a local, persistent ChromaDB collection (chroma_db/), so you only need to re-run ingestion when your documents change.
🎯 Source-grounded answers	rag.py retrieves the top-5 most similar chunks for a question and prompts the LLM to answer using only that context, citing the source filename in brackets.
⚡ FastAPI backend	A minimal POST /chat endpoint (app.py) wraps the retrieval + generation pipeline.
💬 Streamlit chat UI	ui.py provides a simple browser chat interface backed by the FastAPI API.
🖥️ CLI mode	Run python3 rag.py for a REPL-style loop that answers questions directly in the terminal, no API/UI needed.
🐳 Docker support	Dockerfile and docker-compose.yml build and run the FastAPI service in a container, persisting chroma_db/ via a bind-mounted volume and reaching the host's Ollama server through host.docker.internal.

📸 More screenshots

CLI mode (python3 rag.py), answering a question grounded in an ingested PDF, with retrieval done in under a second:

Pipeline architecture:

🛠️ Tech Stack

• Backend: FastAPI
• UI: Streamlit
• Vector store: ChromaDB (persistent, local)
• Embeddings: sentence-transformers (all-MiniLM-L6-v2)
• LLM: llama3.2:3b served locally via Ollama
• Containerization: Docker + Docker Compose

🧠 Architecture

The flow, end to end:

1. ingest.py reads PDFs from data/, splits them into overlapping text chunks, embeds them with all-MiniLM-L6-v2, and stores them in a persistent ChromaDB collection (chroma_db/).
2. rag.py embeds the incoming question, retrieves the top-5 most similar chunks from ChromaDB, and builds a prompt that instructs the model to answer using only that context (with source citations).
3. app.py exposes this as a FastAPI POST /chat endpoint.
4. ui.py is a Streamlit chat interface that calls the FastAPI backend and renders the conversation.
5. The actual text generation happens through a local Ollama server running llama3.2:3b.

✅ Prerequisites

• Python 3.13 (matches the version used in Dockerfile; other recent 3.x versions will likely work too).
• Ollama installed and running locally, with the llama3.2:3b model pulled (see setup below). rag.py and chat_test.py talk to Ollama's /api/chat endpoint on localhost:11434; when running the API in Docker, it's reached at host.docker.internal:11434 instead.
• Enough disk space/RAM to run llama3.2:3b and to download the all-MiniLM-L6-v2 embedding model (fetched automatically by sentence-transformers on first run).

⚙️ Setup

# 1. Clone the repo
git clone https://github.com/destivano/docuchat.git
cd docuchat

# 2. Create a virtual environment and install dependencies
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# 3. Pull the local LLM
ollama pull llama3.2:3b

# 4. Add your PDFs to data/, then build the vector index
python3 ingest.py

# 5. Run the API
fastapi run app.py --port 8000
# or with Docker: docker compose up --build

# 6. In a separate terminal, run the UI
streamlit run ui.py

Open the Streamlit URL printed in the terminal (usually http://localhost:8501) and start asking questions about your documents.

💬 Usage

Running without the API/UI

You can skip FastAPI and Streamlit entirely and just chat from the terminal once the vector index has been built:

python3 rag.py

This starts a simple REPL: type a question at the Ask: prompt and it prints the model's answer.

📦 Project Structure

.
├── app.py               # FastAPI app exposing POST /chat
├── ingest.py             # Reads PDFs from data/, chunks + embeds them, writes to ChromaDB
├── rag.py                # Embeds a question, retrieves context from ChromaDB, queries Ollama, and offers a CLI REPL
├── ui.py                 # Streamlit chat UI that calls the FastAPI /chat endpoint
├── chat_test.py           # Standalone example of streaming a chat request to Ollama
├── embed_test.py          # Standalone example of embedding a sentence with sentence-transformers
├── Dockerfile             # Builds the FastAPI service (python:3.13-slim base)
├── docker-compose.yml     # Runs the API container, mounting chroma_db/ and reaching host Ollama
├── requirements.txt       # Pinned Python dependencies
├── data/                  # Put your source PDFs here (created by you, git-ignored)
└── chroma_db/             # Persistent ChromaDB store (created by ingest.py, git-ignored)

📝 Notes

• data/ and chroma_db/ are git-ignored — you need to create data/ yourself and populate it with PDFs before running ingest.py.
• The embedding model and Ollama model are currently hard-coded (all-MiniLM-L6-v2 and llama3.2:3b respectively); there are no environment variables or config files to change them — edit ingest.py/rag.py directly if you want a different model.
• chat_test.py and embed_test.py are small standalone scripts used to sanity-check the Ollama chat endpoint and the embedding model in isolation; they aren't part of the app's runtime path.

📄 License

This project is licensed under the MIT License.