Chat with PDF — RAG-Powered Document Q&A

A Retrieval-Augmented Generation (RAG) pipeline that lets you ask questions about a PDF document and get concise, grounded answers. The system extracts text from a PDF, embeds it into chunks, builds a FAISS vector index, and uses Google Gemma 2B Instruct to generate answers from the most relevant retrieved passages.

How It Works

The notebook implements a classic RAG flow:

PDF → Text Extraction → Chunking → Embedding → FAISS Index
                                                  ↓
  Question → Query Embedding → Similarity Search (Top-k)
                                                  ↓
                        Retrieved Context + Question
                                                  ↓
                     Gemma 2B Instruct → Answer

1. Text extraction — PyPDF2 reads the PDF page by page.
2. Chunking — extracted text is split into overlapping word-based chunks (configurable size and overlap) so each chunk stays contextually coherent.
3. Embedding — sentence-transformers/all-MiniLM-L6-v2 converts chunks into dense vector representations.
4. Indexing — FAISS (IndexFlatL2) stores the embeddings for fast nearest-neighbor search.
5. Retrieval — the user's question is embedded and matched against the index to fetch the top-k most relevant chunks.
6. Generation — the retrieved chunks are injected into a prompt as context, and Gemma 2B Instruct produces a concise answer. If the context doesn't contain the answer, the model responds with Information not found.

Features

• 🤖 Local LLM inference with Google Gemma 2B Instruct
• 🔎 Semantic search with sentence-transformers + FAISS
• 📄 PDF text extraction with PyPDF2
• ⚙️ Configurable chunk size and overlap
• 🛡️ Grounded answers — the model only uses retrieved context (with an Information not found fallback)

Requirements

• Python 3.10+
• PyTorch
• Transformers
• accelerate
• sentence-transformers
• PyPDF2
• faiss-cpu
• huggingface_hub

Installation

pip install -q transformers accelerate sentence-transformers PyPDF2 faiss-cpu huggingface_hub

Usage

1. Open Chat_PDF.ipynb in Jupyter Notebook, JupyterLab, or Kaggle.

2. (Optional) Log in to Hugging Face Hub to access gated Gemma weights:

   from huggingface_hub import login
   login("YOUR_HF_TOKEN")

3. Set pdf_path to your PDF file:

   pdf_path = "/path/to/your/document.pdf"

4. Run all cells. Example questions the notebook answers out of the box:

   • Where is the university located?
   • Does the university offer online programs?
   • Is there financial aid for international students?

Each question is printed along with the retrieved context and the model-generated answer.

Customizing

Parameter	Location	Description
chunk_size	chunk_text()	Number of words per chunk
overlap	chunk_text()	Overlapping words between consecutive chunks
k	search_index()	Number of retrieved chunks passed as context
model_name	top of notebook	Base generation model (default google/gemma-2b-it)

Models Used

Purpose	Model
Embeddings	sentence-transformers/all-MiniLM-L6-v2
Generation	google/gemma-2b-it

Roadmap

• Upload PDF from the UI or command-line argument instead of a hardcoded path
• Support for multiple documents
• Interactive chat loop
• Switch to a streaming web UI (Gradio/Streamlit)

License

This project is for educational purposes. Gemma models are subject to Google's Gemma Terms of Use.

---

Made with 💙 — a hands-on project on Retrieval-Augmented Generation with LLMs.