RAG Chatbot with Google Gemini and LangChain

An interactive RAG (Retrieval-Augmented Generation) chatbot designed to chat with your own documents, powered by Google Gemini, LangChain, and a user-friendly Streamlit interface.

---

Table of Contents

• Quick Start
• Usage Guide
• How It Works
• Project Structure
• Key Features
• Technology Stack
• Roadmap

---

Quick Start

Prerequisites

• Python 3.9 or higher
• Git
• Google API Key for Gemini model

Installation

# Clone the repository
git clone https://github.com/your-username/your-repo-name.git
cd your-repo-name

# Create and activate virtual environment
# Windows
python -m venv venv
.\venv\Scripts\Activate.ps1

# macOS / Linux
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

Running the Application

streamlit run app.py

Your default web browser will automatically open the application.

---

Usage Guide

Step 1: Set Up Your API Key

In the left sidebar, paste your Google API Key into the "Enter your Google API Key" field. The key is securely stored in st.session_state for the duration of your session and is never permanently saved.

Step 2: Upload Documents

• Drag and drop files (.pdf, .txt, .docx, .csv) into the uploader
• Upload multiple files at once if needed
• Click the "Process Documents" button

Step 3: Wait for Processing

The application will display real-time status updates as it:

• Ingests files
• Chunks the text
• Generates and stores embeddings

Step 4: Start Asking Questions

Once processing is complete, type your questions into the chat interface at the bottom of the screen. The chatbot will provide context-aware answers based on your documents.

---

How It Works

The RAG Pipeline Architecture

The application implements a complete Retrieval-Augmented Generation pipeline that enhances LLM capabilities by grounding responses in your specific data.

1. Ingestion & Cleaning (data_ingest.py)

Raw text is extracted from uploaded files and cleaned to remove inconsistencies, non-standard characters, and formatting issues. This ensures high-quality data for downstream processing.

2. Chunking (chunk_data.py)

Large documents are split into smaller, manageable chunks with intentional overlaps. This overlap preservation helps maintain semantic context that might otherwise be lost at chunk boundaries.

3. Embedding & Storage (embed_store.py)

• Each text chunk is converted to a numerical vector using the sentence-transformers/all-MiniLM-L6-v2 model
• Vectors capture semantic meaning of text
• All vectors are stored in a FAISS (Facebook AI Similarity Search) index for fast, efficient searching

4. Retrieval & Generation (retrive_generate.py)

When you ask a question:

1. Your question is converted into a vector
2. The FAISS index searches for the most semantically similar text chunks from your documents
3. Retrieved chunks are formatted with your question into a detailed prompt
4. Google Gemini Pro generates an answer based solely on this context

---

Project Structure

rag_tutorial/
│
├── app.py                    # Main Streamlit interface
├── main.py                   # Optional CLI pipeline execution
├── requirements.txt          # Python dependencies
├── .gitignore               # Git ignore configuration
├── README.md                # Documentation
│
├── vectorstore/             # FAISS index storage directory
│
├── src/                     # Core RAG pipeline modules
│   ├── data_ingest.py       # File extraction and text cleaning
│   ├── chunk_data.py        # Document chunking logic
│   ├── embed_store.py       # Embedding and FAISS indexing
│   └── retrive_generate.py  # Retrieval and answer generation
│
└── data/                    # Sample documents (optional)

---

Key Features

🎨 Interactive UI — Clean, intuitive interface built with Streamlit

📄 Multi-Format Support — Upload .pdf, .txt, .docx, and .csv files

🔐 Secure API Handling — Session-based key management, no persistent storage

⚡ Fast Retrieval — FAISS-powered vector search with instant results

💬 Real-Time Q&A — Get answers synthesized directly from your documents

🧩 Modular Design — Organized codebase that's easy to understand and extend

🔄 Persistent Storage — Embeddings saved locally for quick reloading (source documents discarded for privacy)

---

Technology Stack

Component	Technology
Frontend	Streamlit
LLM	Google Gemini Pro
Framework	LangChain
Embeddings	sentence-transformers/all-MiniLM-L6-v2 (Hugging Face)
Vector Store	FAISS
File Processing	pypdf, python-docx, pandas

---

Roadmap

The following features are planned for future releases:

• Support for additional document types (.pptx, .html)
• UI option to select different embedding models and LLMs
• Display source chunks alongside answers for verification
• Smart caching to prevent re-embedding of unchanged files