LangGraph PDF RAG Chatbot

A multi-tool chatbot built with LangGraph and Streamlit that can answer questions about an uploaded PDF (RAG), search the web, and do arithmetic — all backed entirely by free-tier services, so it costs nothing to run or deploy.

Features

• 📄 PDF-based RAG — upload a PDF per conversation; it's chunked, embedded, and indexed with FAISS for retrieval-augmented answers.
• 🔍 Web search — falls back to DuckDuckGo search for questions outside the uploaded document.
• 🧮 Calculator tool — handles arithmetic (add/sub/mul/div) directly instead of hallucinating math.
• 💬 Multi-conversation threads — each chat is a separate LangGraph thread, persisted in SQLite, with full history recall.
• 🗑️ Delete conversations — remove any past conversation (and its indexed document) permanently, with a confirm step.
• ⚡ Streaming responses — assistant replies stream token-by-token in the UI, with live tool-usage indicators.
• 🆓 Zero-cost stack — LLM via OpenRouter's free tier, embeddings run locally on CPU, no paid API required.

Architecture

frontend.py   → Streamlit UI: chat interface, sidebar (threads, PDF upload, delete)
backend.py    → LangGraph agent: state graph, tools, LLM, embeddings, persistence
chatbot.db    → SQLite database (LangGraph checkpointer) — stores all conversation history

Agent graph: a single chat_node decides whether to answer directly or call a tool (rag_tool, search_tool, or calculator) via LangGraph's tools_condition routing, loops back after each tool call, and streams the final response.

Tech stack:

Component	Choice
Orchestration	LangGraph
LLM	OpenRouter (openai/gpt-oss-120b:free by default)
Embeddings	sentence-transformers/all-MiniLM-L6-v2, run locally via langchain-huggingface
Vector store	FAISS (in-memory, per conversation)
Persistence	SQLite via langgraph-checkpoint-sqlite
Web search	DuckDuckGo (ddgs)
UI	Streamlit

Setup

1. Clone and create a virtual environment

python3.12 -m venv venv
source venv/bin/activate        # Windows: venv\Scripts\activate

Note: Python 3.14 has known compatibility issues with some LangGraph/Pydantic dependencies. Python 3.12 is recommended.

2. Install dependencies

pip install -r requirements.txt

3. Configure environment variables

Create a .env file in the project root:

OPENROUTER_API_KEY=sk-or-v1-your-key-here
LLM_MODEL=openai/gpt-oss-120b:free
EMBED_MODEL=sentence-transformers/all-MiniLM-L6-v2
APP_URL=https://localhost:8501
APP_NAME=LangGraph PDF Chatbot

Get a free OpenRouter API key at openrouter.ai/keys — no card required.

⚠️ Every key in .env must have a value. An empty KEY= line is treated as an empty string by os.getenv(), not as "unset" — this silently overrides the code's default and is a common source of confusing errors (e.g. "No models provided" or embedding failures).

4. Run it

python -m streamlit run frontend.py

Using python -m streamlit instead of the bare streamlit command avoids picking up a different Python environment (e.g. a conda env) that might be earlier on your PATH.

Visit http://localhost:8501.

Deploying to Streamlit Community Cloud (free)

1. Push this repo to GitHub.
2. Go to share.streamlit.io, create a new app, and point it at frontend.py.
3. In the app's Settings → Secrets, add the same key/value pairs from your .env file (do not commit .env to GitHub).
4. Deploy.

Notes for cloud deployment:

• First load will be slower — the ~80MB embedding model downloads and caches on first run.
• SQLite storage (chatbot.db) is ephemeral on Community Cloud's free tier — the container can restart and wipe it. For durable history across restarts, swap SqliteSaver for PostgresSaver pointed at a free tier like Neon.
• Free OpenRouter models are rate-limited and rotate availability — if you hit errors, check openrouter.ai/models for the current free model list.

Project structure

.
├── frontend.py       # Streamlit UI
├── backend.py         # LangGraph agent, tools, persistence
├── requirements.txt
├── .env               # not committed — see Setup step 3
└── chatbot.db          # created automatically on first run

Known limitations

• SqliteSaver is intended for lightweight/single-user use and isn't safe for high-concurrency multi-user traffic.
• PDF indexes (FAISS retrievers) are held in memory per thread and are lost on app restart — only the chat history persists, not the indexed document. Re-upload the PDF after a restart to resume RAG on that thread.
• Free-tier LLM/search services may rate-limit under heavy use.