PaperBrain

Chat with your documents. Upload PDFs, paste web URLs or YouTube links, and ask questions — PaperBrain retrieves the most relevant passages and generates grounded answers with source citations.

---

Features

• User accounts — Register and log in; all your chats and sources are private to your account
• Multiple isolated chats — Create as many chats as you like, each with its own ingested sources and conversation history
• Multi-source ingestion — PDFs (up to 50 MB), web pages, and YouTube transcripts
• Conversational memory — Multi-turn chat with context retention (last 3 turns)
• Source citations — Every answer cites the document, page, or URL it came from
• Answer format presets — Choose how the model responds: Auto, Brief, Explain, Pointers, or Detailed
• Multiple LLMs via OpenRouter — Switch per-chat between Gemini 2.5 Flash Lite, Llama 3.3 70B, DeepSeek V3, Claude Sonnet, GPT-4o, and DeepSeek R1
• Persistent storage — SQLite for users/chats/messages, ChromaDB on disk for vectors; everything survives restarts
• Decoupled architecture — FastAPI backend + static HTML/CSS/JS frontend (no build step)

---

Tech Stack

Layer	Technology
Web server	FastAPI & Uvicorn
Orchestration	LangChain >= 0.3
LLM gateway	OpenRouter (langchain-openai)
Embeddings	Google gemini-embedding-001
Vector store	ChromaDB >= 0.5
Relational DB	SQLite (stdlib sqlite3)
PDF loader	PyMuPDF
Web loader	LangChain WebBaseLoader (BeautifulSoup4 / lxml)
YouTube loader	youtube-transcript-api / YoutubeLoader
Frontend	Vanilla HTML + CSS + JS (no framework, no build step)

---

Project Structure

PaperBrain/
├── .env                    # API keys (not committed)
├── .env.example            # Template with placeholder keys
├── paperbrain.db           # SQLite DB — users, chats, messages, documents (auto-created)
├── chroma_db/              # ChromaDB vector store (auto-created on first ingest)
├── backend/
│   ├── main.py             # FastAPI app & all route handlers (v3.0)
│   ├── auth.py             # Password hashing (PBKDF2) + hand-rolled HS256 JWT
│   ├── db.py               # SQLite helpers (plain sqlite3, no ORM)
│   ├── config.py           # Models, formats, RAG settings
│   ├── ingest.py           # Document ingestion pipeline
│   ├── rag_chain.py        # RAG retrieval & streaming generation
│   ├── requirements.txt    # Backend dependencies (loose/readable)
│   ├── requirements.lock.txt # Pinned dependencies (used by Dockerfile)
│   ├── Dockerfile          # Production container build
│   └── .dockerignore       # Exclude runtime data from Docker image
└── frontend/
    ├── login.html          # Register / log in
    ├── home.html           # Chat list & create new chat
    ├── index.html          # Add sources to a chat
    ├── chat.html           # Conversation view
    ├── css/style.css       # Dark/light theme stylesheet
    └── js/
        ├── auth.js         # Register/login form logic
        ├── home.js         # Chat list page logic
        ├── upload.js       # Source ingestion UI
        ├── chat.js         # Chat/streaming UI
        ├── session.js      # Token storage, authFetch, requireAuth guard
        ├── nav.js          # Shared top-nav renderer
        ├── theme.js        # Dark/light theme toggle (localStorage)
        └── config.js       # API base URL, model & format lists

Note: A fresh clone starts with an empty chroma_db/ directory — this is expected. It will be created automatically on the first document ingest.

---

Setup

1. Clone & create a virtual environment

git clone https://github.com/Zentise/PaperBrain.git
cd PaperBrain
python -m venv .venv

# Windows
.venv\Scripts\activate

# macOS / Linux
source .venv/bin/activate

2. Install dependencies

cd backend
pip install -r requirements.txt

3. Configure API keys

Copy .env.example to .env in the project root (next to backend/) and fill in your keys:

cp .env.example .env

GEMINI=your_gemini_api_key_for_embeddings
OPENROUTER_API_KEY=your_openrouter_api_key_for_llms

• Gemini key — Google AI Studio
• OpenRouter key — OpenRouter Console

---

Running the App

You need two terminals running simultaneously.

1. Start the backend

cd backend
uvicorn main:app --host 0.0.0.0 --port ${PORT:-8000}

The API will be available at http://localhost:8000.

2. Start the frontend

cd frontend
python -m http.server 5500

Open http://localhost:5500 in your browser.

Important: The frontend must be served over http://, not opened directly as a file:// URL — pages use <script type="module">, which browsers block on file:// due to CORS.

Docker (production)

cd backend
docker build -t paperbrain-api .
docker run -p 8080:8080 --env-file ../.env paperbrain-api

---

Usage

1. Register / log in — Create an account on the login page.
2. Create a chat — From the home screen, start a new chat and give it a name.
3. Add sources — In the sources page, upload PDFs or paste web / YouTube URLs and click Ingest.
4. Chat — Switch to the conversation view, type a question, and press Enter.
5. Customize — Use the model selector to switch LLMs per-chat and the format selector to control answer style.

---

API Reference

All document and chat endpoints are scoped to a specific chat and require a JWT Authorization: Bearer <token> header.

Auth

Method	Endpoint	Description
POST	/api/auth/register	Create a new account
POST	/api/auth/login	Log in, returns a JWT token
GET	/api/auth/me	Returns the current user
POST	/api/auth/sse-ticket	Issue a 60-second JWT for SSE connections

Chats

Method	Endpoint	Description
GET	/api/chats	List all chats for the current user
POST	/api/chats	Create a new chat
GET	/api/chats/{chat_id}	Get chat details, messages, and documents
PATCH	/api/chats/{chat_id}	Update chat title, model, or format
DELETE	/api/chats/{chat_id}	Delete a chat and all its data

Ingestion & Documents

Method	Endpoint	Description
POST	/api/chats/{chat_id}/ingest/upload	Upload and ingest a PDF
POST	/api/chats/{chat_id}/ingest/url	Ingest a web page or YouTube URL
GET	/api/chats/{chat_id}/documents	List ingested sources for a chat
DELETE	/api/chats/{chat_id}/documents/{source}	Remove a source from a chat

Chat

Method	Endpoint	Description
POST	/api/chats/{chat_id}/chat	Non-streaming chat
GET	/api/chats/{chat_id}/chat/stream	SSE streaming chat (?token= auth)

Meta

Method	Endpoint	Description
GET	/	Root health probe
GET	/api/health	Backend health & version
GET	/api/models	Available LLM list
GET	/api/formats	Available answer format presets

---

How It Works

1. Ingest — Documents are loaded (PyMuPDF / WebBaseLoader / YoutubeLoader), split into 800-character chunks (80-character overlap), embedded with gemini-embedding-001, and stored in a per-chat ChromaDB collection (chat_<id>).
2. Retrieve — At query time, the top-3 chunks with a similarity score >= 0.50 are fetched from that chat's collection.
3. Generate — Retrieved context + the last 3 turns of history + the chosen format instruction are sent to the selected LLM via OpenRouter.
4. Stream — The response is streamed back to the browser via Server-Sent Events and rendered with source-citation chips.

---

Notes

• YouTube ingestion requires the video to have captions/transcripts enabled.
• ChromaDB data persists in chroma_db/ at the repo root. Delete that folder (and paperbrain.db) to wipe all data and start fresh.
• Temperature is fixed at 0.2 for consistent, factual responses.
• The PDF upload limit is 50 MB per file.
• OpenRouter model IDs can change over time. If a chat request fails with an invalid model ID error, check openrouter.ai/models for the current slug and update backend/config.py and frontend/js/config.js.