AIChat

A modern AI chat application with RAG capabilities, multi-provider LLM support, memory persistence, and real-time streaming.

Screenshots

Chat Interface

Admin Panel

Features

• Multi-Provider LLM Support - OpenAI, Anthropic Claude, and HuggingFace models
• Real-time Streaming - SSE-based token streaming for responsive chat
• Long-term Memory - Remembers user preferences across sessions (name, age, preferences)
• RAG Integration - Upload documents (PDF, TXT, MD, JSON) for context-aware Q&A
• Web Search - Tavily-powered web search for current information
• Thinking Mode - Extended chain-of-thought reasoning with visual indicator
• Session Management - Persistent chat sessions with history
• Cost Tracking - Per-session token and cost monitoring
• Admin Panel - Upload and manage company knowledge base documents
• Guardrails - Markdown formatting, safety filters, and tool output sanitization
• Grafana Dashboard - Pre-configured monitoring visualizations

Tech Stack

Component	Technology
Backend	FastAPI (Python 3.12)
Frontend	Next.js 15 (App Router)
LLM	OpenAI, Anthropic, HuggingFace Router
Vector DB	ChromaDB
Cache	Redis
Monitoring	Prometheus + Grafana
Logging	structlog (JSON)

Quick Start

Prerequisites

• Docker & Docker Compose
• API Keys (at least one): OpenAI, Anthropic, or HuggingFace

Launch with Docker

# Clone the repository
git clone git@github.com:Default-bit/aichat.git
cd aichat

# Copy the example environment file and add your API keys
cp backend/.env.example backend/.env
# Edit backend/.env with your actual API keys (at least one LLM provider required)

# Start all services
docker-compose up --build

Access:

• Chat UI: http://localhost:3001
• API Docs: http://localhost:8000/docs
• Grafana: http://localhost:3002 (admin/admin)
• Prometheus: http://localhost:9090

Note: On first run, the Admin Panel will have no documents. Upload company documents via the Admin Panel to enable RAG queries about your organization.

Local Development

Backend:

cd backend
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Set environment variables
export OPENAI_API_KEY=your_key_here
export ANTHROPIC_API_KEY=your_key_here  # Optional
export HF_API_KEY=your_key_here          # Optional
export TAVILY_API_KEY=your_key_here      # Optional
export REDIS_HOST=localhost

# Run backend
uvicorn app.main:app --reload --port 8000

Frontend:

cd frontend
npm install
npm run dev  # Runs on port 3001

Redis (required):

docker run -d -p 6379:6379 redis:7-alpine

API Endpoints

Chat

Method	Endpoint	Description
POST	/chat/stream	Stream chat response (SSE)
POST	/chat/completion	Non-streaming chat response
GET	/chat/history/{session_id}	Get session history
DELETE	/chat/history/{session_id}	Clear session history
GET	/chat/sessions	List all sessions
DELETE	/chat/sessions/{session_id}	Delete a session
GET	/chat/memory/{user_id}	Get user's stored memories

Documents (User)

Method	Endpoint	Description
POST	/documents/upload	Upload document for session RAG

Admin (Company Knowledge Base)

Method	Endpoint	Description
POST	/admin/upload	Upload document to knowledge base
GET	/admin/documents	List uploaded documents
POST	/admin/documents/delete	Delete document from knowledge base

Monitoring

Method	Endpoint	Description
GET	/monitor/metrics/{session_id}	Session metrics JSON
GET	/monitor/dashboard/{session_id}	HTML dashboard
GET	/monitor/logs/{session_id}	Session logs
GET	/metrics	Prometheus metrics
GET	/health	Health check

Example Requests

Stream Chat

curl -X POST http://localhost:8000/chat/stream \
  -H "Content-Type: application/json" \
  -d '{
    "message": "What is machine learning?",
    "session_id": "my-session",
    "user_id": "user-123",
    "model": "gpt-4o-mini",
    "temperature": 0.7,
    "thinking_mode": false
  }'

Upload Admin Document

curl -X POST http://localhost:8000/admin/upload \
  -F "file=@company-docs.pdf"

Get Sessions

curl http://localhost:8000/chat/sessions

Environment Variables

Variable	Description	Default
OPENAI_API_KEY	OpenAI API key	Optional
ANTHROPIC_API_KEY	Anthropic API key	Optional
HF_API_KEY	HuggingFace API key	Optional
TAVILY_API_KEY	Tavily API key (web search)	Optional
REDIS_HOST	Redis hostname	localhost
REDIS_PORT	Redis port	6379
NEXT_PUBLIC_BACKEND_URL	Backend URL for frontend	http://localhost:8000

Supported Models

OpenAI

• gpt-4o - Latest GPT-4o
• gpt-4o-mini - Fast & affordable
• gpt-4-turbo - GPT-4 Turbo
• gpt-3.5-turbo - Fast & cheap

Anthropic

• claude-3-5-sonnet-20241022 - Claude 3.5 Sonnet
• claude-3-opus-20240229 - Claude 3 Opus
• claude-3-sonnet-20240229 - Claude 3 Sonnet
• claude-3-haiku-20240307 - Claude 3 Haiku

HuggingFace (via Router)

• meta-llama/Meta-Llama-3-8B-Instruct - Llama 3 8B

Project Structure

aichat/
├── backend/
│   ├── app/
│   │   ├── api/          # FastAPI routes
│   │   ├── core/         # Business logic
│   │   │   ├── llm_provider/  # LangChain LLM integration
│   │   │   ├── agent.py       # Chat agent
│   │   │   ├── memory.py      # Memory system
│   │   │   └── rag_store.py   # RAG retrieval
│   │   ├── db/           # Database clients
│   │   ├── guardrails/   # Output sanitization
│   │   └── tools/        # Agent tools
│   └── tests/            # Unit & integration tests
├── frontend/
│   ├── app/              # Next.js App Router pages
│   ├── components/       # React components
│   └── lib/              # Utilities
├── grafana/              # Grafana provisioning
├── docker-compose.yml
└── prometheus.yml

Testing

cd backend
python -m pytest tests/ -v

Current test coverage:

• ✅ Guardrails (markdown, safety, tool sanitization)
• ✅ Intent classifier
• ✅ Memory triggers
• ✅ Pricing calculations
• ✅ API endpoints

Monitoring

The Grafana dashboard (/grafana/dashboards/aichat.json) includes:

• LLM Latency P95 - Response time percentiles
• Total Tokens - Input/output token counts
• Estimated Cost - Running cost tracker
• Tool Usage - Tool invocation counts
• Request Rate - Requests per second
• Latency Over Time - Latency trends

Architecture

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   Frontend  │────▶│   Backend   │────▶│ LLM Provider│
│  (Next.js)  │◀────│  (FastAPI)  │◀────│ (Multi-LLM) │
└─────────────┘     └─────────────┘     └─────────────┘
       :3001              :8000          OpenAI/Claude/HF
                           │
              ┌────────────┼────────────┐
              ▼            ▼            ▼
        ┌─────────┐  ┌─────────┐  ┌─────────┐
        │  Redis  │  │ ChromaDB│  │Prometheus│
        │(Sessions│  │ (RAG &  │  │(Metrics) │
        │& History│  │ Memory) │  │   :9090  │
        └─────────┘  └─────────┘  └─────────┘
              :6379                     │
                                        ▼
                                  ┌─────────┐
                                  │ Grafana │
                                  │  :3002  │
                                  └─────────┘

License

MIT