Chat History

Chat History

Persistent conversation management with automatic history injection

---

Overview

Store and manage chat conversations with:

• Session management - Create, list, delete sessions
• Message persistence - Store all messages
• Automatic injection - Auto-include history in requests
• Context limits - Configurable message windows
• User isolation - Sessions per user

---

Quick Start

Create Session

curl -X POST http://localhost:8000/chat/sessions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "session_name": "My Conversation",
    "model": "gpt-4"
  }'

Response:

{
  "session_id": "sess_abc123",
  "session_name": "My Conversation",
  "model": "gpt-4",
  "created_at": "2024-12-15T10:30:00Z"
}

Send Message with History

curl -X POST http://localhost:8000/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4",
    "messages": [{"role": "user", "content": "Hello!"}],
    "session_id": "sess_abc123"
  }'

History is automatically injected before your message.

List Sessions

curl http://localhost:8000/chat/sessions \
  -H "Authorization: Bearer YOUR_API_KEY"

Delete Session

curl -X DELETE http://localhost:8000/chat/sessions/sess_abc123 \
  -H "Authorization: Bearer YOUR_API_KEY"

---

API Endpoints

1. Create Session

POST /chat/sessions

Request:

{
  "session_name": "Project Discussion",
  "model": "gpt-4",
  "max_history_messages": 20
}

2. List Sessions

GET /chat/sessions

Returns all sessions for authenticated user.

3. Get Session

GET /chat/sessions/{session_id}

Includes message count and last activity.

4. Update Session

PUT /chat/sessions/{session_id}

Update name or settings.

5. Delete Session

DELETE /chat/sessions/{session_id}

Soft delete - marks as inactive.

6. Get Messages

GET /chat/sessions/{session_id}/messages

Query params:

• limit - Max messages (default: 50)
• offset - Pagination offset

---

Automatic History Injection

When you include session_id in chat request:

{
  "model": "gpt-4",
  "messages": [
    {"role": "user", "content": "What's the weather?"}
  ],
  "session_id": "sess_abc123"
}

Backend automatically:

1. Fetches last N messages from session
2. Prepends to your messages array
3. Sends combined context to model
4. Stores response in session

Result:

{
  "messages": [
    {"role": "user", "content": "Hello"},          // ← From history
    {"role": "assistant", "content": "Hi there!"}, // ← From history
    {"role": "user", "content": "What's the weather?"} // ← Your message
  ]
}

---

Configuration

Message Limits

# Default: Keep last 20 messages
MAX_HISTORY_MESSAGES = 20

# Per session override
{
  "max_history_messages": 50
}

Context Window

Automatically truncates to fit model context:

• GPT-4: 8K tokens → ~15-20 messages
• GPT-3.5: 4K tokens → ~10-15 messages
• Claude: 100K tokens → ~200+ messages

---

Database Schema

chat_sessions Table

CREATE TABLE chat_sessions (
  id TEXT PRIMARY KEY,
  user_id INTEGER NOT NULL,
  session_name TEXT,
  model TEXT,
  max_history_messages INTEGER DEFAULT 20,
  message_count INTEGER DEFAULT 0,
  is_active BOOLEAN DEFAULT true,
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW(),
  last_message_at TIMESTAMP
);

chat_messages Table

CREATE TABLE chat_messages (
  id INTEGER PRIMARY KEY,
  session_id TEXT NOT NULL,
  role TEXT NOT NULL,              -- 'user', 'assistant', 'system'
  content TEXT NOT NULL,
  tokens INTEGER,
  created_at TIMESTAMP DEFAULT NOW(),

  FOREIGN KEY (session_id) REFERENCES chat_sessions(id)
);

CREATE INDEX idx_messages_session ON chat_messages(session_id, created_at);

---

Use Cases

1. Chatbot Conversations

// Initialize session
const session = await createSession("Customer Support");

// Send messages
await chat({
  session_id: session.id,
  messages: [
    {role: "user", content: "I need help with my order"}
  ]
});

// Continue conversation (history auto-included)
await chat({
  session_id: session.id,
  messages: [
    {role: "user", content: "What's the status?"}
  ]
});

2. Multi-turn Conversations

# Start conversation
session_id = create_session("Research Chat")

# Turn 1
ask(session_id, "Explain quantum computing")

# Turn 2 (remembers previous context)
ask(session_id, "How does it compare to classical?")

# Turn 3 (full context maintained)
ask(session_id, "Give me a practical example")

3. Context Management

# Long conversation - auto-prune old messages
session = create_session("Long Chat", max_history=10)

# Sends 50 messages, only keeps last 10 in context
for i in range(50):
    ask(session.id, f"Question {i}")

---

Best Practices

For Users

1. Name sessions clearly: "Q4 Planning", "Bug #123"
2. Set appropriate limits: Longer history = more tokens
3. Clean up old sessions: Delete when done
4. Monitor message count: Large sessions cost more
5. Use system messages: Set context once per session

For Developers

1. Validate session ownership: Check user_id
2. Implement pagination: Don't load all messages
3. Cache session data: Reduce DB queries
4. Set TTL: Auto-delete old sessions
5. Monitor storage: Large sessions use space

---

Troubleshooting

Session not found

Cause: Invalid session_id or deleted session

Solution: Create new session

Messages not persisting

Cause: session_id not included in request

Solution: Add "session_id": "sess_xxx" to request

Context too long error

Cause: Too many messages for model context

Solution: Reduce max_history_messages or use larger context model

---

Implementation

Location:

• DB: src/db/chat_history.py
• Routes: src/routes/chat_history.py
• Integration: src/routes/chat.py

Auto-injection in src/routes/chat.py:115:

if session_id:
    history = get_session_messages(session_id, limit=20)
    request.messages = history + request.messages

---

Related Documentation

• Chat API - Chat completions
• API Keys - Authentication

---

Last Updated: December 2024 Status: Production Ready

---

Related

• Feature-Flags — Feature flags control chat features
• Activity-Logging — Chat usage is logged