Skip to content

Update documentation for multi-provider AI model support #855

New issue
New issue
Closed
#858
Closed
Update documentation for multi-provider AI model support#855
#858
Labels
documentationImprovements or additions to documentation
@moncapitaine

Description

@moncapitaine
moncapitaine
opened on Nov 16, 2025

Update Documentation for Multi-Provider Support

Parent Issue: #846
Related PR: #850
Priority: 🟡 IMPORTANT - Users need to know about new features

Overview

PR #850 implemented optional multi-provider AI model support (Ollama, OpenAI, future: Anthropic, Azure). Documentation needs to be updated to reflect these changes.

This issue tracks both user-facing documentation (Astro website) and technical documentation (repository /docs folder).

Documentation Strategy

✅ User Documentation → Single Source of Truth

Location: apps/georgeai-web/src/pages/docs/admin/ai-models.astro (Astro website)
Purpose: End-user guides accessible at https://george-ai.net/docs/admin/ai-models
Audience: Business users, admins using the George AI UI

✅ Technical Documentation → Links to User Docs

Location: /docs/ folder (repository root)
Purpose: Developer/self-hoster setup guides, architecture, migrations
Audience: Developers, DevOps engineers, self-hosters
Key Principle: Link to Astro docs instead of duplicating content

Tasks

Task 1: Create User Documentation (Astro) ✅ COMPLETED

File: apps/georgeai-web/src/pages/docs/admin/ai-models.astro (NEW)

Content (for end users):

  • Overview of multi-provider support
  • Supported providers (Ollama, OpenAI, future providers)
  • How to navigate to /admin/ai-models in the UI
  • How to sync models (click "Sync Models" button)
  • How to enable/disable models via UI
  • Understanding model capabilities (embedding, chat, vision, function calling)
  • Where to select models:
    • Library Settings → Embedding Model
    • Library Settings → OCR Model
    • Assistant Settings → Chat Model
    • List Field Settings → Enrichment Model
  • Monitoring usage statistics in the UI
  • Advanced: Multi-instance Ollama load balancing (collapsible section)
  • Troubleshooting from user perspective
  • Best practices

Also Update: apps/georgeai-web/src/pages/docs/index.astro

  • Replace "AI Service Clustering" card with "AI Models & Providers" card

Status: ✅ Completed

Task 2: Update Developer Setup Guide

File: /docs/developer-setup.md

What to add:

Section: AI Provider Configuration (Optional)

## AI Provider Configuration (Optional)

George AI supports multiple AI providers. All providers are optional - configure only what you need.

### Ollama (Local Models)

For self-hosted, privacy-focused AI:

```bash
# .env
OLLAMA_BASE_URL=http://ollama:11434
OLLAMA_API_KEY=  # Optional, leave empty if not using auth

OpenAI (Cloud Models)

For cloud-based AI with latest models:

  1. Get an OpenAI API key: https://platform.openai.com/api-keys
  2. Add to .env:
OPENAI_API_KEY=sk-your-api-key-here
# OPENAI_BASE_URL=https://api.openai.com/v1  # Optional, for Azure or compatible endpoints

After Configuration

  1. Start the application: pnpm dev
  2. Navigate to Admin → AI Models in the UI
  3. Click "Sync Models" to discover available models
  4. Configure models in Library/Assistant/List settings

For detailed instructions on managing models via the UI, see the AI Models & Providers guide.

Multi-Instance Ollama (Advanced)

For self-hosted deployments with multiple GPU servers, see the AI Models & Providers guide.


**Estimated Effort**: 30 minutes

---

### Task 3: Update Architecture Documentation

**File**: `/docs/architecture.md`

**What to add**:

#### Section: Multi-Provider AI Architecture

```markdown
## Multi-Provider AI Architecture

George AI supports multiple AI providers simultaneously, with all providers being optional.

### Database-Driven Model Management

All AI models are stored in the `AiLanguageModel` database table with auto-detected capabilities:

- **Provider**: `ollama`, `openai`, `anthropic`, etc.
- **Capabilities**: `canDoEmbedding`, `canDoChatCompletion`, `canDoVision`, `canDoFunctionCalling`
- **Status**: `enabled` (admin control), `deleted` (soft delete for historical data)
- **Usage Tracking**: `AiModelUsage` table tracks tokens, requests, duration, costs

### Model Discovery & Sync

Admin users can sync models from all configured providers via the Admin UI. See the [AI Models & Providers guide](https://george-ai.net/docs/admin/ai-models) for user-facing documentation.

**Technical Details**:

1. System queries all configured providers (Ollama instances, OpenAI API)
2. Discovers available models and their capabilities
3. Auto-detects capabilities based on model name patterns and provider metadata
4. Upserts models in database (updates capabilities if model already exists)
5. Deduplicates models across multi-instance Ollama setups

### Supported Providers

| Provider | Status | Configuration | Capabilities |
|----------|--------|---------------|--------------|
| Ollama | ✅ Stable | `OLLAMA_BASE_URL`, `OLLAMA_API_KEY` | Chat, Embedding, Vision |
| OpenAI | ✅ Stable | `OPENAI_API_KEY`, `OPENAI_BASE_URL` | Chat, Embedding, Vision |
| Anthropic | 🚧 Planned | Coming soon | Chat, Vision |
| Azure OpenAI | 🚧 Planned | Coming soon | Chat, Embedding, Vision |

### Multi-Instance Ollama Support

The existing multi-instance Ollama load balancing is preserved and enhanced:

```bash
# Primary instance
OLLAMA_BASE_URL=http://ollama:11434
OLLAMA_VRAM_GB=32

# Additional instances (up to 10)
OLLAMA_BASE_URL_1=http://ollama-gpu-1:11434
OLLAMA_VRAM_GB_1=24

OLLAMA_BASE_URL_2=http://ollama-gpu-2:11434
OLLAMA_VRAM_GB_2=24

Features:

  • Intelligent load balancing based on GPU VRAM and current load
  • Model-aware routing (only sends requests to instances with required model)
  • Automatic failover on instance failure
  • Model deduplication across instances

For user-facing setup instructions, see the AI Models & Providers guide.

Graceful Degradation

If no AI providers are configured:

  • Application runs normally
  • AI features show "No models available" message
  • Users can configure providers at any time via environment variables
  • No application restart required - sync models after configuration

**Estimated Effort**: 1 hour

---

### Task 4: Create Migration Guide

**File**: `/docs/migration-guide.md` (NEW)

**Content**:

```markdown
# Migration Guide: Multi-Provider AI Support

This guide helps existing George AI deployments migrate to the new multi-provider AI architecture (introduced in v2.0.0, PR #850).

## Overview

**What changed**:
- AI models now stored in `AiLanguageModel` database table
- Library, Assistant, and List Field now reference models via foreign keys
- Model capabilities auto-detected on sync
- OpenAI support added
- Admin UI for model management at `/admin/ai-models`

**Breaking changes**:
- ⚠️ OCR models need runtime migration (automatic, see below)
- ✅ Existing Ollama models automatically migrated
- ✅ No downtime required
- ✅ Backwards compatible

---

## Migration Steps

### 1. Update Application

Pull the latest code and rebuild:

```bash
git pull
pnpm install
pnpm build

2. Run Database Migrations

Migrations run automatically on application start, or manually:

cd packages/pothos-graphql
pnpm prisma migrate deploy
pnpm prisma generate --sql

What it does:

  • Creates AiLanguageModel and AiModelUsage tables
  • Adds embeddingModelId, ocrModelId, languageModelId foreign keys
  • Migrates existing Ollama model names to database records
  • Preserves all existing data (no data loss)

3. Sync Models

After deployment, sync models from your configured providers:

  1. Navigate to /admin/ai-models in the UI
  2. Click "Sync Models"
  3. Verify all expected models appear
  4. Enable/disable models as needed

For detailed UI instructions, see the AI Models & Providers guide.

4. Optional: Add OpenAI Support

If you want to add OpenAI support:

  1. Get an API key: https://platform.openai.com/api-keys
  2. Add to .env: 
    OPENAI_API_KEY=sk-your-api-key-here
  3. Restart application (required for env var changes)
  4. Sync models via UI (see step 3)

5. OCR Model Migration (Automatic)

Existing libraries with OCR configured:

  • OCR model names stored in fileConverterOptions JSON field
  • Runtime migration happens automatically on first OCR use
  • Model record auto-created in database
  • ocrModelId column populated
  • No manual action required

To verify:

SELECT id, "ocrModelId", "fileConverterOptions"
FROM "AiLibrary"
WHERE "fileConverterOptions"::text LIKE '%ocrModel%';

Verification

Check Migration Success

-- Should have models from Ollama
SELECT * FROM "AiLanguageModel" WHERE provider = 'ollama';

-- Should have models from OpenAI (if configured)
SELECT * FROM "AiLanguageModel" WHERE provider = 'openai';

-- Check libraries migrated
SELECT id, name, "embeddingModelId", "ocrModelId" FROM "AiLibrary";

-- Check usage tracking is working
SELECT * FROM "AiModelUsage"
ORDER BY "createdAt" DESC
LIMIT 10;

Test AI Operations

See the AI Models & Providers guide for testing and troubleshooting.

Rollback Plan

If you need to rollback:

  1. Code rollback:

    git checkout <previous-version>
pnpm install
pnpm build

  2. Database rollback:

    cd packages/pothos-graphql
pnpm prisma migrate resolve --rolled-back <migration-name>

Note: Soft deletes preserve all data - no data loss during rollback.

Troubleshooting

See the AI Models & Providers guide for common issues and solutions.

Support

If you encounter issues:

  1. Check the AI Models & Providers guide
  2. Review backend logs: docker logs backend
  3. Open an issue: https://github.com/progwise/george-ai/issues
  4. Discord: https://discord.gg/GbQFKb2MNJ

**Estimated Effort**: 1.5 hours

---

### Task 5: Update README (High-Level Only)

**File**: `/README.md`

**What to add**:

#### In Features Section

```markdown
### AI Provider Flexibility

- **Multi-Provider Support**: Use Ollama, OpenAI, or both simultaneously
- **Optional Providers**: All providers are optional - run with what you need
- **Database-Driven**: Models stored in database with auto-detected capabilities
- **Admin UI**: Full model management interface at `/admin/ai-models`
- **Usage Tracking**: Monitor token usage, requests, and costs per model
- **Graceful Degradation**: App runs even if no providers configured

**[Learn more about AI Models & Providers →](https://george-ai.net/docs/admin/ai-models)**

In Quick Start Section (Replace AI Configuration)

### AI Provider Setup

**Option 1: Ollama Only (Local Models)**
```bash
OLLAMA_BASE_URL=http://ollama:11434

Option 2: OpenAI Only (Cloud Models)

OPENAI_API_KEY=sk-your-api-key-here

Option 3: Both Providers

OLLAMA_BASE_URL=http://ollama:11434
OPENAI_API_KEY=sk-your-api-key-here

After configuration:

  1. Start the application: pnpm dev
  2. Navigate to Admin → AI Models
  3. Click "Sync Models"
  4. Select models in Library/Assistant/List settings

Full setup guide →


**Estimated Effort**: 20 minutes

---

## Summary

**Total Estimated Effort**: 3-4 hours

### Checklist

- [x] Task 1: Create user documentation (Astro) - **COMPLETED**
- [ ] Task 2: Update developer setup guide with provider config + links to Astro
- [ ] Task 3: Update architecture docs with technical details + links to Astro
- [ ] Task 4: Create migration guide with links to Astro
- [ ] Task 5: Update README with high-level info + links to Astro
- [ ] Review all changes for accuracy
- [ ] Verify all links work correctly

---

## Acceptance Criteria

- [ ] User documentation available at https://george-ai.net/docs/admin/ai-models
- [ ] Technical docs link to user docs (no duplication)
- [ ] Migration guide created and tested
- [ ] Examples verified to work
- [ ] Links tested and working
- [ ] README updated with high-level overview

---

## Documentation Hierarchy

User Docs (Astro - Single Source of Truth)
├── /docs/admin/ai-models.astro - Complete UI guide
│
Technical Docs (Repository /docs/ - Reference + Links)
├── developer-setup.md - Environment setup → Links to Astro
├── architecture.md - Technical implementation → Links to Astro
├── migration-guide.md - Migration steps → Links to Astro
└── README.md - High-level features → Links to Astro


**Key Principle**: Avoid duplication. Technical docs focus on environment variables, database schema, and architecture, then link to Astro docs for user-facing instructions.

---

## Related Issues

- #846 - Parent issue (Optional AI providers)
- #850 - PR with implementation
- #853 - Complete usage tracking integration
- #854 - OCR runtime migration

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions