Skip to content

remove local-ai from docs #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
wtfsayo merged 1 commit into from
Jul 16, 2025
Merged

remove local-ai from docs #29

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions docs.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,8 +259,7 @@
"plugins/llm/anthropic",
"plugins/llm/google-genai",
"plugins/llm/ollama",
"plugins/llm/openrouter",
"plugins/llm/local-ai"
"plugins/llm/openrouter"
]
},
{
Expand Down
157 changes: 107 additions & 50 deletions plugins/llm.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,33 @@ ElizaOS uses a plugin-based architecture for integrating different Language Mode

### Model Types

ElizaOS supports three types of model operations:
ElizaOS supports many types of AI operations. Here are the most common ones:

1. **TEXT_GENERATION** - Generating conversational responses
2. **EMBEDDING** - Creating vector embeddings for memory and similarity search
3. **OBJECT_GENERATION** - Structured output generation (JSON/XML)
1. **TEXT_GENERATION** (`TEXT_SMALL`, `TEXT_LARGE`) - Having conversations and generating responses
2. **TEXT_EMBEDDING** - Converting text into numbers for memory and search
3. **OBJECT_GENERATION** (`OBJECT_SMALL`, `OBJECT_LARGE`) - Creating structured data like JSON

Think of it like different tools in a toolbox:
- **Text Generation** = Having a conversation
- **Embeddings** = Creating a "fingerprint" of text for finding similar things later
- **Object Generation** = Filling out forms with specific information

### Plugin Capabilities

Not all LLM plugins support all model types:
Not all LLM plugins support all model types. Here's what each can do:

| Plugin | Text Chat | Embeddings | Structured Output | Runs Offline |
|--------|-----------|------------|-------------------|--------------|
| OpenAI | ✅ | ✅ | ✅ | ❌ |
| Anthropic | ✅ | ❌ | ✅ | ❌ |
| Google GenAI | ✅ | ✅ | ✅ | ❌ |
| Ollama | ✅ | ✅ | ✅ | ✅ |
| OpenRouter | ✅ | ❌ | ✅ | ❌ |

| Plugin | Text Generation | Embeddings | Object Generation |
|--------|----------------|------------|-------------------|
| OpenAI | ✅ | ✅ | ✅ |
| Anthropic | ✅ | ❌ | ✅ |
| Google GenAI | ✅ | ✅ | ✅ |
| Ollama | ✅ | ✅ | ✅ |
| OpenRouter | ✅ | ❌ | ✅ |
| Local AI | ✅ | ✅ | ✅ |
**Key Points:**
- 🌟 **OpenAI & Google GenAI** = Do everything (jack of all trades)
- 💬 **Anthropic & OpenRouter** = Amazing at chat, need help with embeddings
- 🏠 **Ollama** = Your local hero - does almost everything, no internet needed!

## Plugin Loading Order

Expand All @@ -39,57 +48,109 @@ plugins: [
'@elizaos/plugin-sql',

// Text-only plugins (no embedding support)
...(process.env.ANTHROPIC_API_KEY ? ['@elizaos/plugin-anthropic'] : []),
...(process.env.OPENROUTER_API_KEY ? ['@elizaos/plugin-openrouter'] : []),

// Embedding-capable plugins last (lowest priority for embedding fallback)
...(process.env.OPENAI_API_KEY ? ['@elizaos/plugin-openai'] : []),
...(process.env.OLLAMA_API_ENDPOINT ? ['@elizaos/plugin-ollama'] : []),
...(process.env.GOOGLE_GENERATIVE_AI_API_KEY ? ['@elizaos/plugin-google-genai'] : []),

// Fallback when no other LLM is configured
...(!process.env.GOOGLE_GENERATIVE_AI_API_KEY &&
!process.env.OLLAMA_API_ENDPOINT &&
!process.env.OPENAI_API_KEY
? ['@elizaos/plugin-local-ai']
: []),
...(process.env.ANTHROPIC_API_KEY?.trim() ? ['@elizaos/plugin-anthropic'] : []),
...(process.env.OPENROUTER_API_KEY?.trim() ? ['@elizaos/plugin-openrouter'] : []),

// Embedding-capable plugins (optional, based on available credentials)
...(process.env.OPENAI_API_KEY?.trim() ? ['@elizaos/plugin-openai'] : []),
...(process.env.GOOGLE_GENERATIVE_AI_API_KEY?.trim() ? ['@elizaos/plugin-google-genai'] : []),

// Ollama as fallback (only if no main LLM providers are configured)
...(process.env.OLLAMA_API_ENDPOINT?.trim() ? ['@elizaos/plugin-ollama'] : []),
]
```

### Understanding the Order

1. **Text-only plugins first** - Anthropic and OpenRouter are loaded first for text generation
2. **Embedding-capable plugins last** - These serve as fallbacks for embedding operations
3. **Local AI as ultimate fallback** - Only loads when no cloud providers are configured
Think of it like choosing team players - you pick specialists first, then all-rounders:

1. **Anthropic & OpenRouter go first** - They're specialists! They're great at text generation but can't do embeddings. By loading them first, they get priority for text tasks.

2. **OpenAI & Google GenAI come next** - These are the all-rounders! They can do everything: text generation, embeddings, and structured output. They act as fallbacks for what the specialists can't do.

3. **Ollama comes last** - This is your local backup player! It supports almost everything (text, embeddings, objects) and runs on your computer. Perfect when cloud services aren't available.

### Why This Order Matters

When you ask ElizaOS to do something, it looks for the best model in order:

- **Generate text?** → Anthropic gets first shot (if loaded)
- **Create embeddings?** → Anthropic can't, so OpenAI steps in
- **No cloud API keys?** → Ollama handles everything locally

This smart ordering means:
- You get the best specialized models for each task
- You always have fallbacks for missing capabilities
- You can run fully offline with Ollama if needed

### Real Example: How It Works

Let's say you have Anthropic + OpenAI configured:

```
Task: "Generate a response"
1. Anthropic: "I got this!" ✅ (Priority 100 for text)
2. OpenAI: "I'm here if needed" (Priority 50)
Task: "Create embeddings for memory"
1. Anthropic: "Sorry, can't do that" ❌
2. OpenAI: "No problem, I'll handle it!" ✅
Task: "Generate structured JSON"
1. Anthropic: "I can do this!" ✅ (Priority 100 for objects)
2. OpenAI: "Standing by" (Priority 50)
```

## Model Registration

Each LLM plugin registers its models with the runtime during initialization:
When plugins load, they "register" what they can do. It's like signing up for different jobs:

```typescript
// Example from a plugin's init method
runtime.registerModel({
type: ModelType.TEXT_GENERATION,
handler: generateText,
provider: 'openai',
priority: 1
});
// Each plugin says "I can do this!"
runtime.registerModel(
ModelType.TEXT_LARGE, // What type of work
generateText, // How to do it
'anthropic', // Who's doing it
100 // Priority (higher = goes first)
);
```

### Priority System
### How ElizaOS Picks the Right Model

When you ask ElizaOS to do something, it:

Models are selected based on:
1. **Explicit provider** - If specified, uses that provider's model
2. **Priority** - Higher priority models are preferred
3. **Registration order** - First registered wins for same priority
1. **Checks what type of work it is** (text? embeddings? objects?)
2. **Looks at who signed up** for that job
3. **Picks based on priority** (higher number goes first)
4. **If tied, first registered wins**

**Example**: You ask for text generation
- Anthropic registered with priority 100 ✅ (wins!)
- OpenAI registered with priority 50
- Ollama registered with priority 10

But for embeddings:
- Anthropic didn't register ❌ (can't do it)
- OpenAI registered with priority 50 ✅ (wins!)
- Ollama registered with priority 10

## Embedding Fallback Strategy

Since not all plugins support embeddings, ElizaOS uses a fallback strategy:
Remember: Not all plugins can create embeddings! Here's how ElizaOS handles this:

**The Problem**:
- You're using Anthropic (great at chat, can't do embeddings)
- But ElizaOS needs embeddings for memory and search

**The Solution**:
ElizaOS automatically finds another plugin that CAN do embeddings!

```typescript
// If primary plugin (e.g., Anthropic) doesn't support embeddings,
// the runtime will automatically use the next available embedding provider
// What happens behind the scenes:
// 1. "I need embeddings!"
// 2. "Can Anthropic do it?" → No ❌
// 3. "Can OpenAI do it?" → Yes ✅
// 4. "OpenAI, you're up!"
```

### Common Patterns
Expand Down Expand Up @@ -149,9 +210,6 @@ OPENROUTER_API_KEY=sk-or-...
OPENROUTER_SMALL_MODEL=google/gemini-2.0-flash-001 # Optional: any available model
OPENROUTER_LARGE_MODEL=anthropic/claude-3-opus # Optional: any available model

# Local AI (no API key needed)
```

**Important**: The model names shown are examples. You can use any model available from each provider.

### Character-Specific Secrets
Expand Down Expand Up @@ -182,7 +240,6 @@ You can also configure API keys per character:
### Local/Self-Hosted

- [Ollama Plugin](./ollama) - Run models locally with Ollama
- [Local AI Plugin](./local-ai) - Fully offline operation

## Best Practices

Expand Down
Loading