From be5d88dad8d67f3101e9f4261357663bc825f508 Mon Sep 17 00:00:00 2001 From: Sri Date: Fri, 12 Sep 2025 14:26:33 -0700 Subject: [PATCH 1/4] Added custom kb docs and trieve migration docs --- fern/docs.yml | 9 +- fern/knowledge-base/custom-knowledge-base.mdx | 537 ++++++++++++++++++ fern/knowledge-base/migrating-from-trieve.mdx | 407 +++++++++++++ 3 files changed, 950 insertions(+), 3 deletions(-) create mode 100644 fern/knowledge-base/custom-knowledge-base.mdx create mode 100644 fern/knowledge-base/migrating-from-trieve.mdx diff --git a/fern/docs.yml b/fern/docs.yml index 12fb055b9..add14f79c 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -214,9 +214,12 @@ navigation: - page: Query tool path: knowledge-base/using-query-tool.mdx icon: fa-light fa-magnifying-glass - - page: Trieve integration - path: knowledge-base/integrating-with-trieve.mdx - icon: fa-light fa-brain + - page: Custom knowledge base + path: knowledge-base/custom-knowledge-base.mdx + icon: fa-light fa-server + - page: Migrating from Trieve + path: knowledge-base/migrating-from-trieve.mdx + icon: fa-light fa-triangle-exclamation - section: Structured outputs icon: fa-light fa-database path: assistants/structured-outputs.mdx diff --git a/fern/knowledge-base/custom-knowledge-base.mdx b/fern/knowledge-base/custom-knowledge-base.mdx new file mode 100644 index 000000000..af7bbeefc --- /dev/null +++ b/fern/knowledge-base/custom-knowledge-base.mdx @@ -0,0 +1,537 @@ +--- +title: Custom Knowledge Base +subtitle: Create and implement your own knowledge base server for full control over document retrieval +slug: knowledge-base/custom-knowledge-base +--- + +## Overview + +Custom Knowledge Bases allow you to implement your own document retrieval server, giving you complete control over how your assistant searches and retrieves information. Instead of relying on Vapi's built-in knowledge base providers, you can integrate your own search infrastructure, vector databases, or custom retrieval logic. + +**With Custom Knowledge Bases, you can:** +- Use your own vector database or search infrastructure +- Implement custom retrieval algorithms and scoring +- Integrate with existing document management systems +- Apply custom business logic to document filtering +- Maintain full control over data security and privacy + +## How Custom Knowledge Bases Work + +Custom Knowledge Bases operate through a webhook-style integration where Vapi forwards search requests to your server and expects structured responses containing relevant documents. + + + + User asks assistant a question during conversation + + + Vapi sends search request to your custom endpoint + + + Your server returns relevant documents or direct response + + + +## Creating a Custom Knowledge Base + +### Step 1: Create the Knowledge Base + +Use the Vapi API to create a custom knowledge base configuration: + + +```bash title="cURL" +curl --location 'https://api.vapi.ai/knowledge-base' \ +--header 'Content-Type: application/json' \ +--header 'Authorization: Bearer YOUR_VAPI_API_KEY' \ +--data '{ + "provider": "custom-knowledge-base", + "server": { + "url": "https://your-domain.com/kb/search", + "secret": "your-webhook-secret" + } +}' +``` + +```typescript title="TypeScript SDK" +import { VapiClient } from "@vapi-ai/server-sdk"; + +const vapi = new VapiClient({ + token: process.env.VAPI_API_KEY +}); + +try { + const knowledgeBase = await vapi.knowledgeBases.create({ + provider: "custom-knowledge-base", + server: { + url: "https://your-domain.com/kb/search", + secret: "your-webhook-secret" + } + }); + + console.log(`Custom Knowledge Base created: ${knowledgeBase.id}`); +} catch (error) { + console.error("Failed to create knowledge base:", error); +} +``` + +```python title="Python SDK" +import os +from vapi import Vapi + +client = Vapi(token=os.getenv("VAPI_API_KEY")) + +try: + knowledge_base = client.knowledge_bases.create( + provider="custom-knowledge-base", + server={ + "url": "https://your-domain.com/kb/search", + "secret": "your-webhook-secret" + } + ) + + print(f"Custom Knowledge Base created: {knowledge_base.id}") +except Exception as error: + print(f"Failed to create knowledge base: {error}") +``` + + +### Step 2: Attach to Your Assistant + + + Custom knowledge bases can **only** be attached to assistants via the API. This functionality is not available through the dashboard interface. + + +To attach a custom knowledge base to your assistant, update the assistant's model configuration. You must provide the **complete** model configuration including all existing messages, as partial patches are not supported for nested objects: + + +```bash title="cURL" +curl --location --request PATCH 'https://api.vapi.ai/assistant/YOUR_ASSISTANT_ID' \ +--header 'Content-Type: application/json' \ +--header 'Authorization: Bearer YOUR_VAPI_API_KEY' \ +--data '{ + "model": { + "model": "gpt-4o", + "provider": "openai", + "messages": [ + { + "role": "system", + "content": "Your existing system prompt and instructions..." + } + ], + "knowledgeBaseId": "YOUR_KNOWLEDGE_BASE_ID" + } +}' +``` + +```typescript title="TypeScript SDK" +// First, get the existing assistant to preserve current configuration +const existingAssistant = await vapi.assistants.get("YOUR_ASSISTANT_ID"); + +const updatedAssistant = await vapi.assistants.update("YOUR_ASSISTANT_ID", { + model: { + ...existingAssistant.model, // Preserve existing model configuration + knowledgeBaseId: "YOUR_KNOWLEDGE_BASE_ID" // Add knowledge base + } +}); +``` + +```python title="Python SDK" +# First, get the existing assistant to preserve current configuration +existing_assistant = client.assistants.get(id="YOUR_ASSISTANT_ID") + +updated_assistant = client.assistants.update( + id="YOUR_ASSISTANT_ID", + model={ + **existing_assistant.model, # Preserve existing model configuration + "knowledgeBaseId": "YOUR_KNOWLEDGE_BASE_ID" # Add knowledge base + } +) +``` + + + + When updating an assistant's model, you must include the **complete model object** including all existing messages and configuration. The API replaces the entire model object and doesn't support partial updates for nested objects. + + +## Implementing the Custom Endpoint + +Your custom knowledge base server must handle POST requests at the configured URL and return structured responses. + +### Request Structure + +Vapi will send requests to your endpoint with the following structure: + +```json title="Request Format" +{ + "message": { + "type": "knowledge-base-request", + "messages": [ + { + "role": "user", + "content": "What is your return policy?" + }, + { + "role": "assistant", + "content": "I'll help you with information about our return policy." + }, + { + "role": "user", + "content": "How long do I have to return items?" + } + ] + // Additional metadata fields about the call or chat will be included here + } +} +``` + +### Response Options + +Your endpoint can respond in two ways: + +#### Option 1: Return Documents for AI Processing + +Return an array of relevant documents that the AI will use to formulate a response: + +```json title="Document Response" +{ + "documents": [ + { + "content": "Our return policy allows customers to return items within 30 days of purchase for a full refund. Items must be in original condition with tags attached.", + "similarity": 0.92, + "uuid": "doc-return-policy-1" // optional + }, + { + "content": "Extended return periods apply during holiday seasons - customers have up to 60 days to return items purchased between November 1st and December 31st.", + "similarity": 0.78, + "uuid": "doc-return-policy-holiday" // optional + } + ] +} +``` + +#### Option 2: Return Direct Response + +Return a complete response that the assistant will speak directly: + +```json title="Direct Response" +{ + "message": { + "role": "assistant", + "content": "You have 30 days to return items for a full refund. Items must be in original condition with tags attached. During the holiday season (November 1st to December 31st), you get an extended 60-day return period." + } +} +``` + +### Implementation Examples + +Here are complete server implementations in different languages: + + +```typescript title="Node.js/Express" +import express from 'express'; +import crypto from 'crypto'; + +const app = express(); +app.use(express.json()); + +// Your knowledge base data (replace with actual database/vector store) +const documents = [ + { + id: "return-policy-1", + content: "Our return policy allows customers to return items within 30 days of purchase for a full refund. Items must be in original condition with tags attached.", + category: "returns" + }, + { + id: "shipping-info-1", + content: "We offer free shipping on orders over $50. Standard shipping takes 3-5 business days.", + category: "shipping" + } +]; + +app.post('/kb/search', (req, res) => { + try { + // Verify webhook secret (recommended) + const signature = req.headers['x-vapi-signature']; + const secret = process.env.VAPI_WEBHOOK_SECRET; + + if (signature && secret) { + const expectedSignature = crypto + .createHmac('sha256', secret) + .update(JSON.stringify(req.body)) + .digest('hex'); + + if (signature !== `sha256=${expectedSignature}`) { + return res.status(401).json({ error: 'Invalid signature' }); + } + } + + const { message } = req.body; + + if (message.type !== 'knowledge-base-request') { + return res.status(400).json({ error: 'Invalid request type' }); + } + + // Get the latest user message + const userMessages = message.messages.filter(msg => msg.role === 'user'); + const latestQuery = userMessages[userMessages.length - 1]?.content || ''; + + // Simple keyword-based search (replace with vector search) + const relevantDocs = documents + .map(doc => ({ + ...doc, + similarity: calculateSimilarity(latestQuery, doc.content) + })) + .filter(doc => doc.similarity > 0.1) + .sort((a, b) => b.similarity - a.similarity) + .slice(0, 3); + + // Return documents for AI processing + res.json({ + documents: relevantDocs.map(doc => ({ + content: doc.content, + similarity: doc.similarity, + uuid: doc.id + })) + }); + + } catch (error) { + console.error('Knowledge base search error:', error); + res.status(500).json({ error: 'Internal server error' }); + } +}); + +function calculateSimilarity(query: string, content: string): number { + // Simple similarity calculation (replace with proper vector similarity) + const queryWords = query.toLowerCase().split(' '); + const contentWords = content.toLowerCase().split(' '); + const matches = queryWords.filter(word => + contentWords.some(cWord => cWord.includes(word)) + ).length; + + return matches / queryWords.length; +} + +app.listen(3000, () => { + console.log('Custom Knowledge Base server running on port 3000'); +}); +``` + +```python title="Python/FastAPI" +from fastapi import FastAPI, HTTPException, Request +import hashlib +import hmac +import os +from typing import List, Dict, Any +import uvicorn + +app = FastAPI() + +# Your knowledge base data (replace with actual database/vector store) +documents = [ + { + "id": "return-policy-1", + "content": "Our return policy allows customers to return items within 30 days of purchase for a full refund. Items must be in original condition with tags attached.", + "category": "returns" + }, + { + "id": "shipping-info-1", + "content": "We offer free shipping on orders over $50. Standard shipping takes 3-5 business days.", + "category": "shipping" + } +] + +@app.post("/kb/search") +async def knowledge_base_search(request: Request): + try: + body = await request.json() + + # Verify webhook secret (recommended) + signature = request.headers.get('x-vapi-signature') + secret = os.getenv('VAPI_WEBHOOK_SECRET') + + if signature and secret: + body_bytes = await request.body() + expected_signature = f"sha256={hmac.new(secret.encode(), body_bytes, hashlib.sha256).hexdigest()}" + + if signature != expected_signature: + raise HTTPException(status_code=401, detail="Invalid signature") + + message = body.get('message', {}) + + if message.get('type') != 'knowledge-base-request': + raise HTTPException(status_code=400, detail="Invalid request type") + + # Get the latest user message + user_messages = [msg for msg in message.get('messages', []) if msg.get('role') == 'user'] + latest_query = user_messages[-1].get('content', '') if user_messages else '' + + # Simple keyword-based search (replace with vector search) + relevant_docs = [] + for doc in documents: + similarity = calculate_similarity(latest_query, doc['content']) + if similarity > 0.1: + relevant_docs.append({ + **doc, + 'similarity': similarity + }) + + # Sort by similarity and take top 3 + relevant_docs.sort(key=lambda x: x['similarity'], reverse=True) + relevant_docs = relevant_docs[:3] + + # Return documents for AI processing + return { + "documents": [ + { + "content": doc['content'], + "similarity": doc['similarity'], + "uuid": doc['id'] + } + for doc in relevant_docs + ] + } + + except Exception as error: + print(f"Knowledge base search error: {error}") + raise HTTPException(status_code=500, detail="Internal server error") + +def calculate_similarity(query: str, content: str) -> float: + """Simple similarity calculation (replace with proper vector similarity)""" + query_words = query.lower().split() + content_words = content.lower().split() + + matches = sum(1 for word in query_words + if any(word in cword for cword in content_words)) + + return matches / len(query_words) if query_words else 0 + +if __name__ == "__main__": + uvicorn.run(app, host="0.0.0.0", port=8000) +``` + + +## Advanced Implementation Patterns + +### Vector Database Integration + +For production use, integrate with a proper vector database: + + +```typescript title="Pinecone Integration" +import { PineconeClient } from '@pinecone-database/pinecone'; +import OpenAI from 'openai'; + +const pinecone = new PineconeClient(); +const openai = new OpenAI(); + +app.post('/kb/search', async (req, res) => { + try { + const { message } = req.body; + const latestQuery = getLatestUserMessage(message); + + // Generate embedding for the query + const embedding = await openai.embeddings.create({ + model: 'text-embedding-ada-002', + input: latestQuery + }); + + // Search vector database + const index = pinecone.Index('knowledge-base'); + const searchResults = await index.query({ + vector: embedding.data[0].embedding, + topK: 5, + includeMetadata: true + }); + + // Format response + const documents = searchResults.matches.map(match => ({ + content: match.metadata.content, + similarity: match.score, + uuid: match.id + })); + + res.json({ documents }); + } catch (error) { + console.error('Vector search error:', error); + res.status(500).json({ error: 'Search failed' }); + } +}); +``` + +```python title="Weaviate Integration" +import weaviate +from sentence_transformers import SentenceTransformer + +client = weaviate.Client("http://localhost:8080") +model = SentenceTransformer('all-MiniLM-L6-v2') + +@app.post("/kb/search") +async def search_with_weaviate(request: Request): + try: + body = await request.json() + message = body.get('message', {}) + latest_query = get_latest_user_message(message) + + # Search using Weaviate + result = client.query.get("Document", ["content", "title"]) \ + .with_near_text({"concepts": [latest_query]}) \ + .with_limit(5) \ + .with_additional(["certainty"]) \ + .do() + + documents = [] + for doc in result['data']['Get']['Document']: + documents.append({ + "content": doc['content'], + "similarity": doc['_additional']['certainty'], + "uuid": doc.get('title', 'unknown') + }) + + return {"documents": documents} + except Exception as error: + raise HTTPException(status_code=500, detail=str(error)) +``` + + +## Security and Best Practices + +### Performance Optimization + + + **Response time is critical**: Your endpoint should respond in **milliseconds** (ideally under ~50ms) for optimal user experience. While Vapi allows up to 10 seconds timeout, slower responses will significantly affect your assistant's conversational flow and response quality. + + + + **Cache frequently requested documents** and implement request timeouts to ensure fast response times. Consider using in-memory caches, CDNs, or pre-computed embeddings for faster retrieval. + + +### Error Handling + +Always handle errors gracefully and return appropriate HTTP status codes: + +```typescript +app.post('/kb/search', async (req, res) => { + try { + // Your search logic here + } catch (error) { + console.error('Search error:', error); + + // Return empty documents rather than failing + res.json({ + documents: [], + error: "Search temporarily unavailable" + }); + } +}); +``` + +## Next Steps + +Now that you have a custom knowledge base implementation: + +- **[Query Tool Configuration](/knowledge-base/using-query-tool):** Learn advanced query tool configurations +- **[Vector Databases](/knowledge-base/integrating-with-trieve):** Explore vector database integrations +- **[Assistant Configuration](/assistants):** Optimize your assistant's use of knowledge bases + + + Custom Knowledge Bases require a webhook endpoint that's publicly accessible. For production deployments, ensure your server can handle concurrent requests and has appropriate error handling and monitoring in place. + diff --git a/fern/knowledge-base/migrating-from-trieve.mdx b/fern/knowledge-base/migrating-from-trieve.mdx new file mode 100644 index 000000000..ab25feefc --- /dev/null +++ b/fern/knowledge-base/migrating-from-trieve.mdx @@ -0,0 +1,407 @@ +--- +title: Migrating from Trieve +subtitle: Essential migration guide for Trieve users - service ending November 1st, 2024 +slug: knowledge-base/migrating-from-trieve +--- + + + **URGENT: Trieve is shutting down on November 1st, 2024.** All Trieve-based knowledge bases must be migrated before this date to avoid service disruption. + + +## Overview + +If you're currently using Trieve for your Vapi knowledge bases, you need to migrate before November 1st, 2024. This guide provides two migration paths to ensure your assistant continues working seamlessly. + +**Available migration options:** +- **Custom Knowledge Base**: Full control with your own search infrastructure +- **Google Knowledge Base**: Managed solution using Google's Gemini models + + + **Important Behavioral Difference**: Google knowledge bases work as tools that the assistant calls when needed based on your prompt instructions. Custom knowledge bases are queried on every user request automatically. Choose based on whether you want selective or automatic knowledge retrieval. + + + + **Recommendation**: To maintain the same assistant behavior as your previous Trieve setup, we recommend migrating to a **Custom Knowledge Base**. This ensures your assistant continues to automatically query your knowledge on every user message, just like it did with Trieve. + + +## Migration Options Comparison + +Choose the migration path that best fits your needs: + + + + **Best for:** Advanced users wanting automatic knowledge retrieval + + - Queried on every user message + - Full control over search logic + - Use any vector database or search system + - Requires server development + + + **Best for:** Quick migration with selective knowledge use + + - Works as a tool called when needed + - Simple file upload process + - No server maintenance required + - Assistant decides when to use knowledge + + + +### Decision Matrix + +| Factor | Custom Knowledge Base | Google Knowledge Base | +|--------|----------------------|----------------------| +| **Retrieval Behavior** | Queried on every user message | Called as tool when needed by assistant | +| **Latency** | Lower (direct database query) | Higher (Google Gemini model processing) | +| **Technical Complexity** | High | Low | +| **Ongoing Maintenance** | Server management required | None | +| **Customization** | Complete flexibility | Limited | +| **Cost** | Server hosting costs | Included in Vapi | +| **Search Quality** | Depends on implementation | Google Gemini quality | + +## Migrating to Custom Knowledge Base + +### Step 1: Export Your Data + +Your export process depends on how your Trieve knowledge base was originally created: + +#### Option A: User-Managed Trieve Dataset (Import Plan) +If you created your knowledge base with your own Trieve dataset: + +1. Access your Trieve dashboard directly +2. Export your datasets in your preferred format +3. Download all document chunks and metadata +4. Save any custom embeddings if applicable + +#### Option B: Vapi-Managed Trieve Dataset (Create Plan) +If Vapi created and managed your Trieve dataset: + +1. Navigate to the **Files** page in your Vapi dashboard +2. Download the individual files that were uploaded to create the knowledge base +3. These files are stored in Vapi and are not affected by the Trieve shutdown + + + **Files uploaded to Vapi are safe**: If your knowledge base was created using files uploaded to Vapi, those files remain available in your Vapi account and are not affected by the Trieve shutdown. + + +### Step 2: Set Up Your Custom Server + +Choose your infrastructure and implement the knowledge base endpoint: + + +```typescript title="Node.js Example" +import express from 'express'; +import { PineconeClient } from '@pinecone-database/pinecone'; + +const app = express(); +const pinecone = new PineconeClient(); + +app.post('/kb/search', async (req, res) => { + try { + const { message } = req.body; + const userMessages = message.messages.filter(msg => msg.role === 'user'); + const latestQuery = userMessages[userMessages.length - 1]?.content || ''; + + // Search your vector database + const index = pinecone.Index('your-knowledge-base'); + const searchResults = await index.query({ + vector: await getEmbedding(latestQuery), + topK: 5, + includeMetadata: true + }); + + // Format response + const documents = searchResults.matches.map(match => ({ + content: match.metadata.content, + similarity: match.score, + uuid: match.id + })); + + res.json({ documents }); + } catch (error) { + res.status(500).json({ error: 'Search failed' }); + } +}); + +app.listen(3000); +``` + +```python title="Python Example" +from fastapi import FastAPI +import weaviate +from sentence_transformers import SentenceTransformer + +app = FastAPI() +client = weaviate.Client("http://localhost:8080") + +@app.post("/kb/search") +async def search_knowledge_base(request: dict): + try: + message = request.get('message', {}) + user_messages = [msg for msg in message.get('messages', []) if msg.get('role') == 'user'] + latest_query = user_messages[-1].get('content', '') if user_messages else '' + + # Search your vector database + result = client.query.get("Document", ["content", "title"]) \ + .with_near_text({"concepts": [latest_query]}) \ + .with_limit(5) \ + .with_additional(["certainty"]) \ + .do() + + documents = [] + for doc in result['data']['Get']['Document']: + documents.append({ + "content": doc['content'], + "similarity": doc['_additional']['certainty'], + "uuid": doc.get('title', 'unknown') + }) + + return {"documents": documents} + except Exception as error: + return {"error": str(error)} +``` + + +### Step 3: Create Custom Knowledge Base in Vapi + + +```bash title="cURL" +curl --location 'https://api.vapi.ai/knowledge-base' \ +--header 'Content-Type: application/json' \ +--header 'Authorization: Bearer YOUR_VAPI_API_KEY' \ +--data '{ + "provider": "custom-knowledge-base", + "server": { + "url": "https://your-domain.com/kb/search", + "secret": "your-webhook-secret" + } +}' +``` + +```typescript title="TypeScript SDK" +const knowledgeBase = await vapi.knowledgeBases.create({ + provider: "custom-knowledge-base", + server: { + url: "https://your-domain.com/kb/search", + secret: "your-webhook-secret" + } +}); +``` + + +### Step 4: Update Your Assistant + +Replace your Trieve knowledge base with the custom one: + + +```bash title="cURL" +curl --location --request PATCH 'https://api.vapi.ai/assistant/YOUR_ASSISTANT_ID' \ +--header 'Content-Type: application/json' \ +--header 'Authorization: Bearer YOUR_VAPI_API_KEY' \ +--data '{ + "model": { + "model": "gpt-4o", + "provider": "openai", + "messages": [ + { + "role": "system", + "content": "Your existing system prompt..." + } + ], + "knowledgeBaseId": "YOUR_NEW_KNOWLEDGE_BASE_ID" + } +}' +``` + +```typescript title="TypeScript SDK" +// Get existing configuration +const existingAssistant = await vapi.assistants.get("YOUR_ASSISTANT_ID"); + +// Update with new knowledge base +const updatedAssistant = await vapi.assistants.update("YOUR_ASSISTANT_ID", { + model: { + ...existingAssistant.model, + knowledgeBaseId: knowledgeBase.id + } +}); +``` + + +## Migrating to Google Knowledge Base + +### Step 1: Prepare Your Files + +Your file preparation depends on how your Trieve knowledge base was originally created: + +#### Option A: User-Managed Trieve Dataset (Import Plan) +If you created your knowledge base with your own Trieve dataset: + +1. Export your Trieve datasets as individual files +2. Organize files by topic or domain +3. Ensure files are in supported formats (`.txt`, `.pdf`, `.docx`, `.md`, etc.) +4. Keep individual files under 300KB for optimal processing +5. **Upload these files to Vapi** (you'll need them in Vapi to create the query tool) + +#### Option B: Vapi-Managed Trieve Dataset (Create Plan) +If Vapi created and managed your Trieve dataset: + +1. Navigate to the **Files** page in your Vapi dashboard +2. Locate the files that were used to create your Trieve knowledge base +3. Note the **file IDs** - you'll use these directly to create your query tool +4. **No need to download or re-upload** - the files are already in Vapi + + + **Files uploaded to Vapi are safe**: If your knowledge base was created using files uploaded to Vapi, those files remain available and are not affected by the Trieve shutdown. + + +### Step 2: Upload Files to Vapi (Option A Only) + + + **This step is only required for Option A (User-Managed Trieve)**. If you're using Option B (Vapi-Managed), skip to Step 3 as your files are already in Vapi. + + +If you exported files from your own Trieve dataset (Option A): + +1. Navigate to **Build > Files** in your Vapi dashboard +2. Click **Upload** and select your exported files +3. Wait for processing and note the generated file IDs + +### Step 3: Create Query Tool + +Create a query tool using your file IDs: + +- **Option A**: Use the file IDs from the files you just uploaded in Step 2 +- **Option B**: Use the existing file IDs you noted from your Vapi Files page + +For detailed instructions, see our [Query Tool documentation](/knowledge-base/using-query-tool). + + +```bash title="cURL" +curl --location 'https://api.vapi.ai/tool' \ +--header 'Content-Type: application/json' \ +--header 'Authorization: Bearer YOUR_VAPI_API_KEY' \ +--data '{ + "type": "query", + "function": { + "name": "knowledge-search" + }, + "knowledgeBases": [ + { + "provider": "google", + "name": "migrated-knowledge-base", + "description": "Migrated knowledge base from Trieve containing product documentation and support information", + "fileIds": [ + "FILE_ID_1", + "FILE_ID_2", + "FILE_ID_3" + ] + } + ] +}' +``` + +```typescript title="TypeScript SDK" +const queryTool = await vapi.tools.create({ + type: "query", + function: { + name: "knowledge-search" + }, + knowledgeBases: [ + { + provider: "google", + name: "migrated-knowledge-base", + description: "Migrated knowledge base from Trieve containing product documentation and support information", + fileIds: [ + "FILE_ID_1", + "FILE_ID_2", + "FILE_ID_3" + ] + } + ] +}); +``` + + +### Step 4: Update Your Assistant + +Remove Trieve tools and add your new query tool: + + +```bash title="cURL" +curl --location --request PATCH 'https://api.vapi.ai/assistant/YOUR_ASSISTANT_ID' \ +--header 'Content-Type: application/json' \ +--header 'Authorization: Bearer YOUR_VAPI_API_KEY' \ +--data '{ + "model": { + "model": "gpt-4o", + "provider": "openai", + "messages": [ + { + "role": "system", + "content": "Your existing system prompt..." + } + ], + "toolIds": ["YOUR_NEW_QUERY_TOOL_ID"] + } +}' +``` + +```typescript title="TypeScript SDK" +// Get existing assistant +const existingAssistant = await vapi.assistants.get("YOUR_ASSISTANT_ID"); + +// Update with new tool +const updatedAssistant = await vapi.assistants.update("YOUR_ASSISTANT_ID", { + model: { + ...existingAssistant.model, + toolIds: [queryTool.id] + } +}); +``` + + +## Post-Migration Checklist + +After completing your migration: + +- [ ] **Test all assistants** to ensure knowledge retrieval is working +- [ ] **Monitor response quality** and adjust configurations if needed +- [ ] **Remove the old Trieve knowledge base** from Vapi using API. +- [ ] **Remove Trieve credentials** from Vapi dashboard + +## Troubleshooting Common Issues + +### Poor Search Quality After Migration + +**Custom Knowledge Base:** +- Verify your embedding model matches or exceeds Trieve's quality +- Adjust similarity thresholds based on your model +- Test different chunking strategies +- Consider implementing hybrid search (semantic + keyword) + +**Google Knowledge Base:** +- Review file formatting and structure +- Optimize knowledge base descriptions +- Test with different query phrasings +- Consider splitting large files into smaller, focused documents + +### Performance Issues + +**Custom Knowledge Base:** +- Implement caching for frequently searched content +- Optimize your vector database configuration +- Consider using faster embedding models for latency-critical applications +- Monitor and optimize server response times + +**Google Knowledge Base:** +- Ensure files are under 300KB each +- Use clear, well-structured file formats +- Minimize the number of files per knowledge base where possible + +## Next Steps + +Once you've completed your migration: + +- **[Custom Knowledge Base](/knowledge-base/custom-knowledge-base):** Learn advanced customization options +- **[Query Tool Configuration](/knowledge-base/using-query-tool):** Optimize your Google knowledge base setup +- **[Assistant Configuration](/assistants):** Fine-tune your assistant's knowledge base usage From c442256674922334238cf905cd2cb65a88bd5873 Mon Sep 17 00:00:00 2001 From: Sri Date: Fri, 12 Sep 2025 14:31:43 -0700 Subject: [PATCH 2/4] updated gemini-1.5 note --- fern/knowledge-base/knowledge-base.mdx | 2 +- fern/knowledge-base/using-query-tool.mdx | 5 ----- 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/fern/knowledge-base/knowledge-base.mdx b/fern/knowledge-base/knowledge-base.mdx index 9d167d8dd..ca9cd4225 100644 --- a/fern/knowledge-base/knowledge-base.mdx +++ b/fern/knowledge-base/knowledge-base.mdx @@ -139,7 +139,7 @@ By following these guidelines, you can create a comprehensive Knowledge Base tha Currently, Vapi's Knowledge Base functionality supports Google as a provider - with the gemini-1.5-flash model for knowledge retrieval. For the most + with Gemini models for knowledge retrieval. For the most up-to-date information on supported providers and models, please refer to our [API documentation](api-reference/tools/create#request.body.query.knowledgeBases). diff --git a/fern/knowledge-base/using-query-tool.mdx b/fern/knowledge-base/using-query-tool.mdx index 22747a331..8c957d801 100644 --- a/fern/knowledge-base/using-query-tool.mdx +++ b/fern/knowledge-base/using-query-tool.mdx @@ -15,11 +15,6 @@ The Query Tool is a powerful feature that allows your voice AI assistant to acce - **Improved response accuracy**: Responses are based on your verified information rather than general knowledge. - **Customizable knowledge retrieval**: Configure multiple knowledge bases for different topics or domains. - - Currently, the Query Tool only supports Google as a provider with the - gemini-1.5-flash model for knowledge base retrieval. - - ## **How to Configure a Query Tool for Knowledge Bases** ### **Step 1: Upload Your Files** From 624868453a7f8d250d4468a126bb2ad868c929a4 Mon Sep 17 00:00:00 2001 From: Sri Date: Fri, 12 Sep 2025 14:46:28 -0700 Subject: [PATCH 3/4] updated the query tool docs --- fern/knowledge-base/migrating-from-trieve.mdx | 12 +++-- fern/knowledge-base/using-query-tool.mdx | 45 +++++++++++++++---- 2 files changed, 45 insertions(+), 12 deletions(-) diff --git a/fern/knowledge-base/migrating-from-trieve.mdx b/fern/knowledge-base/migrating-from-trieve.mdx index ab25feefc..e273fcf99 100644 --- a/fern/knowledge-base/migrating-from-trieve.mdx +++ b/fern/knowledge-base/migrating-from-trieve.mdx @@ -275,6 +275,10 @@ Create a query tool using your file IDs: For detailed instructions, see our [Query Tool documentation](/knowledge-base/using-query-tool). + + **Don't forget**: You must update your assistant's system prompt to explicitly name the tool to use. Add instructions like: "When users ask about products or support topics, use the 'knowledge-search' tool to search the knowledge base." Replace 'knowledge-search' with your actual tool function name. + + ```bash title="cURL" curl --location 'https://api.vapi.ai/tool' \ @@ -288,8 +292,8 @@ curl --location 'https://api.vapi.ai/tool' \ "knowledgeBases": [ { "provider": "google", - "name": "migrated-knowledge-base", - "description": "Migrated knowledge base from Trieve containing product documentation and support information", + "name": "product-support-documentation", + "description": "Contains comprehensive information about our products, services, features, pricing, and technical support documentation. Includes troubleshooting guides and company information.", "fileIds": [ "FILE_ID_1", "FILE_ID_2", @@ -309,8 +313,8 @@ const queryTool = await vapi.tools.create({ knowledgeBases: [ { provider: "google", - name: "migrated-knowledge-base", - description: "Migrated knowledge base from Trieve containing product documentation and support information", + name: "product-support-documentation", + description: "Contains comprehensive information about our products, services, features, pricing, and technical support documentation. Includes troubleshooting guides and company information.", fileIds: [ "FILE_ID_1", "FILE_ID_2", diff --git a/fern/knowledge-base/using-query-tool.mdx b/fern/knowledge-base/using-query-tool.mdx index 8c957d801..b1806e51a 100644 --- a/fern/knowledge-base/using-query-tool.mdx +++ b/fern/knowledge-base/using-query-tool.mdx @@ -53,7 +53,7 @@ Create a query tool that references your knowledge base files: - **Tool Name**: "Product Query" - **Knowledge Bases**: Add your knowledge base with: - **Name**: `product-kb` - - **Description**: "Use this knowledge base when the user asks or queries about the product or services" + - **Description**: "Contains comprehensive product information, service details, and company offerings" - **File IDs**: Select the files you uploaded in Step 1 #### Option 2: Using the API @@ -73,7 +73,7 @@ curl --location 'https://api.vapi.ai/tool/' \ { "provider": "google", "name": "product-kb", - "description": "Use this knowledge base when the user asks or queries about the product or services", + "description": "Contains comprehensive product information, service details, and company offerings", "fileIds": [ "41a2bd44-d13c-4914-bbf7-b19807dd2cf4", "ef82ae15-21b2-47bd-bde4-dea3922c1e49" @@ -99,7 +99,8 @@ After creating the query tool, you need to attach it to your assistant: 2. Select the assistant you want to configure 3. Go to the **Tools** section 4. Click **Add Tool** and select your query tool from the dropdown -5. Save and publish your assistant +5. **Update the system prompt** to instruct when to use the tool (see examples below) +6. Save and publish your assistant #### Option 2: Using the API @@ -125,6 +126,10 @@ curl --location --request PATCH 'https://api.vapi.ai/assistant/ASSISTANT_ID' \ just the toolIds field. This will overwrite any existing model configuration. + + **Don't forget**: After attaching the tool via API, you must also update your assistant's system prompt (in the `messages` array) to instruct when to use the tool by name. See the system prompt examples below. + + + **Important**: You must explicitly instruct your assistant in its system prompt about when to use the query tool. The knowledge base description alone is not sufficient for the assistant to know when to search. + + +Add clear instructions to your assistant's system messages, **explicitly naming the tool**: + +```json +{ + "role": "system", + "content": "You are a helpful customer support assistant. When users ask about products, pricing, features, or need troubleshooting help, use the 'knowledge-search' tool to search our knowledge base for accurate information. Always call the knowledge-search tool before providing answers to ensure accuracy." +} +``` + + + **Be specific**: Replace `'knowledge-search'` with your actual tool's function name from the query tool configuration. + + +**Example system prompt instructions with specific tool names:** +- **Product Support**: "When users ask about product features, specifications, or troubleshooting, use the 'product-support-search' tool to find relevant information." +- **Billing Questions**: "For any questions about pricing, billing, or subscription plans, use the 'billing-query' tool to find the most current information." +- **Technical Documentation**: "When users need API documentation, code examples, or integration help, use the 'api-docs-search' tool to search the technical knowledge base." + ## **Best Practices for Query Tool Configuration** - **Organize by topic**: Create separate knowledge bases for distinct topics to improve retrieval accuracy. - **Use descriptive names**: Name your knowledge bases clearly to help your assistant understand their purpose. -- **Keep descriptions specific**: Write clear descriptions that tell the assistant exactly when to use each knowledge base. +- **Include system prompt instructions**: Always add explicit instructions to your assistant's system prompt about when to use the query tool. - **Update regularly**: Refresh your knowledge bases as information changes to ensure accuracy. - **Test thoroughly**: After configuration, test your assistant with various queries to ensure it retrieves information correctly. From e8b7bd6865bedbf48b0003bcb3959b1ccdd42faf Mon Sep 17 00:00:00 2001 From: Sri Date: Fri, 12 Sep 2025 14:49:55 -0700 Subject: [PATCH 4/4] updated the docs --- fern/knowledge-base/migrating-from-trieve.mdx | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/fern/knowledge-base/migrating-from-trieve.mdx b/fern/knowledge-base/migrating-from-trieve.mdx index e273fcf99..520921ac1 100644 --- a/fern/knowledge-base/migrating-from-trieve.mdx +++ b/fern/knowledge-base/migrating-from-trieve.mdx @@ -1,16 +1,18 @@ --- title: Migrating from Trieve -subtitle: Essential migration guide for Trieve users - service ending November 1st, 2024 +subtitle: Essential migration guide for Trieve users - service ending November 1st, 2025 slug: knowledge-base/migrating-from-trieve --- - **URGENT: Trieve is shutting down on November 1st, 2024.** All Trieve-based knowledge bases must be migrated before this date to avoid service disruption. + **URGENT: Trieve is shutting down on November 1st, 2025.** All Trieve-based knowledge bases must be migrated before this date to avoid service disruption. ## Overview -If you're currently using Trieve for your Vapi knowledge bases, you need to migrate before November 1st, 2024. This guide provides two migration paths to ensure your assistant continues working seamlessly. +Trieve [is shutting down their cloud service](https://www.trieve.ai/blog/trieve-is-being-acquired-by-mintlify) on November 1st, 2025. If you're currently using Trieve for your Vapi knowledge bases, you need to migrate before this date to avoid service disruption. + +This guide provides two migration paths to ensure your assistant continues working seamlessly. **Available migration options:** - **Custom Knowledge Base**: Full control with your own search infrastructure