Enhancing Voice Assistant with Context Retrieval using Pinecone

[pablojrosa]

This PR introduces an improvement to LiveKit’s Voice Assistant by integrating a context retrieval layer using Pinecone. The implementation allows the assistant to access relevant stored information dynamically, improving response quality and contextual awareness.

Key Enhancements:

• Pinecone Integration: Retrieves relevant data based on user input to provide more accurate and informed responses.
• Custom Embeddings with OpenAI: Converts user queries into vector embeddings (text-embedding-ada-002) for efficient similarity search.
• Preprocessing Step (before_llm_cb): Injects retrieved context into the conversation before querying the LLM, ensuring continuity in interactions.
• Deepgram TTS Integration: Uses Deepgram Aura Asteria for high-quality text-to-speech synthesis.

Why This Matters:

By leveraging vector search and context-aware responses, the assistant can provide more intelligent, memory-like interactions, especially useful in long-form conversations or domain-specific applications.

[changeset-bot]

⚠️ No Changeset found

Latest commit: 9069d40

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[CLAassistant]

All committers have signed the CLA.