feat(gemini): Add support for RAG documents in dynamic context by snaumov · Pull Request #1205 · 0xPlaygrounds/rig

snaumov · 2026-01-04T03:46:25Z

Add RAG Documents Support to Gemini Provider

Summary

This PR adds support for RAG (Retrieval-Augmented Generation) documents in the Gemini provider's dynamic_context() functionality. Previously, documents retrieved from vector stores were silently ignored, causing RAG agents to fail.

Problem

When using .dynamic_context() with Gemini agents, the retrieved documents were never passed to the Gemini API. The create_request_body() function in the Gemini provider ignored the documents field from CompletionRequest, making it impossible to use RAG with Gemini.

Solution

Changes Made

Document Injection: Modified create_request_body() to check for documents and inject them as a user message at the beginning of chat history using the existing normalized_documents() helper method.
Text Document Handling: Added special handling for DocumentMediaType::TXT and other text-based documents to convert them to plain text parts (PartKind::Text) instead of trying to send them as base64-encoded inline data.
Backward Compatibility: Preserved existing behavior for other document types (images, PDFs, etc.) that use InlineData or FileData.

Code Changes

File: rig/rig-core/src/providers/gemini/completion.rs

Lines ~189-195: Added document injection logic
Lines ~773-838: Added TXT and other text-based document to text conversion

Testing

Tested with a real-world RAG application:

Application: Obsidian vault note organizer with 175+ markdown notes
Vector Store: In-memory vector store with Ollama embeddings (nomic-embed-text)
Query: "<some_query>"
Result: Successfully retrieved 10 relevant notes and Gemini correctly received and processed them

Before Fix

=== NOTES IN CONTEXT ===
NO NOTES IN CONTEXT
=== END OF NOTES ===

After Fix

=== NOTES IN CONTEXT ===
<list_of_notes>
=== END OF NOTES ===

Impact

✅ Enables RAG functionality for Gemini provider
✅ No breaking changes
✅ Maintains backward compatibility
✅ Follows existing patterns (uses normalized_documents())

Additional Notes

The fix aligns Gemini's behavior with other providers like Cohere and Anthropic that already support documents in RAG scenarios.

joshua-mo-143

Hey, apologies for the late review - I meant to get around to this but I wasn't able to do so.

Needs cargo fmt then lgtm

This commit adds support for passing RAG documents to the Gemini API when using dynamic_context() with agents. Previously, documents retrieved from vector stores were ignored by the Gemini provider, causing RAG systems to fail silently. Changes: - Modified create_request_body() to inject documents as user messages at the beginning of chat history - Added special handling for DocumentMediaType::TXT to convert text documents to plain text parts instead of base64-encoded inline data - Preserved existing behavior for other document types (images, PDFs, etc.) Fixes: - Documents from dynamic_context() now properly reach the Gemini model - RAG-based agents can now successfully retrieve and use context from vector stores Testing: - Tested with a note-taking RAG system using Obsidian vault notes - Verified that documents are correctly passed and interpreted by Gemini - Confirmed backward compatibility with existing document handling

Added comprehensive tests for the new document handling functionality: 1. test_txt_document_conversion_to_text_part: Verifies that TXT documents are converted to PartKind::Text instead of InlineData 2. test_create_request_body_with_documents: Tests that documents are correctly injected into chat history at the beginning 3. test_create_request_body_without_documents: Ensures backward compatibility when no documents are present All tests pass successfully.

snaumov · 2026-02-01T02:47:14Z

Hey, apologies for the late review - I meant to get around to this but I wasn't able to do so.

Needs cargo fmt then lgtm

updated, thank you!

atellou · 2026-02-04T15:04:24Z

@snaumov I am working on a feature, and this solved it for me. I was getting random result till I used your branch. Thank you. Looking forward to seeing this merged so my own PR actually works

snaumov · 2026-02-04T16:06:22Z

@atellou glad to hear it helped. @joshua-mo-143 let me know please if there is anything else to be done before we can merge

snaumov · 2026-02-04T16:06:23Z

@atellou glad to hear it helped. @joshua-mo-143 let me know please if there is anything else to be done before we can merge

joshua-mo-143 · 2026-02-04T16:09:12Z

Hey - sorry been meaning to get around to this but haven't kept my eye on it. I'm currently travelling but will have a look when I'm available - doesn't look like a lot needs to be done if anything iirc

joshua-mo-143 · 2026-02-04T16:09:13Z

Hey - sorry been meaning to get around to this but haven't kept my eye on it. I'm currently travelling but will have a look when I'm available - doesn't look like a lot needs to be done if anything iirc

…xPlaygrounds#1205 - Convert text-based documents (TXT, Markdown, HTML, etc.) to plain text parts instead of inline data - Inject documents as user message at beginning of chat history in create_request_body - Add base64 decoding for text documents when needed - Fix TypedPromptResponse to use `usage` field instead of `total_usage` Includes comprehensive tests for: - TXT and Markdown document conversion to text parts - Document injection into request

atellou · 2026-03-03T14:56:18Z

I created a version of this PR up to date with the main branch and included my change: #1456

joshua-mo-143 · 2026-03-03T17:51:55Z

I created a version of this PR up to date with the main branch and included my change: #1456

I'll be merging this PR first. Trying to solve multiple issues with a single PR is generally not a great idea.

atellou · 2026-03-03T18:02:56Z

Works for me

joshua-mo-143 · 2026-03-03T21:27:31Z

Finally merged!

@snaumov Thanks for your hard work on the initial implementation of this, I know it's taken a while to get this merged. I have made necessary changes and merged this PR as this was supposed to be merged at an earlier point but I got distracted and kept forgetting.

snaumov · 2026-03-07T15:59:27Z

@joshua-mo-143 thank you for pushing it across finish line! Sorry got distracted myself and didn't rebase on time.

snaumov marked this pull request as ready for review January 4, 2026 05:03

bajie-git mentioned this pull request Jan 30, 2026

feat/bug: Gemini/mistral.rs: dynamic_context is ignored/leads to an error #1179

Open

1 task

joshua-mo-143 suggested changes Jan 30, 2026

View reviewed changes

snaumov added 5 commits January 31, 2026 21:45

adding more doc types

45fbeb0

cleanup

9f2aa9f

fmt

c0c6b31

snaumov force-pushed the feature/gemini-rag-documents-support branch from f960195 to c0c6b31 Compare February 1, 2026 02:46

snaumov requested a review from joshua-mo-143 February 1, 2026 02:47

atellou mentioned this pull request Mar 3, 2026

feat(gemini): include dynamic context for extraction builder #1456

Merged

Merge branch 'main' into feature/gemini-rag-documents-support

0ccca94

joshua-mo-143 added 2 commits March 3, 2026 20:04

chore: amendments

c7ab7ee

chore: amendments

648eeb2

joshua-mo-143 added this pull request to the merge queue Mar 3, 2026

Merged via the queue into 0xPlaygrounds:main with commit 99df454 Mar 3, 2026
6 checks passed

github-actions Bot mentioned this pull request Mar 3, 2026

chore: release #1414

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(gemini): Add support for RAG documents in dynamic context#1205

feat(gemini): Add support for RAG documents in dynamic context#1205
joshua-mo-143 merged 8 commits into0xPlaygrounds:mainfrom
snaumov:feature/gemini-rag-documents-support

snaumov commented Jan 4, 2026 •

edited

Loading

Uh oh!

joshua-mo-143 left a comment

Uh oh!

snaumov commented Feb 1, 2026

Uh oh!

atellou commented Feb 4, 2026

Uh oh!

snaumov commented Feb 4, 2026

Uh oh!

snaumov commented Feb 4, 2026

Uh oh!

joshua-mo-143 commented Feb 4, 2026

Uh oh!

joshua-mo-143 commented Feb 4, 2026

Uh oh!

atellou commented Mar 3, 2026

Uh oh!

joshua-mo-143 commented Mar 3, 2026

Uh oh!

atellou commented Mar 3, 2026

Uh oh!

Uh oh!

joshua-mo-143 commented Mar 3, 2026 •

edited

Loading

Uh oh!

snaumov commented Mar 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

snaumov commented Jan 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add RAG Documents Support to Gemini Provider

Summary

Problem

Solution

Changes Made

Code Changes

Testing

Before Fix

After Fix

Impact

Additional Notes

Uh oh!

joshua-mo-143 left a comment

Choose a reason for hiding this comment

Uh oh!

snaumov commented Feb 1, 2026

Uh oh!

atellou commented Feb 4, 2026

Uh oh!

snaumov commented Feb 4, 2026

Uh oh!

snaumov commented Feb 4, 2026

Uh oh!

joshua-mo-143 commented Feb 4, 2026

Uh oh!

joshua-mo-143 commented Feb 4, 2026

Uh oh!

atellou commented Mar 3, 2026

Uh oh!

joshua-mo-143 commented Mar 3, 2026

Uh oh!

atellou commented Mar 3, 2026

Uh oh!

Uh oh!

joshua-mo-143 commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

snaumov commented Mar 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

snaumov commented Jan 4, 2026 •

edited

Loading

joshua-mo-143 commented Mar 3, 2026 •

edited

Loading