Add support for multi-modal search #712
Conversation
WalkthroughAdds multimodal support across the SDK: new experimental feature flag, search query media payload, and embedder fragment settings for indexing and search. Updates code samples to demonstrate media usage and adjusts a sample template. Includes constructors/builders, serialization, and tests for new fields. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant App as Client App
participant SDK as SDK (this repo)
participant MS as Meilisearch Server
rect rgba(230,240,255,0.6)
note over SDK: Settings update with fragments
App->>SDK: Embedder{ indexingFragments, searchFragments }
SDK->>MS: PUT /indexes/:uid/settings/embedders (JSON with fragments)
MS-->>SDK: 200 Accepted/Task
SDK-->>App: Task info
end
rect rgba(230,255,230,0.6)
note over SDK: Enable experimental multimodal
App->>SDK: ExperimentalFeatures.set_multimodal(true)
SDK->>MS: PATCH /experimental-features { multimodal: true }
MS-->>SDK: { multimodal: true, ... }
SDK-->>App: ExperimentalFeaturesResult
end
rect rgba(255,245,230,0.6)
note over SDK: Search with media payload
App->>SDK: SearchQuery.with_media(Value)
SDK->>MS: POST /indexes/:uid/search { media: {...}, ... }
MS-->>SDK: Search results
SDK-->>App: Results
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (5 passed)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (6)
src/search.rs (3)
411-414: Media field shape looks good; keep as optional.Adds
media: Option<Value>with proper serde skipping. Consider documenting expected JSON shape (object of fragments) since this is experimental.
703-707: Builder ergonomics: consider a fallible generic helper.
with_media(Value)is fine. Optionally addtry_with_media<T: Serialize>(&self, media: &T) -> Result<&mut Self, serde_json::Error>orwith_media_json<T: Serialize>(&mut self, media: T)to avoid requiring callers to pre-buildValue.+impl<'a, Http: HttpClient> SearchQuery<'a, Http> { + /// Attach media fragments from any serializable structure. + pub fn with_media_json<'b, T: Serialize>( + &'b mut self, + media: T, + ) -> &'b mut SearchQuery<'a, Http> { + self.media = serde_json::to_value(media).ok(); + self + } +}
1110-1136: Serialization test is solid; add a negative case.Great direct check that
mediaserializes as expected. Consider an extra test assertingmediais omitted when unset..code-samples.meilisearch.yaml (1)
1944-1957: New sample aligns with API additions.Example shows
.with_media(json!(...))alongside hybrid search; this mirrors the new builder. Optionally preface with a note in docs that the multimodal experimental feature must be enabled.src/settings.rs (2)
150-157: Embedder fragments exposed—API surface looks correct.
indexing_fragmentsandsearch_fragmentsasOption<HashMap<String, EmbedderFragment>>with camelCase serialization aligns with expected wire format. Consider adding brief docs stating these are experimental.
3157-3241: Good serialization coverage for fragments.Test validates both
indexingFragmentsandsearchFragments, plusrequest/response. Consider also asserting a round‑tripfrom_value::<Embedder>(serialized)to catch deserialization regressions.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
.code-samples.meilisearch.yaml(1 hunks)src/features.rs(6 hunks)src/search.rs(4 hunks)src/settings.rs(3 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-06-12T13:28:23.700Z
Learnt from: LukasKalbertodt
PR: meilisearch/meilisearch-rust#625
File: src/search.rs:368-370
Timestamp: 2025-06-12T13:28:23.700Z
Learning: In the Meilisearch Rust client, `SearchQuery` serializes its per-query federation settings under the key `federationOptions`; only the top-level multi-search parameter is named `federation`.
Applied to files:
src/search.rs
🧬 Code graph analysis (1)
src/search.rs (2)
src/indexes.rs (26)
client(186-188)client(232-234)client(320-322)client(376-378)client(428-430)client(473-475)client(522-525)client(556-558)client(634-636)client(700-702)client(970-972)client(1038-1040)client(1089-1091)client(1133-1135)client(1186-1188)client(1245-1247)new(81-89)new(1783-1790)new(1994-2000)index(2171-2171)index(2188-2189)index(2226-2226)index(2298-2298)index(2326-2326)index(2352-2352)index(2379-2379)src/features.rs (1)
new(56-66)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: integration-tests
🔇 Additional comments (7)
src/settings.rs (2)
2815-2815: Import addition OK.
use serde_json::json;required for tests.
159-163: Struct matches Meilisearch 1.16 specThe
EmbedderFragment { value: serde_json::Value }definition correctly wraps each fragment’s payload under a"value"key as required by the server schema. No changes needed.src/features.rs (5)
17-19: Backward-compatible result field.
multimodal: boolwith#[serde(default)]avoids breaking on older servers.
50-52: Request knob for multimodal.Optional
multimodalfield mirrors the response; consistent with existing flags.
64-65: Constructor default None—consistent.Keeps updates explicit.
149-152: Setter reads cleanly.
set_multimodalmatches existing builder style.
168-178: Test covers end-to-end toggle.Verifies update + get for
multimodal. Looks good.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #712 +/- ##
==========================================
+ Coverage 85.89% 86.14% +0.24%
==========================================
Files 19 19
Lines 5950 6057 +107
==========================================
+ Hits 5111 5218 +107
Misses 839 839 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Pull Request
Related issue
Fixes #698
What does this PR do?
PR checklist
Please check if your PR fulfills the following requirements:
Thank you so much for contributing to Meilisearch!
Summary by CodeRabbit
New Features
Documentation