Skip to content

Commit 0fe8e10

Browse files
authored
Resolve all non-google file uris when calling gemini Fixes #1548 (#1553)
<!-- ELLIPSIS_HIDDEN --> > [!IMPORTANT] > Add `IfMatchesGoogleFileUri` to handle Google file URIs in media URL resolution, updating `GoogleAIClient` and `process_media` logic. > > - **Behavior**: > - Add `IfMatchesGoogleFileUri` variant to `ResolveMediaUrls` enum in `mod.rs` to handle Google file URIs specifically. > - Update `process_media` in `traits/mod.rs` to return the original part if the URL starts with `gs://` when `IfMatchesGoogleFileUri` is used. > - Change `resolve_media_urls` to `IfMatchesGoogleFileUri` in `GoogleAIClient` in `googleai_client.rs`. > - **Dependencies**: > - Add `regex` dependency to `Cargo.toml` and `Cargo.lock` for potential pattern matching use. > - **Misc**: > - Update comments in `mod.rs` and `googleai_client.rs` to reflect new behavior for Google file URIs. > > <sup>This description was created by </sup>[<img alt="Ellipsis" src="https://img.shields.io/badge/Ellipsis-blue?color=175173">](https://www.ellipsis.dev?ref=BoundaryML%2Fbaml&utm_source=github&utm_medium=referral)<sup> for 25f577e. It will automatically update as commits are pushed.</sup> <!-- ELLIPSIS_HIDDEN -->
1 parent be1119f commit 0fe8e10

File tree

5 files changed

+10
-2
lines changed

5 files changed

+10
-2
lines changed

engine/Cargo.lock

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

engine/baml-runtime/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ web-time.workspace = true
6767
static_assertions.workspace = true
6868
mime_guess = "=2.0.5"
6969
mime = "0.3.17"
70+
regex.workspace = true
7071

7172
# For tracing
7273
envy = "0.4.2"

engine/baml-runtime/src/internal/llm_client/mod.rs

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,10 +87,11 @@ pub enum ResolveMediaUrls {
8787

8888
// aws: supports b64 w mime
8989
// anthropic: supports b64 w mime
90-
// google: supports b64 w mime
90+
// google: supports b64 w mime, url if its a google file uri (gs://)
9191
// openai: supports URLs w/o mime (b64 data URLs also work here)
9292
// vertex: supports URLs w/ mime, b64 w/ mime
9393
Always,
94+
IfMatchesGoogleFileUri,
9495
EnsureMime,
9596
Never,
9697
}

engine/baml-runtime/src/internal/llm_client/primitive/google/googleai_client.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,7 @@ impl GoogleAIClient {
138138
chat: true,
139139
completion: false,
140140
max_one_system_prompt: true,
141-
resolve_media_urls: ResolveMediaUrls::Always,
141+
resolve_media_urls: ResolveMediaUrls::IfMatchesGoogleFileUri,
142142
allowed_metadata: properties.allowed_metadata.clone(),
143143
},
144144
retry_policy: client

engine/baml-runtime/src/internal/llm_client/traits/mod.rs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -570,6 +570,11 @@ async fn process_media(
570570
(ResolveMediaUrls::Always, _) => {}
571571
(ResolveMediaUrls::EnsureMime, Some("")) | (ResolveMediaUrls::EnsureMime, None) => {
572572
}
573+
(ResolveMediaUrls::IfMatchesGoogleFileUri, _) => {
574+
if media_url.url.starts_with("gs://") {
575+
return Ok(part.clone());
576+
}
577+
}
573578
(ResolveMediaUrls::Never, _) | (ResolveMediaUrls::EnsureMime, _) => {
574579
return Ok(part.clone());
575580
}

0 commit comments

Comments
 (0)