Prompting with videos return 403 error

[kadzaki]

Prompting with videos return the following error, while it works with classic Gemini API:
HTTP error downloading media 'https://generativelanguage.googleapis.com/v1beta/files/uu320af6jeh8': {
"error": {
"code": 403,
"message": "Method doesn't allow unregistered callers (callers without established identity). Please use API Key or other form of API consumer identity to call this API.",
"status": "PERMISSION_DENIED"
}
}

With Genkit (not working):
const response = await generate({
model: gemini15Pro,
prompt: [
{ text: prompt },
{
media: {
url: uploadResponse.file.uri,
contentType: uploadResponse.file.mimeType,
},
},
]
});

With classic Gemini API (works):
const result = await model.generateContent([
{
fileData: {
mimeType: uploadResponse.file.mimeType,
fileUri: uploadResponse.file.uri
}
},
{ text: "Summarize this video." },
]);

[mbleigh]

I suspect this is because Genkit attempts to download https URLs and inline them as base64-encoded data for Gemini APIs since Gemini hasn't traditionally supports https URLs directly.

We may need to special case the File API URLs.

Can you show me the code that you are running that gave you uploadResponse?

[kadzaki]

const uploadResponse = await fileManager.uploadFile("test-video.mp4", {
mimeType: "video/mp4",
displayName: "Test video",
});
....
const response = await generate({
model: gemini15Pro,
prompt: [
{ text: prompt },
{
media: {
url: uploadResponse.file.uri,
contentType: uploadResponse.file.mimeType,
},
},
]
});

[mbleigh]

This should be fixed by #742 and will be included in the next release after it's merged!