-
Notifications
You must be signed in to change notification settings - Fork 493
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
vertexai gemini has a different URL, we should support that in the vertexai procedure #3939
Comments
Perhaps in addition we should add a generic URL method like we did in bedrock that allows to use more custom new model URLs and perhaps an Image Vision procedure as well as we did for Bedrock. https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/image-generation it seems to support a GCS storage URL as optional parameter, so that the generated images are stored in GCS and an URL returned, we should support that. And also see if something similar is available for the bedrock image-gen API with S3 and add it there too. |
…hat in the vertexai procedure (#3947) * Fixes #3939: vertexai gemini has a different URL, we should support that in the vertexai procedure * updated extended.txt * changed endpoint from String.format to string replace with parameter - updated image and assertion The previoous image was sometimes recognized as a `bagel`, and changed assertion from `tarallo` to `tarall` since the output can be both `tarallo` and `taralli` * Fix compile error after #3940 merge * changed %2s to {project} in docs
see:
currently we use
:predict
as suffix which doesn't seem to work anymore with gemini, also the request body changed a bitwe can probably have a new method called apoc.ml.vertexai.stream
that also works with the new body.
gemini-pro-vision supports also image payloads to e.g. describe or summarize images, not sure if we need that yet
https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini#request_body
POST https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/publishers/google/models/gemini-pro-vision:streamGenerateContent
https://cloud.google.com/vertex-ai/docs/generative-ai/model-reference/gemini#gemini-pro-vision
We probably need that for gemini ultra anyhow.
The text was updated successfully, but these errors were encountered: