Fixes #3939: vertexai gemini has a different URL, we should support that in the vertexai procedure #3947
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vga91
force-pushed
the
issue-3939
branch
2 times, most recently
from
ee93f5f
to
68e7fef
Compare
jexp
reviewed
Feb 23, 2024
|
|
||
| For example, if we define no `endpoint` config., | ||
| the default one `https://%1$s-aiplatform.googleapis.com/v1/projects/%2$s/locations/%1$s/publishers/google/models/%3$s:%4$s` will be valued, where: | ||
| - `%1$s` will be model with the `model` configuration |
There was a problem hiding this comment.
could we do this with named parameters instead, i.e.g {model} etc. will be easier for users to manage and handle.
jexp
requested changes
Feb 23, 2024
…hat in the vertexai procedure
…- updated image and assertion The previoous image was sometimes recognized as a `bagel`, and changed assertion from `tarallo` to `tarall` since the output can be both `tarallo` and `taralli`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #3939
Now the URL can be configured via the apoc.conf
apoc.ml.vertexai.url(like OpenAI procedures)Added
VertexAIHandlerabstract factory to handle the body, jsonpath and URL based on the procedureadded
apoc.ml.vertexai.stream, which calls gemini AI endpoints (which end with:streamGenerateContentinstead of:predict)added
apoc.ml.vertexai.custom, similar toapoc.ml.bedrock.custom:streamGenerateContentor:predict)updated documentation with configs and examples
added test cases with gemini-pro and gemini-pro-vision multimodal APIs: https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-multimodal-embeddings
Other considerations/changes
apoc.ml.VertexAIIT#chatCompletion, due to: class java.util.ArrayList cannot be cast to class java.util.Mapapoc.ml.VertexAIIT#completion, due to: assertEquals(true, result.containsKey("recitationResult"));Most likely because the model is changed.
Hence, I generalized the assertions, as I did in this PR, by just asserting the
textthat should return.apoc.ml.bedrock.imageone, using this API would be great. But currently, it has limited access, so I guess we can wait for public access and implement it later.I opened another issue for that: Create apoc.ml.vertexai.image leveraging vertex ai gemini #3946.