-
-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for Gemini via Vertex API #8
Comments
I'll be honest, I hadn't figured out the difference between AI Studio and Vertex API myself! Thanks for helping clarify. I can see a few options:
I think the |
No promises that I'm 100% correct, the docs aren't great at clarifying so I could've missed a way to get things working :) After thinking about it, I agree with you that For auth, using google's sdk is probably easiest. Usually an app would use either auth set up by the gcloud command, or a .json file pointing to service account in GCP. I couldn't find a way to use just an api key or token. |
Hey @simonw , I took a crack at this tonight and created a plugin based on https://llm.datasette.io/en/stable/plugins/tutorial-model-plugin.html: It works so far, but I left a few TODOs I wasn't sure about. Like should I register all models on an account? And should I add a prefix to the model name like edit: And also, I'm using environment variables to set the gcp project id/location. Would it be better to use |
Thanks for making llm and all of your work around it!
If I understand correctly, this plugin, llm-gemini, only works with the gemini api through Google AI Studio. But it doesn't work with the gemini models through Google Vertex AI which uses different endpoints.
It's kind of confusing because google has two python libraries that are really similar. google-cloud-aiplatform is the one that works with Vertex. Code example: https://cloud.google.com/vertex-ai/generative-ai/docs/start/quickstarts/quickstart-multimodal
Do you think llm-gemini could support using the Vertex API in addition to the aistudio api? (I'm not sure what to call the non-vertex api)
My use case is pretty simple: At work, I can access the Vertex AI API through GCP, but not AI Studio. Vertex also doesn't use your data for training purposes by default.
The text was updated successfully, but these errors were encountered: