Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Google and Vertex system message mapping #370

Closed
VisargD opened this issue May 20, 2024 · 0 comments · Fixed by #371
Closed

Feat: Google and Vertex system message mapping #370

VisargD opened this issue May 20, 2024 · 0 comments · Fixed by #371
Assignees
Labels
enhancement New feature or request triage

Comments

@VisargD
Copy link
Collaborator

VisargD commented May 20, 2024

Gemini models now support systemInstruction param which can be used to send system message. Gateway can map the system message to this new param.

Notes for vertex

Vertex currently fails for gemini 1.5 models when system message is sent because the current config maps system message as a user message and adds an empty model message. This should be fixed if the systemInstruction is mapping is added.

Request body to reproduce this behaviour:

"messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant named Leo"
        },
        {
            "role": "user",
            "content": "Who are you?"
        }
]

Error message:

Unable to submit request because it has an empty text parameter. Add a value to the parameter and try again. Learn more: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/gemini

Notes for google:

For google ai managed API, systemInstruction is only supported from gemini-1.5 model onwards. So older models cannot use the new param. For old models, we can keep send it as a user message like we do now. These are the models which do not support systemInstruction:

const SYSTEM_INSTRUCTION_DISABLED_MODELS = [
  'gemini-1.0-pro',
  'gemini-1.0-pro-001',
  'gemini-1.0-pro-latest',
  'gemini-1.0-pro-vision-latest',
  'gemini-pro',
  'gemini-pro-vision',
];

VertexAI Doc: https://cloud.google.com/vertex-ai/generative-ai/docs/learn/prompts/system-instructions

GoogleAI Doc: https://ai.google.dev/gemini-api/docs/system-instructions#code-generation

@VisargD VisargD added the enhancement New feature or request label May 20, 2024
@VisargD VisargD self-assigned this May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request triage
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant