Skip to content

For each Gemini model, restrict allowed values for candidate count to avoid runtime errors #546

@chrisraygill

Description

@chrisraygill

Describe the bug

When using Gemini 1.5 Flash and 1.5 Pro, when trying to set a candidate count larger than 1, we are greeted with the following error:

Error for candidate count

Vertex response generation failed: ClientError: [VertexAI.ClientError]: got status: 400 Bad Request. {"error":{"code":400,"message":"Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again.","status":"INVALID_ARGUMENT","details":[{"@type":"type.googleapis.com/google.rpc.DebugInfo","detail":"[ORIGINAL ERROR] generic::invalid_argument: Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again. [google.rpc.error_details_ext] { message: \"Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again.\" }"}]}}
Error: Vertex response generation failed: ClientError: [VertexAI.ClientError]: got status: 400 Bad Request. {"error":{"code":400,"message":"Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again.","status":"INVALID_ARGUMENT","details":[{"@type":"type.googleapis.com/google.rpc.DebugInfo","detail":"[ORIGINAL ERROR] generic::invalid_argument: Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again. [google.rpc.error_details_ext] { message: \"Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again.\" }"}]}}
    at /Users/nohe/Documents/git/nohe/tripedia-genkit/node_modules/@genkit-ai/vertexai/lib/gemini.js:482:17
    at Generator.throw (<anonymous>)
    at rejected (/Users/nohe/Documents/git/nohe/tripedia-genkit/node_modules/@genkit-ai/vertexai/lib/gemini.js:50:29)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Lack of support for multiple candidates is not a Genkit issue, it's a limitation of the Gemini APIs for some models at this time.

Expected behavior

However, it would be a better developer experience if we had a way to define the allowed values for number of candidates in the model definition so developers don't hit the error at runtime.

Metadata

Metadata

Assignees

Labels

featureNew feature or requestgojs

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions