-
Notifications
You must be signed in to change notification settings - Fork 568
Closed as not planned
Closed as not planned
Copy link
Labels
Description
Describe the bug
When using Gemini 1.5 Flash and 1.5 Pro, when trying to set a candidate count larger than 1, we are greeted with the following error:
Error for candidate count
Vertex response generation failed: ClientError: [VertexAI.ClientError]: got status: 400 Bad Request. {"error":{"code":400,"message":"Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again.","status":"INVALID_ARGUMENT","details":[{"@type":"type.googleapis.com/google.rpc.DebugInfo","detail":"[ORIGINAL ERROR] generic::invalid_argument: Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again. [google.rpc.error_details_ext] { message: \"Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again.\" }"}]}}
Error: Vertex response generation failed: ClientError: [VertexAI.ClientError]: got status: 400 Bad Request. {"error":{"code":400,"message":"Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again.","status":"INVALID_ARGUMENT","details":[{"@type":"type.googleapis.com/google.rpc.DebugInfo","detail":"[ORIGINAL ERROR] generic::invalid_argument: Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again. [google.rpc.error_details_ext] { message: \"Unable to submit request because candidateCount must be 1 but the entered value was 3. Update the candidateCount value and try again.\" }"}]}}
at /Users/nohe/Documents/git/nohe/tripedia-genkit/node_modules/@genkit-ai/vertexai/lib/gemini.js:482:17
at Generator.throw (<anonymous>)
at rejected (/Users/nohe/Documents/git/nohe/tripedia-genkit/node_modules/@genkit-ai/vertexai/lib/gemini.js:50:29)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
Lack of support for multiple candidates is not a Genkit issue, it's a limitation of the Gemini APIs for some models at this time.
Expected behavior
However, it would be a better developer experience if we had a way to define the allowed values for number of candidates in the model definition so developers don't hit the error at runtime.
Metadata
Metadata
Assignees
Labels
Type
Projects
Status
Done