-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Description
General Improvements
Let's make sure the models are up to date (can we list them dynamically?)
Actions improvement
Action: Chat
Uses the generateContent endpoint with conversation history.
Props:
| Prop | Type | Description | API Mapping |
|---|---|---|---|
model |
string/options/async options | The model to use. | |
prompt |
string | The user's input message for the current turn. | contents.parts.text |
history |
array | The preceding conversation history (user and model turns). | contents (previous items) |
systemMessage |
string | Optional. Instructions that guide the model's behavior throughout the conversation. | system_instruction |
temperature |
number | Optional. Controls randomness (0.0 deterministic, 1.0+ creative). Default varies by model. | generationConfig.temperature |
maxOutputTokens |
integer | Optional. Maximum number of tokens to generate in the response. | generationConfig.maxOutputTokens |
topK |
integer | Optional. Selects tokens from the top K most probable. | generationConfig.topK |
topP |
number | Optional. Selects tokens cumulatively reaching probability P. | generationConfig.topP |
safetySettings |
array | Optional. Configuration for blocking potentially harmful content. | safetySettings |
outputSchema |
object | Optional. JSON schema defining the structure of the desired output. | generationConfig.response_schema |
API Document: https://ai.google.dev/gemini-api/docs/text-generation#chat
Action: Create Embeddings
Uses the embedContent or batchEmbedContents endpoints.
Props:
| Prop | Type | Description | API Mapping |
|---|---|---|---|
content |
string | The text content to generate embeddings for. | content.parts.text (or in requests) |
model |
string | The embedding model to use (e.g., "models/embedding-001"). | Part of the endpoint URL |
taskType |
string | Optional. The intended use case (e.g., "RETRIEVAL_QUERY", "RETRIEVAL_DOCUMENT"). Affects embedding quality. | taskType |
API Document: https://ai.google.dev/gemini-api/docs/embeddings
Action: Summarize
Uses the generateContent endpoint with a specific instruction prompt.
Props:
| Prop | Type | Description | API Mapping |
|---|---|---|---|
model |
string/options/async options | The model to use. | |
content |
string | The text content to summarize (provided within the prompt). | contents.parts.text |
instruction |
string | The prompt instructing the model to summarize the content. | contents.parts.text |
maxOutputTokens |
integer | Optional. Maximum number of tokens for the generated summary. | generationConfig.maxOutputTokens |
temperature |
number | Optional. Controls creativity/factuality in the summary (0.0 more factual). | generationConfig.temperature |
API Document: https://ai.google.dev/gemini-api/docs/text-generation
Action: Translate Text
Uses the generateContent endpoint with a specific instruction prompt.
Props:
| Prop | Type | Description | API Mapping |
|---|---|---|---|
model |
string/options/async options | The model to use. | |
content |
string | The text content to translate (provided within the prompt). | contents.parts.text |
instruction |
string | The prompt instructing the model to translate the content, specifying target language (and optionally source). | contents.parts.text |
temperature |
number | Optional. Controls translation style (0.0 more literal). | generationConfig.temperature |
API Document: https://ai.google.dev/gemini-api/docs/text-generation
Action: Classify Items Into Categories
Uses the generateContent endpoint with a specific instruction prompt. Use Output Schema to produce the output in object as follow:
{"category1": ["item1", "item2"], "category2": ["item3", "item4"]}
Props:
| Prop | Type | Description | API Mapping |
|---|---|---|---|
model |
string/options/async options | The model to use. | |
items |
array | List of items to be classified (provided within the prompt). | contents.parts.text |
categories |
array | List of available categories (provided within the prompt). | contents.parts.text |
instruction |
string | The prompt describing the classification task, items, and categories. | contents.parts.text |
temperature |
number | Optional. Controls randomness in classification decisions (0.0 more deterministic). | generationConfig.temperature |
API Document: https://ai.google.dev/gemini-api/docs/text-generation
Action: Chat with Image/Video
Uses the generateContent endpoint with multimodal input (image/video + text prompt).
Props:
| Prop | Type | Description | API Mapping |
|---|---|---|---|
model |
string/options/async options | The model to use. | |
filePath |
object | File path in tmp dir to image/video |
|
prompt |
string | Instructions . | |
outputSchema |
object | Optional. JSON schema defining the structure of the desired output. | generationConfig.response_schema |
API Document: https://ai.google.dev/gemini-api/docs/vision
Action: Chat with Audio
Uses the generateContent endpoint with multimodal input (audio + text prompt).
Props:
| Prop | Type | Description | API Mapping |
|---|---|---|---|
model |
string/options/async options | The model to use. | |
filePath |
object | File path in tmp dir to the audio |
|
prompt |
string | Instructions for transcription (e.g., "Transcribe this audio."). Can specify language if needed. | contents.parts.text |
outputSchema |
object | Optional. JSON schema defining the structure of the desired output. | generationConfig.response_schema |
API Document: https://ai.google.dev/gemini-api/docs/audio
Metadata
Metadata
Assignees
Labels
Type
Projects
Status