AI Agents #307

pinkeshmars · 2025-03-21T12:01:14Z

Description

AI Agents

Linear ticket and magic word Fixes DEVR-807

Type of change

bolt-new-by-stackblitz · 2025-03-21T12:01:17Z

Run & review this pull request in StackBlitz Codeflow.

docs/ff-integrations/AI/ai-agents.md

MaggieThomann · 2025-03-21T14:30:12Z

docs/ff-integrations/AI/ai-agents.md

+Here, you specify the type of inputs users can send to the AI.
+
+- **Text**: Allows users to send text-based messages.
+- **Image**: Enables image input, allowing the AI to analyze photos.


Note that for the Google agent, we also accept Audio and Video.

MaggieThomann

Looks great, a few minor comments but nothing too crazy

MaggieThomann · 2025-03-21T14:31:01Z

docs/ff-integrations/AI/ai-agents.md

+
+- **Text**: Returns plain text responses.
+- **Markdown**: Allows richer formatting (headings, lists, links) if you display content as markdown. For example, An FAQ chatbot that uses formatted bullet points, and bold or italic text to highlight key info.
+- **Data Type (JSON)**: Returns structured data, which can be parsed programmatically. For example, a restaurant finder app might need structured data, e.g., `{ name: 'Pizza Palace', distance: '2.4 miles' }` to display a dynamic map.


Can we just refer to this as JSON for now? Technically it's just a JSON object. Later, we will add a 4th option (Data Type) and the JSON will be automatically parsed into the data type. But for now, it's just JSON.

MaggieThomann · 2025-03-21T14:32:10Z

docs/ff-integrations/AI/ai-agents.md

+Here, you can fine-tune how the agent generates responses.
+
+- **Temperature**: Controls how creative or random the AI’s responses can be on a scale of 0 to 1. A lower value (e.g., 0.1) makes responses more factual and consistent. A higher value (e.g., 1.0) makes responses more creative and varied (e.g., brainstorming ideas).
+- **Max Tokens**: Sets the response length limit—helpful for cost control or for ensuring short, direct replies.


I learned recently that the max tokens includes both the user's request AND the response. So maybe specify that here. I updated the tooltip in the product to: This is the maximum combined length of the request and response in tokens.

MaggieThomann · 2025-03-21T14:32:58Z

docs/ff-integrations/AI/ai-agents.md

+
+:::info
+
+After you successfully deploy the agent, any changes made to its configuration—such as modifying the system message, model, or temperature will require you to redeploy the Agent.


Maybe mention here that deploying and re-deploys are only necessary for non Google agents. For Google agents, the configuration of the agent is stored client side, so we dont need to redeploy anything.

MaggieThomann · 2025-03-21T14:34:44Z

docs/ff-integrations/AI/ai-agents.md

+- **Select Agent**: Here, you select the specific AI Agent you previously configured.
+- **Conversation ID**: The Conversation ID is a unique identifier you assign to maintain context and continuity across multiple interactions within the same conversation. Using a consistent ID (e.g., `user123_AIStylist_202503181200`) allows the AI to remember past interactions and keep conversations coherent and contextual.
+- **Text Input**: This is where you specify the user's message or input text that the AI agent will process. Typically, this input comes from a widget state (e.g., TextField).
+- **Image Input**: If your agent supports image processing, you can provide an image either from [local device](../../ff-concepts/file-handling/displaying-media.md#uploaded-file) storage or a [network URL](../../ff-concepts/file-handling/displaying-media.md#network).


Actually, it's tricky -- for non-Google agents, we only allow network URLs. If we allowed uploaded files, we'd have to pass base64 data to the cloud function and I found that they ran out of memory. The workaround for non-Google agents is that users can simply have users choose a local image, upload it to a bucket, and then pass the URL of that bucket.

But for Google agents, we allow both uploaded image data and network URLs.

sure, updated!

MaggieThomann · 2025-03-21T14:35:19Z

docs/ff-integrations/AI/ai-agents.md

+
+## Clear Chat History [Action]
+
+The **Clear Chat History** action allows you to clear the remembered context. It takes the **Conversation ID** and stops referencing the existing thread ID when you next send a message. For example, you can add this action on the refresh button inside the chat to manually reset a conversation and start a fresh one with a new context.


Perfect explanation. That's all they need to know.

MaggieThomann · 2025-03-21T14:35:50Z

docs/ff-integrations/AI/gemini.md

@@ -2,8 +2,8 @@
 slug: /integrations/gemini
 title: Gemini


We are going to deprecate the Gemini action eventually. Maybe can you add some sort of callout here redirecting users to use the new AI Agent instead?

MaggieThomann · 2025-03-21T14:37:05Z

docs/resources/projects/libraries.md

@@ -191,7 +191,8 @@ Once the library is imported, following resources are accessible for use:
 - [Assets](../../resources/projects/settings/general-settings.md#app-assets) (Note: These are not versioned) 

 :::note
-Pages and Firestore Collections are still being worked on and may come in future updates.
+- Pages and Firestore Collections are still being worked on and may come in future updates.
+- Creation of [**AI Agents**](../../ff-integrations/AI/ai-agents.md) is not yet supported in the Library project


Another note is that cloud functions are also not available in Library projects. I learned that recently :)

Co-authored-by: Maggie McRoskey <mmthomann@gmail.com>

pinkeshmars · 2025-03-24T07:18:04Z

Hi @MaggieThomann all the review comments are addressed now! :)

PoojaB26 · 2025-03-24T07:57:13Z

@pinkeshmars please rename the folder AI to "ai" to follow the naming convention.

MaggieThomann · 2025-03-24T09:20:29Z

docs/ff-integrations/AI/ai-agents.md

+#### Model Settings
+
+- **Provider**: Allows you to select the AI vendor for this agent. Currently, we support **OpenAI**, **Google,** and **Anthropic**.
+    - **OpenAI & Anthropic**: If you choose OpenAI or Anthropic, FlutterFlow will create a [Cloud Function](https://firebase.google.com/docs/functions) in Firebase to relay requests to the AI API securely. Hence, your Firebase project must be on a [Blaze](https://firebase.google.com/pricing) plan (paid) to deploy the necessary cloud function.


Can you add that for OpenAI and Anthropic, the deployed cloud function will only be accessible if the user calling it is authenticated? We do this in other cloud functions we deploy for the user too (like the Stripe payment one). Thanks!

Yes, added!

…tterflow-documentation into feature/ai-agent

AI Agents

0dd11f4

pinkeshmars requested a review from MaggieThomann March 21, 2025 12:01

github-actions bot requested a review from PoojaB26 March 21, 2025 12:01

MaggieThomann reviewed Mar 21, 2025

View reviewed changes

docs/ff-integrations/AI/ai-agents.md Outdated Show resolved Hide resolved

MaggieThomann reviewed Mar 21, 2025

View reviewed changes

MaggieThomann approved these changes Mar 21, 2025

View reviewed changes

pinkeshmars and others added 2 commits March 24, 2025 12:45

Address review comments

1b19c45

Update docs/ff-integrations/AI/ai-agents.md

c9418d4

Co-authored-by: Maggie McRoskey <mmthomann@gmail.com>

MaggieThomann reviewed Mar 24, 2025

View reviewed changes

pinkeshmars and others added 5 commits March 24, 2025 20:24

Address review comments part 2

71644f2

Merge branch 'feature/ai-agent' of https://github.com/FlutterFlow/flu…

b0f7260

…tterflow-documentation into feature/ai-agent

Renamed folder AI to ai

8408a06

Folder name correction

2237878

Merge branch 'main' into feature/ai-agent

bd3711e

pinkeshmars merged commit 1ff69b8 into main Mar 26, 2025
1 check passed


		:::info

		After you successfully deploy the agent, any changes made to its configuration—such as modifying the system message, model, or temperature will require you to redeploy the Agent.


		## Clear Chat History [Action]

		The Clear Chat History action allows you to clear the remembered context. It takes the Conversation ID and stops referencing the existing thread ID when you next send a message. For example, you can add this action on the refresh button inside the chat to manually reset a conversation and start a fresh one with a new context.

AI Agents #307

AI Agents #307

Uh oh!

Conversation

pinkeshmars commented Mar 21, 2025

Description

Type of change

Uh oh!

bolt-new-by-stackblitz bot commented Mar 21, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MaggieThomann left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pinkeshmars commented Mar 24, 2025

Uh oh!

PoojaB26 commented Mar 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!