Messages and Content

Messages and Content

Understanding the message and content system in Flutter AI SDK.

Message Structure

A Message represents a single message in a conversation:

class Message {
  final MessageRole role;        // system, user, assistant, tool
  final List<Content> content;   // The message content
  final String? name;            // Optional name identifier
}

Message Roles

enum MessageRole {
  system,     // System instructions
  user,       // User input
  assistant,  // AI response
  tool,       // Tool/function results
}

System Message

Sets the AI's behavior and personality:

final systemMessage = Message(
  role: MessageRole.system,
  content: [TextContent('You are a helpful coding assistant.')],
);

User Message

Represents user input:

final userMessage = Message(
  role: MessageRole.user,
  content: [TextContent('Explain Flutter widgets')],
);

Assistant Message

AI responses:

final assistantMessage = Message(
  role: MessageRole.assistant,
  content: [TextContent('Flutter widgets are the building blocks...')],
);

Tool Message

Results from function calls:

final toolMessage = Message(
  role: MessageRole.tool,
  content: [
    ToolResultContent(
      toolCallId: 'call_123',
      name: 'get_weather',
      result: {'temperature': 22},
    ),
  ],
);

Content Types

Messages can contain various content types:

TextContent

const textContent = TextContent('Hello, world!');

// Access text
print(textContent.text);

// Convert to JSON
print(textContent.toJson());
// {'type': 'text', 'text': 'Hello, world!'}

ImageContent

// From URL
const imageUrl = ImageContent.fromUrl(
  'https://example.com/image.jpg',
  detail: ImageDetail.high,
);

// From bytes
final imageBytes = ImageContent.fromBytes(
  bytes,
  mimeType: 'image/png',
);

// From base64
const imageBase64 = ImageContent.fromBase64(
  'iVBORw0KGgo...',
  mimeType: 'image/png',
);

AudioContent

// From bytes
final audio = AudioContent.fromBytes(
  audioBytes,
  mimeType: 'audio/mp3',
);

// From base64
const audioBase64 = AudioContent.fromBase64(
  'base64data...',
  mimeType: 'audio/wav',
);

DocumentContent

final document = DocumentContent.fromBytes(
  pdfBytes,
  mimeType: 'application/pdf',
  fileName: 'report.pdf',
);

ToolCallContent

Represents an AI's request to call a tool:

const toolCall = ToolCallContent(
  id: 'call_abc123',
  name: 'get_weather',
  arguments: {'location': 'Paris'},
);

ToolResultContent

Contains the result of a tool call:

const toolResult = ToolResultContent(
  toolCallId: 'call_abc123',
  name: 'get_weather',
  result: {'temperature': 22, 'condition': 'sunny'},
  isError: false,
);

Creating Messages

Simple Text Message

final message = Message.user('Hello, how are you?');
// or
final message = Message(
  role: MessageRole.user,
  content: [TextContent('Hello, how are you?')],
);

Multimodal Message

final message = Message(
  role: MessageRole.user,
  content: [
    const TextContent('What is in this image?'),
    const ImageContent.fromUrl('https://example.com/photo.jpg'),
  ],
);

Message with Name

final message = Message(
  role: MessageRole.user,
  content: [TextContent('Hello')],
  name: 'Alice',
);

Using Messages with FlutterAI

Simple Chat (Automatic)

// SDK creates messages automatically
final response = await ai.chat('Hello');

With Content Array

final response = await ai.chatWithContent([
  const TextContent('Analyze this:'),
  const ImageContent.fromUrl('https://example.com/chart.png'),
]);

Manual Message Management

// Add messages to context manually
ai.context.addMessage(Message(
  role: MessageRole.user,
  content: [TextContent('Previous context...')],
));

// Get messages for inspection
final messages = ai.context.messages;
for (final msg in messages) {
  print('${msg.role}: ${msg.textContent}');
}

Message Helper Methods

Get Text Content

final message = Message(
  role: MessageRole.assistant,
  content: [
    TextContent('Hello!'),
    TextContent(' How can I help?'),
  ],
);

// Get combined text
print(message.textContent); // 'Hello! How can I help?'

Check Content Types

// Check if message has images
final hasImages = message.content.any(
  (c) => c is ImageContent,
);

// Get all images
final images = message.content.whereType<ImageContent>();

// Check if it's a tool call response
final hasToolCalls = message.content.any(
  (c) => c is ToolCallContent,
);

Conversation

The Conversation class manages the full conversation:

final conversation = ai.conversation;

// Properties
conversation.messages        // All messages
conversation.allMessages     // Including system message
conversation.length          // Message count
conversation.systemPrompt    // System prompt

// Methods
conversation.addUserMessage('Hello');
conversation.addAssistantMessage('Hi there!');
conversation.clear();
conversation.toJson();

Export/Import Conversation

// Export
final json = conversation.toJson();

// Import
final imported = Conversation.fromJson(json);

ContentType Enum

enum ContentType {
  text,        // Plain text
  image,       // Image content
  audio,       // Audio content
  video,       // Video content
  document,    // Document/file
  toolCall,    // Tool/function call
  toolResult,  // Tool/function result
}

Image Detail Levels

For image content:

enum ImageDetail {
  low,   // Fast, cheaper, less detail
  high,  // Slower, accurate, more detail
  auto,  // Let the model decide
}

Best Practices

1. Use Convenience Methods

// Prefer
await ai.chat('Hello');

// Over
await ai.chatWithContent([TextContent('Hello')]);

2. Combine Content Types Thoughtfully

// Put text first for clarity
final response = await ai.chatWithContent([
  const TextContent('Compare these images:'),
  ImageContent.fromUrl(image1Url),
  const TextContent('With this one:'),
  ImageContent.fromUrl(image2Url),
]);

3. Check Response Content

final response = await ai.chat('Hello');

// Check for tool calls first
if (response.hasToolCalls) {
  // Handle tool calls
} else {
  // Use text response
  print(response.text);
}

Related

• Content Types - Full content API
• Multimodal Content - Images, audio, docs
• Function Calling - Tool system