Context Management

Context Management

The SDK provides automatic conversation context management to maintain coherent multi-turn conversations.

Overview

The ContextManager handles:

• Message history management
• Token counting and limits
• Context window management
• Automatic truncation strategies

Basic Usage

Context is managed automatically:

final ai = FlutterAI(
  provider: AIProvider.openai,
  config: AIConfig(
    apiKey: 'your-key',
    systemPrompt: 'You are a helpful assistant.',
  ),
);

// Context is automatically maintained
await ai.chat('My name is Alice.');
await ai.chat('I live in Paris.');
await ai.chat('What do you know about me?');
// AI will remember: Alice, lives in Paris

Custom Context Manager

For more control, create a custom ContextManager:

final contextManager = ContextManager(
  maxTokens: 8000,           // Maximum context window
  reservedTokens: 1000,      // Reserve for response
  systemPrompt: 'You are a coding assistant.',
  windowStrategy: WindowStrategy.slidingWindow,
);

final ai = FlutterAI(
  provider: AIProvider.openai,
  config: AIConfig(apiKey: 'your-key'),
  contextManager: contextManager,
);

ContextManager Options

Parameter	Type	Default	Description
maxTokens	int	8000	Maximum tokens for context
reservedTokens	int	1000	Tokens reserved for response
systemPrompt	String?	null	System instructions
windowStrategy	WindowStrategy	slidingWindow	Truncation strategy

Window Strategies

When the context exceeds the token limit, the SDK uses a strategy to manage it:

enum WindowStrategy {
  slidingWindow,  // Remove oldest messages first
  summarize,      // Summarize old messages (if supported)
  truncate,       // Hard truncate at limit
}

Sliding Window (Default)

Removes the oldest messages to make room:

final manager = ContextManager(
  maxTokens: 4000,
  windowStrategy: WindowStrategy.slidingWindow,
);

Accessing Context

Get Current Conversation

// Access the conversation
final conversation = ai.conversation;

// Get message count
print('Messages: ${conversation.length}');

// Get all messages
for (final message in conversation.messages) {
  print('${message.role}: ${message.textContent}');
}

Check Token Usage

final context = ai.context;

print('Estimated tokens: ${context.estimatedTokens}');
print('Available tokens: ${context.availableTokens}');
print('Max tokens: ${context.maxTokens}');

Managing Context

Clear Context

// Clear all messages, keep system prompt
ai.clearContext();

Export Conversation

// Export to JSON
final json = ai.conversation.toJson();
saveToFile(json);

// Import from JSON
final imported = Conversation.fromJson(json);

Add Messages Manually

// Access context manager
final context = ai.context;

// Add messages
context.addUserMessage('Hello');
context.addAssistantMessage('Hi there!');

// Add tool results
context.addToolResult(
  toolCallId: 'call_123',
  name: 'get_weather',
  result: {'temperature': 22},
);

Context Updates

Listen to context changes:

final contextManager = ContextManager(
  maxTokens: 4000,
  systemPrompt: 'You are helpful.',
);

// Listen to updates
contextManager.updates.listen((update) {
  print('Update type: ${update.type}');
  print('Message count: ${update.messageCount}');
  print('Tokens: ${update.estimatedTokens}');
});

final ai = FlutterAI(
  provider: AIProvider.openai,
  config: AIConfig(apiKey: 'your-key'),
  contextManager: contextManager,
);

Update Types

enum ContextUpdateType {
  messageAdded,
  messageRemoved,
  contextCleared,
  contextTruncated,
}

Without Context

For stateless requests, disable context:

// Single message, no context
final response = await ai.chat(
  'What is 2+2?',
  addToContext: false,
);

Conversation Model

The Conversation class holds the message history:

final conversation = ai.conversation;

// Properties
conversation.messages       // All user/assistant messages
conversation.allMessages    // Including system message
conversation.length         // Message count
conversation.systemPrompt   // System prompt if set

// Methods
conversation.toJson()       // Export to JSON
conversation.clear()        // Clear messages

Best Practices

1. Set Appropriate Limits

// For GPT-4-turbo (128K context)
final context = ContextManager(
  maxTokens: 100000,      // Leave room
  reservedTokens: 4000,   // For response
);

// For Claude (200K context)
final context = ContextManager(
  maxTokens: 180000,
  reservedTokens: 4000,
);

2. Use System Prompts

final context = ContextManager(
  systemPrompt: '''
You are a helpful coding assistant.
- Always provide working code examples
- Explain your reasoning
- Ask clarifying questions if needed
''',
);

3. Monitor Token Usage

// Before making requests
if (ai.context.availableTokens < 500) {
  print('Warning: Context almost full');
  ai.clearContext();
}

4. Handle Long Conversations

// Periodic cleanup
if (ai.conversation.length > 50) {
  // Export for history
  final history = ai.conversation.toJson();
  await saveHistory(history);
  
  // Clear and summarize
  final summary = await ai.chat(
    'Summarize our conversation so far',
    addToContext: false,
  );
  
  ai.clearContext();
  ai.context.addUserMessage('Previous context: ${summary.text}');
}

Complete Example

Future<void> contextExample() async {
  // Create custom context manager
  final contextManager = ContextManager(
    maxTokens: 8000,
    reservedTokens: 1000,
    systemPrompt: 'You are a helpful assistant that remembers everything.',
    windowStrategy: WindowStrategy.slidingWindow,
  );

  // Listen to updates
  contextManager.updates.listen((update) {
    print('Context: ${update.messageCount} messages, '
          '${update.estimatedTokens} tokens');
  });

  final ai = FlutterAI(
    provider: AIProvider.openai,
    config: AIConfig(
      apiKey: 'your-key',
      model: 'gpt-4-turbo',
    ),
    contextManager: contextManager,
  );

  try {
    // Multi-turn conversation
    await ai.chat('My name is Alice and I am a software developer.');
    await ai.chat('I specialize in Flutter and Dart.');
    await ai.chat('My favorite framework feature is hot reload.');
    
    // AI should remember everything
    final response = await ai.chat('Summarize what you know about me.');
    print(response.text);
    
    // Check context state
    print('\nContext state:');
    print('  Messages: ${ai.conversation.length}');
    print('  Tokens: ${ai.context.estimatedTokens}');
    
  } finally {
    ai.dispose();
    contextManager.dispose();
  }
}

Related

• Messages and Content - Message types
• Token Management - Token counting
• API-ContextManager - Full API reference