Repository navigation
v0.6.0 - Multimodal Image Support
🖼️ Multimodal Image Support
Added comprehensive support for sending images alongside text to vision-capable models following the OpenAI Vision API format.
New Features
Image Loading Methods:
ImageBlock::from_url(url)- HTTP/HTTPS URLsImageBlock::from_file_path(path)- Local files with automatic base64 encoding- Supports:
.jpg,.jpeg,.png,.gif,.webp,.bmp,.svg - MIME type automatically inferred from file extension
- Supports:
ImageBlock::from_base64(data, mime)- Manual base64 with explicit MIME type
Convenience Helpers:
Message::user_with_image(text, url)- Simple text + imageMessage::user_with_image_detail(text, url, detail)- With detail level controlMessage::user_with_base64_image(text, data, mime)- From base64 data
New Types:
ImageBlock- Represents an image in a messageImageDetail- Control image processing (Low/High/Auto)OpenAIContent- Message content format (Text or Parts)OpenAIContentPart- Content part for multimodal messages
Examples:
examples/vision_example.rs- Comprehensive vision API demonstration
🛡️ Defensive Programming Enhancements
Enhanced Base64 Validation:
- Character set validation (rejects spaces, special chars)
- Length validation (must be multiple of 4)
- Padding validation (max 2 '=' characters at end)
- MIME injection prevention
- Large data warnings (>10MB)
Enhanced URL Validation:
- Control character detection
- Data URI base64 validation
- Long URL warnings (>2000 chars)
- Scheme validation (blocks javascript:, file:, etc.)
Debug Logging:
- Optional image serialization logging
- URL truncation for privacy
- Detail level logging
⚠️ Breaking Changes
ToolUseBlock and ToolResultBlock fields now private - Following Rust API Guidelines (C-STRUCT-PRIVATE):
Migration:
// Before:
tool_use.id
tool_use.name
tool_use.input
// After:
tool_use.id() // Returns &str
tool_use.name() // Returns &str
tool_use.input() // Returns &Value📦 Installation
[dependencies]
open-agent-sdk = "0.6.0"Related: Closes #2
Full Changelog: v0.5.0...v0.6.0