Skip to content

## Version 1.2.2 - Camera Snapshot & AI Task Integration Fixed

Choose a tag to compare

View all tags
@brouwerict brouwerict released this 27 Aug 19:32
· 10 commits to main since this release
v1.2.2
afefb22

OpenRouter Fixed - Release Notes

Version 1.2.2 - Camera Snapshot & AI Task Integration Fixed

This release provides a fully functional OpenRouter integration for Home Assistant with comprehensive AI Task support and camera snapshot analysis capabilities. This custom component resolves multiple compatibility issues present in the original Home Assistant OpenRouter integration.

🎯 Primary Purpose

This integration enables Home Assistant users to:

  • Use OpenRouter AI models for AI Task entity operations
  • Analyze camera snapshots with vision-capable models (Claude, GPT-4V, Qwen-VL)
  • Process structured JSON responses for automation workflows
  • Integrate with Home Assistant's native AI Task service

🔧 Key Fixes & Improvements

1. OpenAI Library Compatibility (Critical Fix)

Problem: Original integration fails with different OpenAI library versions due to import incompatibilities.

Solution: Added comprehensive import compatibility layer across multiple files:

# Enhanced compatibility in entity.py
try:
    from openai.types.chat import ChatCompletionFunctionToolParam
    from openai.types.shared_params import ResponseFormatJSONSchema
    from openai.types.shared_params.response_format_json_schema import JSONSchema
except ImportError:
    # Fallback implementations with TypedDict
    ChatCompletionFunctionToolParam = Dict[str, Any]
    # ... additional fallbacks

Files Modified: entity.py, ai_task.py, __init__.py

2. AI Task Entity Support (New Feature)

Problem: Original integration lacks AI Task entity support, preventing use with Home Assistant's AI Task service.

Solution: Complete AI Task entity implementation:

  • Created ai_task.py with full AITaskEntity support
  • Added attachment processing for camera snapshots
  • Implemented structured JSON response handling
  • Added support for ai_task.generate_data service calls

Files Added: ai_task.py (entirely new)
Files Modified: manifest.json, __init__.py, strings.json

3. Camera Snapshot & Attachment Processing (Major Feature)

Problem: No support for processing camera images or file attachments with AI models.

Solution: Comprehensive attachment handling system:

# Multi-format attachment support
content_attrs = ['content', 'data', 'binary_data', 'bytes', 'file_content', 'image_data', 'payload']
for attr in content_attrs:
    if hasattr(attachment, attr):
        image_content = getattr(attachment, attr)
        break

# File path fallback support
if hasattr(attachment, 'file_path') and attachment.file_path:
    with open(attachment.file_path, 'rb') as f:
        image_content = f.read()

Supported Attachment Sources:

  • Direct camera streams (media-source://camera/camera_name)
  • File paths (/config/www/snapshot.jpg)
  • Base64 encoded images
  • Binary data objects

4. Model Availability & Selection (Enhancement)

Problem: Limited model selection and filtering preventing access to vision-capable models.

Solution: Removed restrictive model filtering:

# config_flow.py - Removed structured output requirement
# OLD: Only models with structured output support
# NEW: All available OpenRouter models accessible

Result: Access to 200+ OpenRouter models including:

  • Claude 3.5 Sonnet, Claude 3 Haiku
  • GPT-4o, GPT-4o-mini
  • Qwen2.5-VL-72B (free vision model)
  • Gemini Pro Vision
  • And many more...

5. JSON Response Processing (Reliability Fix)

Problem: Inconsistent JSON parsing causing automation failures.

Solution: Robust JSON handling with graceful fallbacks:

try:
    data = json_loads(text)
    _LOGGER.debug("Successfully parsed structured response")
except JSONDecodeError as err:
    _LOGGER.error("Failed to parse JSON response: %s", text[:500])
    # Return raw text instead of crashing
    return ai_task.GenDataTaskResult(
        conversation_id=chat_log.conversation_id,
        data=text,
    )

6. UI Translations & Strings (Bug Fix)

Problem: Missing UI strings causing empty button labels in configuration.

Solution: Complete translation support:

Files Added:

  • strings.json - English UI strings
  • translations/en.json - Translation mappings

Fixed UI Elements:

  • "Add AI Task" button labels
  • Configuration descriptions
  • Error messages

📋 Complete File Modifications

Files Modified from Original

File Changes Purpose
manifest.json Domain name, version, flexible OpenAI requirements HACS compatibility
__init__.py AI Task platform loading, error handling Service integration
entity.py OpenAI imports, attachment processing, vision support Core functionality
config_flow.py Model filtering removal, enhanced model loading Model selection

Files Added (New)

File Purpose
ai_task.py Complete AI Task entity implementation
strings.json UI text definitions
translations/en.json English translations

Configuration Files

File Purpose
vehicle_detection_WORKING.yaml Example automation with camera snapshot
vehicle_check_OPTIMIZED.yaml Optimized vehicle detection workflow
dev_tools_CORRECT.yaml Developer Tools service call examples

🚀 Usage Examples

Basic AI Task Service Call

service: ai_task.generate_data
data:
  entity_id: ai_task.anthropic_claude_3_haiku
  task_name: "Image Analysis"
  instructions: "Analyze this image and return JSON with vehicle count"
  attachments:
    media_content_id: media-source://camera/front_door
    media_content_type: image/jpeg
response_variable: result

Camera Snapshot Analysis

# Complete automation example included in release
alias: "Vehicle Detection with Camera"
trigger:
  - trigger: state
    entity_id: binary_sensor.motion_detected
action:
  - service: ai_task.generate_data
    data:
      entity_id: ai_task.qwen_qwen2_5_vl_72b_instruct  # Free model!
      task_name: "Vehicle Detection"
      instructions: |
        Count vehicles in this driveway image.
        Return JSON: {"cars": 0, "motorcycles": 0}
      attachments:
        media_content_id: media-source://camera/driveway
        media_content_type: image/jpeg
    response_variable: vehicle_data

🔄 Migration from Original Integration

Automatic Migration

  • Configuration entries are preserved
  • Existing conversations continue to work
  • Model selections remain intact

New Capabilities Available

  • AI Task entities appear in entity registry
  • Camera snapshot analysis in automations
  • Structured JSON responses
  • Enhanced model selection

🧪 Tested Configurations

Models Tested

  • ✅ Claude 3 Haiku (fast, cost-effective)
  • ✅ Claude 3.5 Sonnet (high accuracy)
  • ✅ GPT-4o-mini (OpenAI vision)
  • ✅ Qwen2.5-VL-72B (free, excellent performance)

Attachment Types Tested

  • ✅ Live camera streams
  • ✅ Static image files
  • ✅ Home Assistant camera snapshots
  • ✅ Media source URLs

Integration Points Tested

  • ✅ Developer Tools > Services
  • ✅ Automation YAML workflows
  • ✅ Template sensors
  • ✅ Script integrations

🔍 Technical Deep Dive

Core Architecture Changes

  1. Modular Import System: Handles OpenAI library version differences gracefully
  2. Attachment Pipeline: Processes multiple input formats with fallback mechanisms
  3. AI Task Bridge: Connects OpenRouter models to Home Assistant AI Task framework
  4. Error Recovery: Comprehensive error handling prevents integration crashes

Performance Optimizations

  1. Lazy Loading: Models load on-demand to reduce startup time
  2. Caching: Configuration and model data cached for faster access
  3. Async Processing: All AI calls use async/await patterns
  4. Memory Management: Efficient image processing with cleanup

📊 Cost Optimization

Model Cost Comparison (per 1000 calls)

  • Claude 3 Haiku: ~$0.61
  • GPT-4o-mini: ~$0.42
  • Qwen2.5-VL-72B: $0.00 ⭐ (Free!)

Recommended Setup

For production use with camera analysis:

entity_id: ai_task.qwen_qwen2_5_vl_72b_instruct
# 72B parameter model with vision capabilities - completely free!

🛠️ Installation & Setup

HACS Installation

  1. Add custom repository: https://github.com/brouwerict/open_router_fixed
  2. Install via HACS
  3. Restart Home Assistant
  4. Configure via Settings > Integrations

Manual Installation

  1. Copy custom_components/openrouter_fixed/ to your HA config
  2. Restart Home Assistant
  3. Add integration via UI

Configuration

  1. Add OpenRouter API key
  2. Select AI models for conversation and/or AI Task
  3. Configure camera entities (if using vision features)

🆕 What's New in This Release

Version 1.2.2

  • ✅ Complete AI Task entity support
  • ✅ Camera snapshot processing
  • ✅ Qwen2.5-VL model support (free tier)
  • ✅ Enhanced error handling
  • ✅ Performance optimizations

Breaking Changes

  • None - fully backward compatible

Deprecations

  • None - all existing functionality preserved

🤝 Contributing

Found an issue or want to contribute?

📄 License

Same as Home Assistant core: Apache 2.0

This integration transforms OpenRouter from a simple conversation tool into a powerful AI Task processor with vision capabilities, enabling advanced home automation scenarios with camera analysis and structured data processing.

Assets 2
Loading