Image and video analyzer for Home Assistant using multimodal LLMs

🌟 Features · 📖 Resources · ⬇️ Installation · 🚧 Roadmap · 🪲 How to report Bugs · ☕ Support

LLM Vision is a Home Assistant integration to analyze images, videos, live camera feeds and frigate events using the vision capabilities of multimodal LLMs. Supported providers are OpenAI, Anthropic, Google Gemini, Groq, LocalAI, Ollama and any OpenAI compatible API.

• Compatible with OpenAI, Anthropic Claude, Google Gemini, Groq, LocalAI, Ollama and custom OpenAI compatible APIs
• Analyzes images and video files
• Analyzes live camera feeds and frigate events
• Remembers Frigate events and camera motion events
• Images can be downscaled for faster processing

With the easy to use blueprint, you'll get important notifications intelligently summarized by AI from either frigate or cameras in Home Assistant. LLM Vision can also remember events, so you can ask about them later. LLM Vision needs to be installed to use the blueprint.

Check the docs for detailed instructions on how to set up LLM Vision and each of the supported providers, get inspiration from examples or join the discussion on the Home Assistant Community.

1. Search for LLM Vision in Home Assistant Settings/Devices & services
2. Select your provider
3. Follow the instructions to add your AI providers.

Detailed instruction on how to set up LLM Vision and each of the supported providers are available here: https://llm-vision.gitbook.io/getting-started/

To enable debugging, add the following to your configuration.yaml:

logger:
  logs:
    custom_components.llmvision: debug

1. New Service: data_analyzer to read data from charts and update a sensor
2. HACS: Include in HACS default repository

For features added in previous versions, check the changelogs in the release notes.

You can support this project by starring this GitHub repository. If you want, you can also buy me a coffee here: