Home-Assistant custom component for image tag detection using Tagbox
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.ipynb_checkpoints
custom_components/image_processing
.gitignore
Development.ipynb
LICENSE
README.md
tagbox_classes.csv
tagbox_usage.png
thebeatles.jpg

README.md

Home-Assistant custom component for image classification (tag detection) using Machinebox.io Tagbox. The component adds an image_processing entity, where the state of the entity is the most likely tag in the image above a given confidence threshold. The attribute tags contains all data returned by the API, whilst the attribute matched_tags contains only tags matched with a confidence greater than a confidence threshold (in %). The event image_processing.detect_tag is fired for each tag above the confidence threshold, and the event data contains the fields name of tag, confidence of detection, and the entity_id that fired the event.

Place the custom_components folder in your configuration directory (or add its contents to an existing custom_components folder).

Add to your HA config:

image_processing:
  - platform: tagbox
    ip_address: localhost
    port: 8080
    confidence: 50
    source:
      - entity_id: camera.local_file

Configuration variables:

  • ip_address: the ip of your Tagbox instance
  • port: the port of your Tagbox instance
  • confidence: (Optional, default 80) The threshold in % above which tags appear in matched_tags and image_processing.detect_tag events are fired.
  • source: must be a camera.

Tagbox

Get/update Tagbox from Dockerhub by running:

sudo docker pull machinebox/tagbox

Run Tagbox with:

MB_KEY="INSERT-YOUR-KEY-HERE"
sudo docker run -p 8080:8080 -e "MB_KEY=$MB_KEY" machinebox/tagbox

To limit tagbox to only custom tags, add to the command -e MB_TAGBOX_ONLY_CUSTOM_TAGS=true. Tagbox will be then only return and calculate custom tags, saving some compute resources.

sudo docker run -p 8080:8080 -e "MB_KEY=$MB_KEY" -e MB_TAGBOX_ONLY_CUSTOM_TAGS=true machinebox/tagbox

You should see a message on startup pretrained tags are disabled, only custom tags will be returned.

Default tags

Tagbox is trained on https://storage.googleapis.com/openimages/web/index.html with the default tags listed in tagbox_classes.csv

Teaching tagbox

I've written a script to allow teaching tagbox from folders of images, the script is at https://github.com/robmarkcole/tagbox_python

Limiting computation

Image-classifier components process the image from a camera at a fixed period given by the scan_interval. This leads to excessive computation if the image on the camera hasn't changed (for example if you are using a local file camera to display an image captured by a motion triggered system and this doesn't change often). The default scan_interval is 10 seconds. You can override this by adding to your config scan_interval: 10000 (setting the interval to 10,000 seconds), and then call the scan service when you actually want to process a camera image. So in my setup, I use an automation to call scan when a new motion triggered image has been saved and displayed on my local file camera.

Local file camera

Note that for development I am using a file camera.

camera:
  - platform: local_file
    file_path: /images/thebeatles.jpg