New Classification

[chaptan]

Hi, Excellent work.
I have a question about the new classification as my wife wants to know which of the neighbours cats is at the door and whether the stray has turned up lol sigh.
i have in the config:
custom: Cats: enabled: true name: Cats threshold: 0.8 object_config: objects: - cat classification_type: sub_label
Also two class [first_cat_name], [second_cat_name] i don't know where they are (in db i suspect).

I have an automation that detects person, cats with this to determine the names:
- variables: face_name: | {% if trigger.id == "Person Detected" %} {% set sub_label = state_attr('sensor.doorbell_camera_person', 'sub_label') %} {% if sub_label and sub_label not in ['none', 'unknown', None] %} {{ sub_label }} {% else %} Unknown Person {% endif %} {% elif trigger.id == "Cat Detected" %} {% set sub_label = state_attr('sensor.doorbell_camera_cat', 'sub_label') %} {% if sub_label and sub_label not in ['none', 'unknown', None] %} {{ sub_label }} {% else %} Cat {% endif %} {% else %} {{ trigger.id }} {% endif %}

At the moment this just says 'Cat at front door'.

I noticed the new sensor 'sensor.doorbell_camera_cats_object_classification' should i use that to get cat name? the sensor most of time says 'none' with the odd 'cat name' or should it indeed change the sensor.doorbell_camera_cat 'sub_label'.

Regards
Steve

[inkeep-ai-assistant[bot]]

Your configuration looks correct. The classification model will assign sub labels to cat objects when both conditions are met: the confidence threshold (0.8) is reached and there's class consensus after at least 3 classification attempts where 60% agree on the same class label(1).

The sub label should appear in the sensor.doorbell_camera_cat sensor's sub_label attribute once assigned(1). However, based on the assignment requirements, the model needs multiple classification attempts before assigning a sub label(1).

For accessing the cat name in Home Assistant, you should use the sub_label from the MQTT frigate/events topic(2). The MQTT payload includes the sub label in the after section when an object is classified(2)(3).

Here's how to access it in your automation:

- variables:
    face_name: |
      {% if trigger.id == "Cat Detected" %}
        {% set sub_label = trigger.payload_json["after"]["sub_label"] %}
        {% if sub_label and sub_label not in ['none', 'unknown', None] %}
          {{ sub_label }}
        {% else %}
          Cat
        {% endif %}
      {% endif %}

(2), (3)

The sensor.doorbell_camera_cats_object_classification sensor showing "none" most of the time is expected behavior(1). If the consensus class is none, no assignment is made(1). This happens when the model doesn't have enough confidence or consensus across multiple frames to assign a specific cat name(1).

The classes you created ([first_cat_name], [second_cat_name]) are stored in Frigate's database and managed through the Classification page in the Frigate UI(1).

📚 Sources:

1. Object Classification
2. MQTT
3. Output object with sub_label for HA

---

Have another question? Just tag @inkeep.

[chaptan]

Ok changed to the above. But 'typical' no cats since lol I will report back.

[hawkeye217]

The AI is hallucinating here and is incorrect.

I would just use the new sensor directly in your automation. We created those sensors for that particular purpose.

[chaptan]

@hawkeye217 thanks josh, trying now. just need a cat to turn up lol

[chaptan]

yep working.
"The AI is hallucinating" is the ai on here something to do with google? last night i was served with this on youtube https://youtu.be/l6ZcFa8pybE?si=301AJIfgS_JMEC-X very interesting (what i understood that is lol confabulation = hallucinating). didn't get the elastic band bit lol

Just seems strange i was served with it, thats all.