Face Recognition

[Solarflor]

Hello,
the latest version of Frigate now includes a face‑recognition feature.
Is it possible to integrate this capability into our HA automations?
Thank you

[inkeep-ai-assistant[bot]]

Yes, you can integrate Frigate's face recognition into Home Assistant automations.(1)

When face recognition is enabled, Frigate adds recognized names as a sub_label to detected person objects.(2) This information is included in the UI, filters, and notifications.(2)

The Home Assistant integration provides sensors that expose face recognition data.(3) Specifically, there are occupancy sensors that track the last recognized face.(4)

For automations, you have multiple options:

Using MQTT Topics:
You can subscribe to the frigate/events or frigate/tracked_object_update MQTT topics to trigger automations based on recognized faces.(5) These topics provide detailed information including sub-labels (the recognized face names).(5)

Using Home Assistant Sensors:
The integration creates a last_recognized_face sensor for each camera.(5) This sensor updates when a face is recognized and resets to none when no person is in frame.(4)

Important Configuration Note:
When using a default COCO model (not a Frigate+ model that detects faces), you should not add face to your list of objects to track.(2)(6)(7) Face detection will run automatically via CV2 when a person is detected.(2)

objects:
  track:
    - person

(2)

📚 Sources:

1. Home Assistant Integration
2. Face Recognition
3. Home Assistant: Recognised Car/Face event entity
4. Can we ge a recognised face and recognised plate occupancy sensors?
5. Help Needed: Home Assistant Automation with Frigate 0.16 Face Recognition
6. recognition to work [Free Plan]
7. [Config Support]: Face recognition

---

Have another question? Just tag @inkeep.

[Solarflor]

@inkeep regarding MQTT topics could you show me the right instuction to include in the config file of Frigate?
At the moment I have:

`mqtt:
  host: 192.168.1.130 #core-mosquitto
  port: 1883
  user: solarflor
  password: fpasswd
  topic_prefix: frigate/tracked_object_update`

It is enough?

[inkeep-ai-assistant[bot]]

Your MQTT configuration has an incorrect topic_prefix value.(1)

The topic_prefix should be just frigate (or another simple name), not frigate/tracked_object_update.(2)(1) The tracked_object_update part is automatically appended by Frigate when publishing messages.(2)

Here's the correct configuration:

mqtt:
  host: 192.168.1.130
  port: 1883
  user: solarflor
  password: floriano1628
  topic_prefix: frigate

(1)

With this configuration, Frigate will automatically publish to topics like frigate/tracked_object_update, frigate/events, and frigate/<camera_name>/<object_name>.(2)

The topic_prefix must be unique if you're running multiple Frigate instances.(1) You should also ensure the client_id is unique across instances (defaults to frigate).(1)(3)

Additional optional MQTT settings you might consider:

mqtt:
  enabled: True
  host: 192.168.1.130
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: solarflor
  password: floriano1628
  stats_interval: 60
  qos: 0

(1)

The stats_interval controls how often stats are published (default 60 seconds), and qos sets the Quality of Service level for MQTT messages (0 = at most once, 1 = at least once, 2 = exactly once).(1)

📚 Sources:

1. Full Reference Config
2. MQTT
3. MQTT "Unable to publish to Road/audio/dBFS: client is not connected" error

---

Have another question? Just tag @inkeep.

[Solarflor]

it seems there is something wrong in my case.

This is the Frigate config

mqtt:
  enabled: True
  host: 192.168.1.130
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: solarflor
  password: passwd
  stats_interval: 60
  qos: 0

detectors:
  coral:
    type: edgetpu
    device: usb
birdseye:
  enabled: true
  restream: false
  width: 1280
  height: 720
  quality: 5
  mode: objects
ffmpeg:
  hwaccel_args: preset-vaapi
motion:
record:
  enabled: true
  expire_interval: 60
  retain:
    days: 0
    mode: all
  alerts:
    retain:
      days: 10
    pre_capture: 3
    post_capture: 3
  detections:
    retain:
      days: 10
    pre_capture: 3
    post_capture: 3
go2rtc:
  streams:
    Ingresso:
      - rtsp://admin:passwd@192.168.1.211:554/cam/realmonitor?channel=1&subtype=2
    Cancello:
      - rtsp://admin:passwd@192.168.1.208:554/cam/realmonitor?channel=1&subtype=2
    Interno:
      - rtsp://admin:passwd@192.168.1.210:554/cam/realmonitor?channel=1&subtype=2
    Interno_2:
      - rtsp://admin:passwd@192.168.1.206:554/cam/realmonitor?channel=1&subtype=2
################################################################################################      
cameras:
  Ingresso: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: 
            rtsp://admin:passwd@192.168.1.211:554/cam/realmonitor?channel=1&subtype=0     # <----- The stream you want to use for detection
          roles:
            - record
        - path: 
            rtsp://admin:passwd@192.168.1.211:554/cam/realmonitor?channel=1&subtype=2     # <----- The stream you want to use for detection
          roles:
            - detect
    detect:
      width: 1280
      height: 720
    objects:
      track:
        - person
        - cat
        - dog
      filters:
        person:
          min_area: 6000
    record: # <----- Enable recording
      enabled: true
    snapshots: # <----- Enable snapshots
      enabled: true
    zones:
      zone_2:
        coordinates: 475,151,673,148,664,200,706,205,762,720,453,720,297,720
    review:
      alerts:
        required_zones: zone_2
      detections:
        required_zones: zone_2
  Cancello: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: 
            rtsp://admin:passwd@192.168.1.208:554/cam/realmonitor?channel=1&subtype=0     # <----- The stream you want to use for detection
          roles:
            - record
        - path: 
            rtsp://admin:passwd@192.168.1.208:554/cam/realmonitor?channel=1&subtype=2     # <----- The stream you want to use for detection
          roles:
            - detect
    detect:
      width: 1280
      height: 720
    objects:
      track:
        - person
        - cat
        - dog
      filters:
        person:
          min_area: 13000
    record: # <----- Enable recording
      enabled: true
    snapshots: # <----- Enable snapshots
      enabled: true
    zones:
      zone_3:
        coordinates: 667,210,1259,470,1259,694,304,699,107,409
    review:
      alerts:
        required_zones: zone_3
      detections:
        required_zones: zone_3
  Interno: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: 
            rtsp://admin:passwd@192.168.1.210:554/cam/realmonitor?channel=1&subtype=0     # <----- The stream you want to use for detection
          roles:
            - record
        - path: 
            rtsp://admin:passwd@192.168.1.210:554/cam/realmonitor?channel=1&subtype=2     # <----- The stream you want to use for detection
          roles:
            - detect
    detect:
      width: 1280
      height: 720
    objects:
      track:
        - person
        - cat
        - dog
    record: # <----- Enable recording
      enabled: true
    snapshots: # <----- Enable snapshots
      enabled: true
    zones:
      zone_4:
        coordinates: 1280,720,1280,456,706,0,439,0,506,720
    review:
      alerts:
        required_zones: zone_4
      detections:
        required_zones: zone_4
  Interno_2: # <------ Name the camera
    ffmpeg:
      inputs:
        - path: 
            rtsp://admin:passwd@192.168.1.206:554/cam/realmonitor?channel=1&subtype=0     # <----- The stream you want to use for detection
          roles:
            - record
        - path: 
            rtsp://admin:passwd@192.168.1.206:554/cam/realmonitor?channel=1&subtype=2    # <----- The stream you want to use for detection
          roles:
            - detect
    detect:
      width: 1280
      height: 720
    objects:
      track:
        - person
        - cat
        - dog
    record: # <----- Enable recording
      enabled: true
    snapshots: # <----- Enable snapshots
      enabled: true
    zones:
      zone_5:
        coordinates: 0,720,758,720,811,232,550,180,0,364
    motion:
      mask:
        - 1280,0,1280,720,767,720,804,201,568,191,0,359,0,0,777,0,985,0
    birdseye:
      enabled: false
#########################################################################################
    review:
      alerts:
        required_zones: zone_5
      detections:
        required_zones: zone_5
detect:
  enabled: true
version: 0.16-0
semantic_search:
  enabled: false
  model_size: small
face_recognition:
  enabled: true
  model_size: large
lpr:
  enabled: false
classification:
  bird:
    enabled: false

and this the HA automation I copied from @hawkeye217

alias: Face Recognition Notification
description: Send notification when a face is recognized by Frigate
triggers:
  - topic: frigate/tracked_object_update
    trigger: mqtt
conditions:
  - condition: template
    value_template: "{{ trigger.payload_json.type == 'face' }}"
  - condition: template
    value_template: "{{ trigger.payload_json.name != 'unknown' }}"
  - condition: template
    value_template: "{{ trigger.payload_json.score > 0.7 }}"
actions:
  - action: tts.cloud_say
    metadata: {}
    data:
      cache: false
      entity_id: media_player.cucina
      message: Attenzione un intruso nel tuo giardino

[hawkeye217]

I'd start by watching the frigate/tracked_object_update topic with something like MQTT Explorer. You can see what payload is being sent.

[Solarflor]

Not sure wht but after the restart of HA the automation is now working.
Many Thanks

[hawkeye217]

Great, glad it's working for you now.