Cam adjustments to get better face detection support - Quality / framerate etc ?

[vw-kombi]

Describe the problem you are having

I have dahua cam as my front door cam. Its substream that I was using for detect only goes up to 640480. I do have control over CBR/VBR, Frame Rate, But Rate and I Frame Rate. I have seen only vide (and google AI) saying that the usual 5/6 FPS may nat be fast enough for face detection, and also that 640480 may not be a high enough resolution. I have modded my config to use detect and record from the same main stream instead and lowered the quiality a bit, and the frame rate.

    - path: 
        rtsp://dahua:pwd@10.10.40.15/cam/realmonitor?channel=1&subtype=0     # <----- The stream you want to use for recording
      input_args: preset-rtsp-restream
      roles:
        - record
        - detect
detect:
  width: 1920
  height: 1080
  fps: 15

Or should I get a new cam like my hikvision on the frnt that can have a better substream.

Version

0.16.2-4d58206

Frigate config file

# yaml-language-server: $schema=http://192.168.30.249:5005/api/config/schema.json

# Include all cameras by default in Birdseye view
birdseye:
  enabled: true
  mode: continuous
  width: 1280
  height: 720

ui:
  # Optional: Set a timezone to use in the UI (default: use browser local time)
  timezone: Australia/Brisbane

database:
  path: /config/frigate.db

mqtt:
  host: 10.10.10.12
  topic_prefix: frigate
  user: MQTT
  password: MQTTMQTT1!

ffmpeg:
  hwaccel_args: preset-vaapi

detectors:
  ov:
    type: openvino
    device: GPU

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  path: /openvino-model/ssdlite_mobilenet_v2.xml
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

semantic_search:
  enabled: true
  model_size: large
  reindex: false

genai:
  enabled: true
  provider: gemini
  api_key: key
  model: gemini-2.5-flash

cameras:
  Front-Cam:
    enabled: true
    ui:
      order: 1
    genai: # <- Custom AI Prompts for objects can go here
      enabled: true
      use_snapshot: true
      prompt: Summarize what's happening in the camera feed (one sentence max). 
        Don't describe the scene! If there is a person, describe what they're 
        doing and what they look like. If they look like a courier mention that!
      objects:
        - person
        - dog
        - motorcycle
      required_zones:
        - front-pathway
        - Postbox
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: 
            rtsp://hikvision:pwd@10.10.40.18:554/Streaming/Channels/102     # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: 
            rtsp://hikvision:pwd@10.10.40.18:554/Streaming/Channels/101     # <----- The stream you want to use for recording
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      enabled: true # <---- disable detection until you have a working camera feed
      width: 1280
      height: 720
      fps: 10
    objects:
      track:
        - person
        - dog
        - motorcycle
      filters:
        motorcycle:
          min_score: 0.50
        person:
          min_score: 0.50
        bicycle:
          min_score: 0.50
    record:
      enabled: true
      retain:
        days: 7
        mode: motion
      alerts:
        retain:
          days: 7
      detections:
        retain:
          days: 7
    snapshots:
      enabled: true
      bounding_box: true
      timestamp: true
      retain:
        default: 7
    motion:
      mask:
        - 0.414,1,0.414,0.939,0.284,0.933,0.174,0.866,0.14,0.946,0,0.936,0.004,0.997
        - 0.707,0,0.604,0,0.849,0.072,0.907,0.085,0.906,0.136,0.918,0.235,0.875,0.408,0.949,0.921,0.999,0.8,1,0.002
        - 0.155,0.043,0.156,0,0.14,0,0.11,0.004,0,0.002,0,0.377,0.112,0.171,0.11,0.087
        - 0.431,0,0.607,0,0.565,0.027,0.555,0.132,0.432,0.116
      threshold: 30
      contour_area: 10
      improve_contrast: true
    zones:
      driveway:
        coordinates: 
          0.262,0.187,0.407,0.242,0.507,0.278,0.721,1,0.409,0.998,0.4,0.935,0.289,0.924,0.173,0.867,0.14,0.94,0.064,0.939,0,0.773,0,0.38,0.118,0.168
        loitering_time: 0
        objects:
          - dog
          - motorcycle
          - person
        inertia: 3
      front-right:
        coordinates: 
          0.623,0,0.818,0.065,0.83,0.13,0.822,0.208,0.808,0.371,0.456,0.245,0.477,0.125,0.554,0.135,0.569,0.027
        loitering_time: 5
        objects:
          - person
          - dog
          - motorcycle
        inertia: 3
      front-left:
        coordinates: 
          0.258,0,0.156,0,0.159,0.041,0.111,0.09,0.114,0.162,0.171,0.17,0.243,0.176,0.455,0.25,0.477,0.123,0.431,0.113,0.43,0
        loitering_time: 6
        objects:
          - dog
          - person
          - motorcycle
        inertia: 3
      front-pathway:
        coordinates: 0.519,0.309,0.878,0.438,0.946,0.92,0.901,0.996,0.724,1
        inertia: 3
        loitering_time: 0
        objects:
          - dog
          - person
      Postbox:
        coordinates: 
          0.822,0.067,0.906,0.092,0.905,0.151,0.916,0.232,0.869,0.433,0.807,0.412,0.832,0.13
        inertia: 3
        loitering_time: 2
        objects:
          - person
          - dog
          - motorcycle
    review:
      alerts:
        required_zones:
          - driveway
          - front-pathway
          - front-left
          - front-right
          - Postbox
      detections:
        required_zones:
          - driveway
          - front-pathway
          - front-left
          - front-right
          - Postbox

################################################################################################################################################

  Front-Door:
    enabled: true
    ui:
      order: 2
    genai: # <- Custom AI Prompts for objects can go here
      enabled: true
      prompt: Summarize what's happening in the camera feed (one sentence max). 
        Don't describe the scene! If there is a person or a dog, describe what 
        they're doing and what they look like. If they look like a courier 
        mention that! Ignore the tortoise.
#      object_prompts:
#         person: "Observe the person in these images, describe what they're doing and what they look like (one sentence max). Don't describe the scene! If they look like a courier mention that! If nothing is happening, say so."
#         cat: "Observe the cat in these images. Focus on its movement and intent (e.g., wandering, hunting, interacting with objects). If the cat is near the flower pots or engaging in any specific actions, mention it."
      objects:
        - person
      required_zones:
        - doorway
        - pathway
    lpr:
      enabled: false
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
#        - path: 
#            rtsp://dahua:pwd@10.10.40.15/cam/realmonitor?channel=1&subtype=1     # <----- The stream you want to use for detection
#          input_args: preset-rtsp-restream
#          roles:
#            - detect
        - path: 
            rtsp://dahua:pwd@10.10.40.15/cam/realmonitor?channel=1&subtype=0     # <----- The stream you want to use for recording
          input_args: preset-rtsp-restream
          roles:
            - record
            - detect
    detect:
      width: 1920
      height: 1080
      fps: 15
    objects:
      track:
        - person
      filters:
        person:
          min_score: 0.40
    record:
      enabled: true
      retain:
        days: 7
        mode: motion
      alerts:
        retain:
          days: 7
      detections:
        retain:
          days: 7
    snapshots:
      enabled: true
      required_zones:
        - doorway
        - pathway
      bounding_box: true
      timestamp: true
      retain:
        default: 7
    motion:
      threshold: 30
      contour_area: 10
      lightning_threshold: 0.8
      improve_contrast: true
      mask: 
        0.679,0,1,0,0.997,0.994,0.593,1,0.569,0.686,0.539,0.386,0.517,0.3,0.463,0.296,0.463,0
    zones:
      pathway:
        coordinates: 0.519,0.305,0.537,0.397,0.543,0.422,0.455,0.418,0.464,0.302
        inertia: 3
        loitering_time: 0
        objects: person
      doorway:
        coordinates: 
          0.54,0.424,0.591,1,0.301,1,0.283,0,0.462,0,0.464,0.317,0.46,0.422
        loitering_time: 0
        objects: person
        inertia: 3
    review:
      alerts:
        required_zones:
          - pathway
          - doorway
      detections:
        required_zones:
          - pathway
          - doorway

#################################################################################################################################################

  Loki-Cam:
    enabled: true
    ui:
      order: 5
    genai: # <- Custom AI Prompts for objects can go here
      enabled: false
      prompt: Summarize what's happening in the camera feed (one sentence max). 
        Don't describe the scene! If there is a person, describe what they're 
        doing and what they look like. If they look like a courier mention that!
        If nothing is happening, say so. Ignore the tortoise.
    lpr:
      enabled: false
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://foscam:pwd@10.10.40.16:88/videoSub # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://foscam:pwd@10.10.40.16:88/videoMain # <----- The stream you want to use for recording
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      width: 640
      height: 480
      fps: 10
    objects:
      track:
        - dog
      filters:
        dog:
          min_score: 0.55
    record:
      enabled: true
      retain:
        days: 1
        mode: motion
      alerts:
        retain:
          days: 1
      detections:
        retain:
          days: 1
    snapshots:
      enabled: true
      bounding_box: true
      timestamp: true
      retain:
        default: 1
    motion:
      threshold: 30
      contour_area: 10
      lightning_threshold: 0.8
      improve_contrast: 'true'
      mask: 0.002,0.001,0.36,0.001,0.36,0.047,-0.001,0.048
    zones:
      Loki-Zone:
        coordinates: 0,0.441,1,0.424,1,1,0,1
        loitering_time: 0
        inertia: 3
        objects: dog
    review:
      alerts:
        required_zones: Loki-Zone
      detections:
        required_zones: Loki-Zone

###################################################################################################################################################

  Kitchen-Cam:
    enabled: false
    ui:
      order: 4
    genai: # <- Custom AI Prompts for objects can go here
      enabled: false
      prompt: Summarize what's happening in the camera feed (one sentence max). 
        Don't describe the scene! If there is a person, describe what they're 
        doing and what they look like. If they look like a courier mention that!
        If nothing is happening, say so. Ignore the tortoise.
    lpr:
      enabled: false
    ffmpeg:
      output_args:
        record: preset-record-generic
      inputs:
        - path: rtsp://10.10.10.13:8554/kitchen/main
          input_args: preset-rtsp-udp
          roles:
            - detect
            - record
#        - path: rtsp://10.10.10.9:8554/kitchen/Sub # <----- The stream you want to use for recording
#          input_args: preset-rtsp-generic
#          roles:
#            - record
    detect:
      enabled: true
      width: 704
      height: 480
      fps: 4
    objects:
      track:
        - person
        - dog
      filters:
        person:
          min_score: 0.65
    record:
      enabled: true
      retain:
        days: 1
        mode: motion
      alerts:
        retain:
          days: 1
      detections:
        retain:
          days: 1
    snapshots:
      enabled: true
      bounding_box: true
      timestamp: true
      retain:
        default: 1
    motion:
      threshold: 30
      contour_area: 20
      lightning_threshold: 0.8
      improve_contrast: true
      mask: 
        0,0,1,0,1,1,0.846,1,0.704,0.534,0.305,0.509,0.375,0.327,0.235,0.326,0,0.662
    zones:
      Lounge-Kitchen:
        coordinates: 
          0.84,1,0,1,0,0.674,0.234,0.329,0.372,0.328,0.305,0.513,0.698,0.537
        loitering_time: 5
        objects:
          - person
          - dog
        inertia: 3
    review:
      alerts:
        required_zones: Lounge-Kitchen
      detections:
        required_zones: Lounge-Kitchen

###################################################################################################################################################

  Back-Cam:
    enabled: true
    ui:
      order: 3
    genai: # <- disable GenAI for your indoor camera
      enabled: false
    lpr:
      enabled: false
    ffmpeg:
      inputs:
        - path: rtsp://foscam:pwd@10.10.40.14:554/videoSub # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - detect
        - path: rtsp://foscam:pwd@10.10.40.14:554/videoMain # <----- The stream you want to use for detection
          input_args: preset-rtsp-restream
          roles:
            - record
    detect:
      width: 640
      height: 360
      fps: 10
    objects:
      track:
        - person
      filters:
        person:
          min_score: 0.70
    record:
      enabled: true
      retain:
        days: 7
        mode: motion
      alerts:
        retain:
          days: 7
      detections:
        retain:
          days: 7
    snapshots:
      enabled: true
      bounding_box: true
      timestamp: true
      retain:
        default: 7
    motion:
      mask: 
        0.998,0.005,0,0.004,0,0.436,0,1,0.297,1,0.315,0.899,0.24,0.765,0.338,0.707,0.335,0.451,0.62,0.447,0.664,0.436,1,0.439
      threshold: 40
      contour_area: 15
      improve_contrast: true
    zones:
      Backyard-Seating:
        coordinates: 
          0.622,0.455,0.672,0.44,1,0.449,1,1,0.299,1,0.317,0.896,0.24,0.765,0.339,0.713,0.338,0.446
        loitering_time: 3
        objects: person
        inertia: 3
    review:
      alerts:
        required_zones: Backyard-Seating
      detections:
        required_zones: Backyard-Seating
version: 0.16-0
detect:
  enabled: true
face_recognition:
  enabled: true
  model_size: large
  unknown_score: 0.6
  recognition_threshold: 0.8
lpr:
  enabled: false
classification:
  bird:
    enabled: false

Relevant Frigate log output

n/a

Relevant go2rtc log output

n/A

Frigate stats

n/A

Operating system

UNRAID

Install method

Home Assistant Add-on

docker-compose file or Docker CLI command

n/a

Object Detector

OpenVino

Screenshots of the Frigate UI's System metrics pages

No response

Any other information that may be helpful

No response

[inkeep-ai-assistant[bot]]

For best face detection performance in Frigate, the detect stream should have a resolution high enough that faces are large and clear in the frame, and a frame rate fast enough to avoid motion blur or missing fast movements(1)(2). The recommended detect stream frame rate is 5 fps, but for fast-moving objects (like faces quickly passing by), you may benefit from 10 fps or higher(1). Your current setting of 1920x1080 at 15 fps for detection is within recommended parameters and should offer strong results, though higher frame rates do increase CPU/GPU load(1).

640x480 is likely too low for consistent face recognition unless people are very close to the camera, as Frigate's face recognition depends on the detect stream's ability to clearly capture facial details(2). Using the main stream for both record and detect (at 1080p or higher) is generally a good solution if your hardware can handle it(1)(3)(2).

Additional camera settings for best results:

• Use CBR (constant bit rate) for image clarity(2).
• Set the I-frame interval to match the frame rate (e.g., 15 fps with I-frame interval 15)(1)(4).
• Use H.264 video and AAC audio for best compatibility(1)(4).

You do not necessarily need to buy a new camera if your current main stream can provide a high-resolution, high-quality detect stream and your hardware can process it(1)(2). If you want to offload processing and bandwidth, a camera with a higher resolution substream would give you more flexibility, but it is not strictly required if your current setup works and you are satisfied with performance(1)(2).

📚 Sources:

1. Camera setup
2. Face Recognition
3. Full Reference Config
4. Live View

---

Have another question? Just tag @inkeep.

[hawkeye217]

The AI is partially correct here. Using the same stream as you use for recordings and resizing it with the detect width and height is the appropriate method. However, 15fps is not necessary whatsoever, especially for a doorbell camera. It only puts unnecessary strain on Frigate's object detection pipeline for no benefit because objects will not be moving quickly. I would reduce it back to 5.

[vw-kombi]

I guess there lies my issue with this cam. I don’t want to drop the now shared main stream down to 5 as I want a better recording. It’s not a door bell can, is is a dahua cam. Just does not have a high enough sub stream.

[NickM-27]

your fps does not need to match, you can leave your camera at whatever fps you want and just leave frigate to default, it will skip frames to only process 5 per second

[vw-kombi]

Well….. I learned something new. I have been setting the cams to match the drove FPS. I was sure I ready that a few years back when I started.

[Code394]

I also have Dahua cameras that support a secondary stream with a maximum resolution of 704×576.
When I tried using the main stream, which is set to 2560×1440 at 25 fps, for detection to achieve good facial recognition

with the configuration:

detect:
  enabled: true
  width: 2560
  height: 1440
  fps: 5

it caused high CPU usage, even though I set it to sample only 5 fps.

[TCS-UK]

I do the following (Please note you need to enable exec in docker compose and this is used with Nvidia GPU)
This creates a sub stream to use for detect at 1280:720, 5fps with gop of 5. All of which are the recommended setting

go2rtc:
  streams:
    # -----------------------------------------------------------------------
    # PATIO — Hikvision 192.168.20.158 | Nvidia GPU 0
    # patio     : 2688x1520      | 15fps         | GOP 15 | H.264 | source passthrough | record
    # patio_sub : 1280x720      | 5fps          | GOP 5  | H.264 (h264_nvenc)        | detect / live SD
    # pipeline  : NVDEC H.264 (GPU 0) → scale_cuda (GPU 0) → h264_nvenc (GPU 0)
    # -----------------------------------------------------------------------
    patio:
      - rtsp://{FRIGATE_CAMLOGIN}@192.168.20.158:554/Streaming/Channels/101
   patio_sub:
      - exec:/usr/lib/ffmpeg/7.0/bin/ffmpeg -hide_banner -rtsp_transport tcp 
        -hwaccel cuda -hwaccel_device 0 -hwaccel_output_format cuda -i 
        rtsp://127.0.0.1:8554/patio -an -vf scale_cuda=1280:720 -c:v h264_nvenc 
        -gpu 0 -preset p2 -tune ll -rc cbr -b:v 2M -maxrate 2M -bufsize 2M -r 5 
        -g 5 -keyint_min 5 -bf 0 -f rtsp -rtsp_transport tcp {{output}}

cameras:
  patio:
    enabled: true
    ffmpeg:
      hwaccel_args: preset-nvidia
      gpu: 0
      inputs:
        - path: rtsp://127.0.0.1:8554/patio
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/patio_sub
          input_args: preset-rtsp-restream
          roles:
            - detect
    detect:
      enabled: true
      width: 1280
      height: 720
      fps: 5
    live:
      streams:
        SD Stream: patio_sub
        HD Stream: patio