Object Detection not working for cameras in portrait orientation

[CornBread81032133]

Hi Everyone,

I'm having some trouble getting cameras that are set in portrait orientation to work with object detection in Frigate.

From my config below, the camera in question is FrontDoor, that has the object detection set as person & cat.
When looking through the debug section and adding bounding boxes, nothing is highlighted like it is on the other cameras.

        - path: rtsp://<user>:<pass>@192.168.0.13:554/axis-media/media.amp?Parameters=Name=LowRes&resolution=640x360&fps=5&compression=30&videocodec=h264
          roles:
            - detect
    detect:
      width: 400
      height: 640
      fps: 5

So, if I physically rotate the camera to landscape and change the config detect values, it then works perfectly with bounding boxes showing around persons in frame. Is this an issue with my config, or is portrait oriented video feeds not supported in Firgate?

        - path: rtsp://<user>:<pass>@192.168.0.13:554/axis-media/media.amp?Parameters=Name=LowRes&resolution=640x360&fps=5&compression=30&videocodec=h264
          roles:
            - detect
    detect:
      width: 640
      height: 400
      fps: 5

For reference, all video feeds appear correctly in the previews and back in home assistant.

Full config extract:

mqtt:
  host: 192.168.0.6
  user: m<User>
  password: <Password>

detectors:
  cpu1:
    type: cpu
    num_threads: 6

cameras:
  GarageFront:
    ffmpeg:
      hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
      inputs:
        - path: rtsp://<user>:<pass>@192.168.0.8/axis-media/media.amp?camera=1
          roles:
            - rtmp
            - record
        - path: rtsp://<user>:<pass>@192.168.0.8/axis-media/media.amp?camera=1&resolution=640x360&fps=5&compression=30&videocodec=h264
          roles:
            - detect
    detect:
      width: 640
      height: 400
      fps: 5

    objects:
      track:
        - person
        - car
      filters:
        person:
          threshold: 0.7
        car:
          threshold: 0.7

    zones:
      front_rollerdoor:
        coordinates: 0,0,640,0,640,400,0,400

  GarageRear:
    ffmpeg:
      hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
      inputs:
        - path: rtsp://<user>:<pass>@192.168.0.8/axis-media/media.amp?camera=2
          roles:
            - rtmp
            - record
        - path: rtsp://<user>:<pass>@192.168.0.8/axis-media/media.amp?camera=2&resolution=640x360&fps=5&compression=30&videocodec=h264
          roles:
            - detect
    detect:
      width: 640
      height: 400
      fps: 5

    objects:
      track:
        - person
        - car
      filters:
        person:
          threshold: 0.7
        car:
          threshold: 0.7

    zones:
      rear_rollerdoor:
        coordinates: 0,0,640,0,640,400,0,400

  FrontGarage:
    ffmpeg:
      hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
      inputs:
        - path: rtsp://<user>:<pass>@192.168.0.12:554/axis-media/media.amp
          roles:
            - rtmp
            - record
        - path: rtsp://<user>:<pass>@192.168.0.12:554/axis-media/media.amp?Parameters=Name=LowRes&resolution=640x360&fps=5&compression=30&videocodec=h264
          roles:
            - detect
    detect:
      width: 640
      height: 400
      fps: 5

    objects:
      track:
        - person
        - car
      filters:
        person:
          threshold: 0.7
          mask:
            - 0,140,0,242,286,158,589,94,595,64,640,79,640,0,0,0
        car:
          threshold: 0.7
          mask:
            - 0,140,0,242,286,158,589,94,595,64,640,79,640,0,0,0

    motion:
      mask:
        - 181,189,369,322,467,336,524,282,580,206,585,67,640,0,465,0,0,0,0,253,21,237,51,224
        - 0,400,114,400,110,368,0,368

    zones:
      garage_outside:
        coordinates: 0,198,285,138,484,94,583,96,640,0,640,400,0,400

  FrontDoor:
    ffmpeg:
      hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
      inputs:
        - path: rtsp://<user>:<pass>@192.168.0.13:554/axis-media/media.amp
          roles:
            - rtmp
            - record
        - path: rtsp://<user>:<pass>@192.168.0.13:554/axis-media/media.amp?Parameters=Name=LowRes&resolution=640x360&fps=5&compression=30&videocodec=h264
          roles:
            - detect
    detect:
      width: 400
      height: 640
      fps: 5

    objects:
      track:
        - person
        - cat
      filters:
        person:
          threshold: 0.7
          mask:
            - 80,0,400,0,400,163,275,117,177,101,96,114
        cat:
          threshold: 0.7
          mask:
            - 80,0,400,0,400,163,275,117,177,101,96,114

    motion:
      mask:
        - 191,289,245,284,400,381,400,0,0,0,0,36,79,36,102,191,129,249

    zones:
      front_door:
        coordinates: 0,0,88,0,108,135,269,112,400,157,400,640,0,640

record:
  enabled: True
  retain:
    days: 14
    mode: motion
  events:
    retain:
      default: 14
      mode: active_objects

[blakeblackshear]

It should work fine. I have a portrait camera at my front door.

[CornBread81032133]

Thanks for confirming.
Any idea why this might be happening then? It just seems odd that swapping the x and y values cause it to start working fine.

[NickM-27]

I'd suggest you start without masks, maybe it's blocking the objects from being detected.

Without seeing an example of the frame it's only guessing though