[Camera Support]: IPC-Color4K-T with Rotation generates garbage preview output

[dakaix]

Describe the problem you are having

When trying to use rotate an RTSP stream from my new IPC-Color4K-T by 90 degrees (-vf transpose=1) the resulting preview within Frigate seems to be corrupted - yet recordings show correctly (albeit seemingly not at proper resolution). Is there a setting I have wrong here?

If I remove "-vf transpose=1" then the preview does show, but obviously in the wrong orientation.

Version

0.13.2-6476F8A

Frigate config file

mqtt:
  enabled: true
  host: ***mqtt_ip***
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: ***mqtt_user***
  password: ***mqtt_password***

detectors: # <---- add detectors
  coral:
    type: edgetpu
    device: usb

record:
  enabled: true
  retain:
    days: 14
    mode: all
  events:
    retain:
      default: 30
      mode: motion

detect:
  width: 896
  height: 672
  fps: 5

cameras:
  Doorbell:
    ffmpeg:
      hwaccel_args: preset-jetson-h264
      input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -rw_timeout 5000000
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy
      inputs:
      - path: 
          http://192.168.18.100/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=frigate&password=***
        roles:
        - detect
      - path: 
          http://192.168.18.100/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=frigate&password=***
        roles:
        - record
    detect:
      width: 896
      height: 672
      fps: 5
    motion:
      mask:
      - 501,501,572,491,587,484,591,433,625,401,710,432,690,402,704,382,713,387,698,332,741,336,875,316,875,200,896,198,896,541,826,672,789,672,646,603
      - 0,672,331,672,328,610,438,592,471,517,443,477,422,477,422,441,376,410,315,401,204,387,189,483,134,488,63,498
      - 0,40,381,40,384,7,0,11
  Driveway:
    ffmpeg:
      hwaccel_args: preset-jetson-h265
      input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -analyzeduration 1000M -probesize 1000M
      output_args:
        detect: -f rawvideo -pix_fmt yuv420p -vf transpose=1
        record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -vf "transpose=1"
        rtmp: -f flv -pix_fmt yuv420p -vf transpose=1
      inputs:
      - path: 
          rtsp://user:***@192.168.18.101/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif
        roles:
        - detect
      - path: 
          rtsp://user:***@192.168.18.101/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif
        roles:
        - record
    detect:
      width: 704
      height: 576
      fps: 5
ffmpeg:
  hwaccel_args: preset-jetson-h265

Relevant log output

OPY TO CLIPBOARD
2024-05-05 20:42:37.122315588  [INFO] Preparing Frigate...
2024-05-05 20:42:37.200274905  [INFO] Starting Frigate...
2024-05-05 20:42:40.765959685  [2024-05-05 20:42:40] frigate.app                    INFO    : Starting Frigate (0.13.2-6476f8a)
2024-05-05 20:42:40.890812024  [2024-05-05 20:42:40] peewee_migrate.logs            INFO    : Starting migrations
2024-05-05 20:42:40.908535441  [2024-05-05 20:42:40] peewee_migrate.logs            INFO    : There is nothing to migrate
2024-05-05 20:42:40.925879674  [2024-05-05 20:42:40] frigate.app                    INFO    : Recording process started: 465
2024-05-05 20:42:40.935978252  [2024-05-05 20:42:40] frigate.app                    INFO    : go2rtc process pid: 111
2024-05-05 20:42:41.001614448  [2024-05-05 20:42:40] frigate.app                    INFO    : Output process started: 476
2024-05-05 20:42:41.013597266  [2024-05-05 20:42:40] detector.coral                 INFO    : Starting detection process: 475
2024-05-05 20:42:43.711664927  [2024-05-05 20:42:41] frigate.app                    INFO    : Camera processor started for Doorbell: 483
2024-05-05 20:42:43.712227001  [2024-05-05 20:42:41] frigate.detectors.plugins.edgetpu_tfl INFO    : Attempting to load TPU as usb
2024-05-05 20:42:43.720524471  [2024-05-05 20:42:41] frigate.app                    INFO    : Camera processor started for Driveway: 485
2024-05-05 20:42:43.721117266  [2024-05-05 20:42:43] frigate.detectors.plugins.edgetpu_tfl INFO    : TPU found
2024-05-05 20:42:43.721464386  [2024-05-05 20:42:41] frigate.app                    INFO    : Capture process started for Doorbell: 488
2024-05-05 20:42:43.721750575  [2024-05-05 20:42:41] frigate.app                    INFO    : Capture process started for Driveway: 490

FFprobe output from your camera

ffprobe version 549430e Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --enable-nvmpi
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
[rtsp @ 0xaaaae955c350] max delay reached. need to consume packet
[rtsp @ 0xaaaae955c350] RTP: missed 1 packets
[rtsp @ 0xaaaae955c350] max delay reached. need to consume packet
[rtsp @ 0xaaaae955c350] RTP: missed 1 packets
[rtsp @ 0xaaaae955c350] max delay reached. need to consume packet
[rtsp @ 0xaaaae955c350] RTP: missed 1 packets
[rtsp @ 0xaaaae955c350] max delay reached. need to consume packet
[rtsp @ 0xaaaae955c350] RTP: missed 1 packets
[rtsp @ 0xaaaae955c350] max delay reached. need to consume packet
[rtsp @ 0xaaaae955c350] RTP: missed 1 packets
[rtsp @ 0xaaaae955c350] max delay reached. need to consume packet
[rtsp @ 0xaaaae955c350] RTP: missed 1 packets
Input #0, rtsp, from 'rtsp://user:***@192.168.18.101:554/cam/realmonitor?channel=1&subtype=0':
  Metadata:
    title           : Media Server
  Duration: N/A, start: 0.016000, bitrate: N/A
  Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 16.42 fps, 16.33 tbr, 90k tbn
  Stream #0:1: Audio: aac (LC), 64000 Hz, mono, fltp

ffprobe version 549430e Copyright (c) 2007-2023 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --enable-nvmpi
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
Input #0, rtsp, from 'rtsp://user:***@192.168.18.101:554/cam/realmonitor?channel=1&subtype=1':
  Metadata:
    title           : Media Server
  Duration: N/A, start: 0.400000, bitrate: N/A
  Stream #0:0: Video: hevc (Main), yuv420p(tv), 704x576, 5 fps, 10 tbr, 90k tbn

Frigate stats

{"cameras":{"Doorbell":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.1,"capture_pid":488,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":23916,"pid":483,"process_fps":5.1,"skipped_fps":0.0},"Driveway":{"audio_dBFS":0.0,"audio_rms":0.0,"camera_fps":5.1,"capture_pid":490,"detection_enabled":1,"detection_fps":0.0,"ffmpeg_pid":499,"pid":485,"process_fps":5.1,"skipped_fps":0.0}},"cpu_usages":{"1":{"cmdline":"/package/admin/s6/command/s6-svscan -d4 -- /run/service","cpu":"0.0","cpu_average":"0","mem":"0.0"},"27":{"cmdline":"s6-supervise s6-linux-init-shutdownd","cpu":"0.0","cpu_average":"0","mem":"0.0"},"29":{"cmdline":"/package/admin/s6-linux-init/command/s6-linux-init-shutdownd -c /run/s6/basedir -g 3000 -C -B","cpu":"0.0","cpu_average":"0","mem":"0.0"},"36":{"cmdline":"s6-supervise s6rc-oneshot-runner","cpu":"0.0","cpu_average":"0","mem":"0.0"},"37":{"cmdline":"s6-supervise frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"38":{"cmdline":"s6-supervise go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"39":{"cmdline":"s6-supervise go2rtc-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"40":{"cmdline":"s6-supervise frigate-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"41":{"cmdline":"s6-supervise go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"42":{"cmdline":"s6-supervise nginx-log","cpu":"0.0","cpu_average":"0","mem":"0.0"},"43":{"cmdline":"s6-supervise nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"44":{"cmdline":"s6-supervise s6rc-fdholder","cpu":"0.0","cpu_average":"0","mem":"0.0"},"55":{"cmdline":"/package/admin/s6-2.11.3.2/command/s6-fdholderd -1 -i data/rules","cpu":"0.0","cpu_average":"0","mem":"0.0"},"56":{"cmdline":"/package/admin/s6/command/s6-ipcserverd -1 -- /package/admin/s6/command/s6-ipcserver-access -v0 -E -l0 -i data/rules -- /package/admin/s6/command/s6-sudod -t 30000 -- /package/admin/s6-rc/command/s6-rc-oneshot-run -l ../.. --","cpu":"0.0","cpu_average":"0","mem":"0.0"},"100":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"103":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/go2rtc","cpu":"0.0","cpu_average":"0","mem":"0.0"},"104":{"cmdline":"s6-log -b -- T 1 n0 s10000000 T /dev/shm/logs/frigate","cpu":"0.0","cpu_average":"0","mem":"0.0"},"111":{"cmdline":"/usr/local/go2rtc/bin/go2rtc -config=/dev/shm/go2rtc.yaml","cpu":"0.0","cpu_average":"0","mem":"0.1"},"121":{"cmdline":"bash ./run.user go2rtc-healthcheck","cpu":"0.0","cpu_average":"0","mem":"0.0"},"123":{"cmdline":"python3 -u -m frigate","cpu":"1.4","cpu_average":"3","mem":"1.2"},"140":{"cmdline":"nginx: master process nginx","cpu":"0.0","cpu_average":"0","mem":"0.0"},"166":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"167":{"cmdline":"nginx: worker process","cpu":"0.1","cpu_average":"0","mem":"0.0"},"168":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"169":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"170":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"171":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"173":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"184":{"cmdline":"nginx: worker process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"242":{"cmdline":"nginx: cache manager process","cpu":"0.0","cpu_average":"0","mem":"0.0"},"463":{"cmdline":"frigate.logger       ","cpu":"0.0","cpu_average":"0","mem":"0.6"},"465":{"cmdline":"frigate.recording_manager","cpu":"2.1","cpu_average":"2","mem":"0.7"},"474":{"cmdline":"/usr/bin/python3 -c from multiprocessing.resource_tracker import main;main(46)","cpu":"0.0","cpu_average":"0","mem":"0.0"},"475":{"cmdline":"frigate.detector.coral","cpu":"0.2","cpu_average":"5","mem":"0.6"},"476":{"cmdline":"frigate.output       ","cpu":"0.3","cpu_average":"1","mem":"0.6"},"483":{"cmdline":"frigate.process:Doorbell","cpu":"0.4","cpu_average":"6","mem":"0.6"},"485":{"cmdline":"frigate.process:Driveway","cpu":"0.5","cpu_average":"11","mem":"0.6"},"486":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 896x672 -i pipe: -f mpegts -s 960x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.1"},"488":{"cmdline":"frigate.capture:Doorbell","cpu":"0.5","cpu_average":"3","mem":"0.6"},"490":{"cmdline":"frigate.capture:Driveway","cpu":"0.3","cpu_average":"2","mem":"0.6"},"491":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 704x576 -i pipe: -f mpegts -s 880x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.2"},"498":{"cmdline":"ffmpeg -f rawvideo -pix_fmt yuv420p -video_size 1280x720 -i pipe: -f mpegts -s 1280x720 -codec:v mpeg1video -q 8 -bf 0 pipe:","cpu":"0.0","cpu_average":"0","mem":"0.1"},"499":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -c:v hevc_nvmpi -resize 704x576 -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -analyzeduration 1000M -probesize 1000M -i rtsp://*:*@192.168.18.101/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif -r 5 -f rawvideo -pix_fmt yuv420p -vf transpose=1 pipe:","cpu":"0.5","cpu_average":"3","mem":"0.2"},"515":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -c:v hevc_nvmpi -resize 704x576 -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -analyzeduration 1000M -probesize 1000M -i rtsp://*:*@192.168.18.101/cam/realmonitor?channel=1&subtype=0&unicast=true&proto=Onvif -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -vf transpose=1 /tmp/cache/Driveway@%Y%m%d%H%M%S%z.mp4","cpu":"7.5","cpu_average":"53","mem":"0.5"},"23916":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -c:v h264_nvmpi -resize 896x672 -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -rw_timeout 5000000 -i http://192.168.18.100/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=*&password=* -r 5 -threads 2 -f rawvideo -pix_fmt yuv420p pipe:","cpu":"1.6","cpu_average":"8","mem":"0.2"},"23918":{"cmdline":"ffmpeg -hide_banner -loglevel warning -threads 2 -c:v h264_nvmpi -resize 896x672 -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -rw_timeout 5000000 -i http://192.168.18.100/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=*&password=* -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy /tmp/cache/Doorbell@%Y%m%d%H%M%S%z.mp4","cpu":"0.3","cpu_average":"0","mem":"0.1"},"24209":{"cmdline":"sleep 30s","cpu":"0.0","cpu_average":"0","mem":"0.0"}},"detection_fps":0.0,"detectors":{"coral":{"detection_start":0.0,"inference_speed":18.91,"pid":475}},"gpu_usages":{"jetson-gpu":{"gpu":"0.0%","mem":"-"}},"processes":{"go2rtc":{"pid":111},"logger":{"pid":463},"recording":{"pid":465}},"service":{"last_updated":1714948684,"latest_version":"0.13.2","storage":{"/dev/shm":{"free":509.0,"mount_type":"tmpfs","total":512.0,"used":3.0},"/media/frigate/clips":{"free":64201.1,"mount_type":"ext4","total":110639.2,"used":40775.9},"/media/frigate/recordings":{"free":64201.1,"mount_type":"ext4","total":110639.2,"used":40775.9},"/tmp/cache":{"free":950.4,"mount_type":"tmpfs","total":953.7,"used":3.3}},"temperatures":{},"uptime":10523,"version":"0.13.2-6476f8a"}}

Operating system

Other Linux

Install method

Docker Compose

Coral version

USB

Network connection

Wired

Camera make and model

EmpireTech IPC-Color4K-T

Any other information that may be helpful

Running on a Jetson Xavier AGX dev kit (aarch64), using JetPack 5.1.1 (Ubuntu 20.04) via Docker Compose.

[NickM-27]

Most likely a bug in the Jetson ffmpeg implementation for transpose. Your camera should have a rotate option built in though

[dakaix]

That was my thought, however the container I'm using here is the Frigate image for Jetpack 5 (stable-tensorrt-jp5). This seems to bundle a different ffmpeg binary than the base OS, I've tried linking that one into the container or one I've compiled myself - but that seems to hit libavcodec compatibility issues with go2rtc.

You're correct that the camera does have a rotate option for it's stream - but rather ridiculously that doesn't allow you to change the output resolution/aspect ratio to match. So you lose significant resolution since the new rotated image must be scaled down to fit within the original resolution's width & height.