Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HW Accel Support]: Adding go2rtc presets #8658

Closed
MarcA711 opened this issue Nov 18, 2023 · 30 comments
Closed

[HW Accel Support]: Adding go2rtc presets #8658

MarcA711 opened this issue Nov 18, 2023 · 30 comments
Labels
beta Related to the current beta version of frigate support triage

Comments

@MarcA711
Copy link
Contributor

MarcA711 commented Nov 18, 2023

Describe the problem you are having

I currently try to find the right ffmpeg options for go2rtc transcoding. However, the recording process does not seem to create recordings. The mse live stream in the browser from go2rtc works.

Version

0.13.0-FE52A49

Frigate config file

detectors:
  rknn:
    type: rknn
    core_mask: 0b111

model:
  input_pixel_format: bgr

objects:
  track:
    - person

mqtt:
  enabled: false

go2rtc:
  streams:
    Cam:
      - exec:ffmpeg -hide_banner -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v h264_rkmpp_encoder -g 50 -bf 0 -pix_fmt:v yuv420p -c:a aac -rtsp_transport tcp -f rtsp {{output}}

detect:
  enabled: true
  width: 1280
  height: 720
  fps: 5
record:
  enabled: true
  events:
    retain:
      default: 10
      mode: motion

ffmpeg:
  hwaccel_args: preset-rk-h265
  output_args:
    record: "preset-record-generic-audio-aac"

cameras:
  Camera:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam
          input_args: "preset-rtsp-generic" # "preset-rtsp-restream"
          roles:
            - record
        - path: rtsp://xxx.xxx.xxx.xxx:554/12
          input_args: "preset-rtsp-generic"
          roles:
            - detect
    live:
      stream_name: Cam

docker-compose file or Docker CLI command

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    image: frigate:latest-rk
    shm_size: "400mb"
#    group_add:
#      - "110" # render
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/ubuntu/frigate-rknn/data/config:/config
      - /home/ubuntu/frigate-rknn/data/media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935"
      - "8554:8554"
      - "8555:8555/tcp"
      - "8555:8555/udp"

Relevant log output

2023-11-18 17:05:26.574363359  [2023-11-18 17:05:26] ffmpeg.Carport.record          ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:05:26.575135977  [2023-11-18 17:05:26] watchdog.Carport               INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:05:26.587095950  [2023-11-18 17:05:26] watchdog.Carport               INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:05:36.603546887  [2023-11-18 17:05:36] ffmpeg.Carport.record          ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:05:36.615360737  [2023-11-18 17:05:36] watchdog.Carport               INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:05:36.615781316  [2023-11-18 17:05:36] watchdog.Carport               INFO    : Waiting for ffmpeg to exit gracefully...

FFprobe output from your camera

$ sudo ./ffprobe -hide_banner -rtsp_transport tcp -i "rtsp://127.0.0.1:8554/Cam"
Input #0, rtsp, from 'rtsp://127.0.0.1:8554/Cam':
  Metadata:
    title           : go2rtc/1.8.3
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 3840x2160, 5 fps, 5 tbr, 90k tbn
  Stream #0:1: Audio: aac (LC), 8000 Hz, mono, fltp
[h264_rkmpp_decoder @ 0xaaab2a13c910] Picture format is yuv420p.

Operating system

Other Linux

Install method

Docker Compose

Network connection

Wired

Camera make and model

Ctronics 8MP PTZ

Any other information that may be helpful

This command produces a working out.mp4:
sudo ./ffmpeg -hide_banner -fflags +genpts -rtsp_transport tcp -i "rtsp://127.0.0.1:8554/Cam" -c copy out.mp4

@NickM-27 NickM-27 added the beta Related to the current beta version of frigate label Nov 18, 2023
@NickM-27
Copy link
Collaborator

Need to see more logs

@MarcA711
Copy link
Contributor Author

MarcA711 commented Nov 18, 2023

This is the full log after some minutes:

2023-11-18 17:49:40.894108138  [INFO] Preparing Frigate...
2023-11-18 17:49:40.938296171  [INFO] Starting Frigate...
2023-11-18 17:49:42.780865077  [2023-11-18 17:49:42] frigate.app                    INFO    : Starting Frigate (0.13.0-fe52a49)
2023-11-18 17:49:42.802333619  [2023-11-18 17:49:42] peewee_migrate.logs            INFO    : Starting migrations
2023-11-18 17:49:42.807866493  [2023-11-18 17:49:42] peewee_migrate.logs            INFO    : There is nothing to migrate
2023-11-18 17:49:42.824678613  [2023-11-18 17:49:42] frigate.app                    INFO    : Recording process started: 455
2023-11-18 17:49:42.827256343  [2023-11-18 17:49:42] frigate.app                    INFO    : go2rtc process pid: 88
2023-11-18 17:49:42.848401138  [2023-11-18 17:49:42] detector.rknn                  INFO    : Starting detection process: 464
2023-11-18 17:49:42.856201411  [2023-11-18 17:49:42] frigate.app                    INFO    : Output process started: 466
2023-11-18 17:49:42.876207839  [2023-11-18 17:49:42] frigate.app                    INFO    : Camera processor started for Camera: 471
2023-11-18 17:49:42.882913787  [2023-11-18 17:49:42] frigate.app                    INFO    : Capture process started for Camera: 473
2023-11-18 17:49:43.068657641  I RKNN: [17:49:43.068] RKNN Runtime Information: librknnrt version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
2023-11-18 17:49:43.068684766  I RKNN: [17:49:43.068] RKNN Driver Information: version: 0.9.2
2023-11-18 17:49:43.069050805  I RKNN: [17:49:43.069] RKNN Model Information: version: 6, toolkit version: 1.5.2+b642f30c(compiler version: 1.5.2 (c6b7b351a@2023-08-23T07:39:01)), target: RKNPU v2, target platform: rk3588, framework name: ONNX, framework layout: NCHW, model inference type: static_shape
2023-11-18 17:50:02.938791592  [2023-11-18 17:50:02] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:50:02.939351879  [2023-11-18 17:50:02] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:50:02.951503785  [2023-11-18 17:50:02] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:50:12.959539181  [2023-11-18 17:50:12] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:50:12.959555806  [2023-11-18 17:50:12] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:50:12.959557264  [2023-11-18 17:50:12] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:50:22.986982486  [2023-11-18 17:50:22] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:50:22.987056860  [2023-11-18 17:50:22] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:50:22.987112568  [2023-11-18 17:50:22] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:50:33.016840166  [2023-11-18 17:50:33] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:50:33.029023280  [2023-11-18 17:50:33] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:50:33.029840523  [2023-11-18 17:50:33] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:50:43.045626540  [2023-11-18 17:50:43] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:50:43.058756104  [2023-11-18 17:50:43] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:50:43.060190510  [2023-11-18 17:50:43] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:50:53.077779574  [2023-11-18 17:50:53] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:50:53.090070312  [2023-11-18 17:50:53] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:50:53.090954930  [2023-11-18 17:50:53] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:51:03.106182948  [2023-11-18 17:51:03] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:51:03.118177355  [2023-11-18 17:51:03] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:51:03.119510845  [2023-11-18 17:51:03] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:51:13.136738202  [2023-11-18 17:51:13] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:51:13.148531360  [2023-11-18 17:51:13] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:51:13.149476936  [2023-11-18 17:51:13] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:51:23.167595890  [2023-11-18 17:51:23] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:51:23.167600557  [2023-11-18 17:51:23] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:51:23.167602307  [2023-11-18 17:51:23] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:51:33.205680710  [2023-11-18 17:51:33] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:51:33.205711335  [2023-11-18 17:51:33] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:51:33.205715710  [2023-11-18 17:51:33] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:51:43.223696958  [2023-11-18 17:51:43] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:51:43.223701333  [2023-11-18 17:51:43] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:51:43.223703083  [2023-11-18 17:51:43] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:51:53.251332861  [2023-11-18 17:51:53] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-18 17:51:53.251337528  [2023-11-18 17:51:53] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:51:53.251339278  [2023-11-18 17:51:53] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:52:03.278780622  [2023-11-18 17:52:03] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:52:03.278785288  [2023-11-18 17:52:03] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:52:03.278787038  [2023-11-18 17:52:03] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:52:13.311435156  [2023-11-18 17:52:13] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:52:13.311439823  [2023-11-18 17:52:13] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:52:13.311441573  [2023-11-18 17:52:13] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:52:23.331548686  [2023-11-18 17:52:23] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:52:23.331553061  [2023-11-18 17:52:23] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:52:23.331554811  [2023-11-18 17:52:23] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:52:33.361220915  [2023-11-18 17:52:33] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:52:33.361225582  [2023-11-18 17:52:33] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:52:33.361227332  [2023-11-18 17:52:33] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:52:43.389970009  [2023-11-18 17:52:43] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:52:43.389974384  [2023-11-18 17:52:43] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:52:43.389976134  [2023-11-18 17:52:43] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:52:53.418087631  [2023-11-18 17:52:53] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:52:53.418091131  [2023-11-18 17:52:53] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:52:53.418157339  [2023-11-18 17:52:53] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:53:03.437735594  [2023-11-18 17:53:03] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:53:03.437740261  [2023-11-18 17:53:03] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:53:03.437742011  [2023-11-18 17:53:03] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:53:13.466422511  [2023-11-18 17:53:13] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:53:13.466430095  [2023-11-18 17:53:13] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:53:13.466431845  [2023-11-18 17:53:13] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:53:23.494088295  [2023-11-18 17:53:23] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:53:23.494092962  [2023-11-18 17:53:23] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:53:23.494094420  [2023-11-18 17:53:23] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:53:33.521525106  [2023-11-18 17:53:33] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:53:33.521529773  [2023-11-18 17:53:33] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:53:33.521531523  [2023-11-18 17:53:33] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:53:43.549259399  [2023-11-18 17:53:43] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:53:43.549264066  [2023-11-18 17:53:43] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:53:43.549265524  [2023-11-18 17:53:43] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:53:53.575774228  [2023-11-18 17:53:53] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:53:53.581733807  [2023-11-18 17:53:53] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:53:53.581738473  [2023-11-18 17:53:53] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:54:03.597947908  [2023-11-18 17:54:03] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:54:03.597952283  [2023-11-18 17:54:03] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:54:03.597954033  [2023-11-18 17:54:03] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:54:13.625666406  [2023-11-18 17:54:13] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:54:13.625671364  [2023-11-18 17:54:13] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:54:13.625672822  [2023-11-18 17:54:13] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:54:23.653267348  [2023-11-18 17:54:23] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:54:23.653271723  [2023-11-18 17:54:23] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:54:23.653273473  [2023-11-18 17:54:23] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:54:33.684556663  [2023-11-18 17:54:33] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:54:33.684561621  [2023-11-18 17:54:33] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:54:33.684563080  [2023-11-18 17:54:33] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:54:43.714317351  [2023-11-18 17:54:43] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:54:43.714322018  [2023-11-18 17:54:43] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:54:43.714323768  [2023-11-18 17:54:43] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:54:53.744754396  [2023-11-18 17:54:53] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:54:53.744758771  [2023-11-18 17:54:53] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:54:53.744760229  [2023-11-18 17:54:53] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:55:03.775987962  [2023-11-18 17:55:03] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:55:03.790806846  [2023-11-18 17:55:03] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:55:03.790810929  [2023-11-18 17:55:03] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:55:13.808665325  [2023-11-18 17:55:13] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:55:13.808669991  [2023-11-18 17:55:13] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:55:13.808741740  [2023-11-18 17:55:13] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:55:23.837077721  [2023-11-18 17:55:23] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:55:23.837851504  [2023-11-18 17:55:23] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:55:23.838375040  [2023-11-18 17:55:23] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:55:33.868891720  [2023-11-18 17:55:33] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:55:33.868896386  [2023-11-18 17:55:33] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:55:33.868897845  [2023-11-18 17:55:33] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:55:43.895784110  [2023-11-18 17:55:43] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:55:43.895788777  [2023-11-18 17:55:43] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:55:43.895790235  [2023-11-18 17:55:43] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:55:53.931131154  [2023-11-18 17:55:53] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:55:53.931369151  [2023-11-18 17:55:53] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:55:53.931694064  [2023-11-18 17:55:53] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:56:03.959596007  [2023-11-18 17:56:03] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:56:03.959843046  [2023-11-18 17:56:03] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:56:03.960334208  [2023-11-18 17:56:03] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:56:13.975939800  [2023-11-18 17:56:13] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:56:13.975944758  [2023-11-18 17:56:13] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:56:13.975946217  [2023-11-18 17:56:13] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:56:24.007279939  [2023-11-18 17:56:24] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:56:24.007820392  [2023-11-18 17:56:24] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:56:24.007993057  [2023-11-18 17:56:24] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:56:34.042068875  [2023-11-18 17:56:34] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:56:34.042073541  [2023-11-18 17:56:34] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:56:34.042075291  [2023-11-18 17:56:34] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:56:44.076377889  [2023-11-18 17:56:44] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:56:44.076382556  [2023-11-18 17:56:44] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:56:44.076384306  [2023-11-18 17:56:44] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:56:54.106568763  [2023-11-18 17:56:54] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:56:54.106787802  [2023-11-18 17:56:54] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:56:54.106790427  [2023-11-18 17:56:54] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:57:04.179088330  [2023-11-18 17:57:04] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:57:04.179102622  [2023-11-18 17:57:04] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:57:04.179106413  [2023-11-18 17:57:04] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:57:14.196578693  [2023-11-18 17:57:14] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:57:14.196583360  [2023-11-18 17:57:14] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:57:14.196585110  [2023-11-18 17:57:14] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:57:24.229533766  [2023-11-18 17:57:24] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:57:24.236124197  [2023-11-18 17:57:24] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:57:24.236689442  [2023-11-18 17:57:24] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:57:34.252477309  [2023-11-18 17:57:34] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:57:34.252481976  [2023-11-18 17:57:34] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:57:34.252483726  [2023-11-18 17:57:34] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:57:44.282190471  [2023-11-18 17:57:44] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:57:44.282194846  [2023-11-18 17:57:44] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:57:44.282311220  [2023-11-18 17:57:44] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-18 17:57:54.310097169  [2023-11-18 17:57:54] watchdog.Camera                ERROR   : No new recording segments were created for Camera in the last 120s. restarting the ffmpeg record process...
2023-11-18 17:57:54.310101835  [2023-11-18 17:57:54] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-18 17:57:54.310103585  [2023-11-18 17:57:54] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...

@NickM-27
Copy link
Collaborator

Do you see anything being created in /tmp/cache ?

@MarcA711
Copy link
Contributor Author

No, the folder is empty.

@NickM-27
Copy link
Collaborator

That would indicate that ffmpeg is not receiving any data. I'm assuming it's the same when using the restream preset?

@MarcA711
Copy link
Contributor Author

Yes, the restream preset doesn't work as well. This is confusing, since I can save the go2rtc stream to a mp4 file on my host machine that uses the same ffmpeg binary as the image. Can I somehow find out the exact ffmpeg command that is used to capture the stream?

@NickM-27
Copy link
Collaborator

Yes if you open http://frigate:5000/api/config you will find the ffmpeg_cmds

@MarcA711
Copy link
Contributor Author

Ah, I see the problem.
My cam streams h265. I want to record h264. So, I transcode it using go2rtc.

Now, I used the hardware accel preset preset-rk-h265, since the detect stream directly connects to the cam and decodes the h265 stream.

However, the hardware accel preset is also applied to the record command (-c:v hevc_rkmpp_decoder):
"ffmpeg -hide_banner -loglevel warning -threads 2 -c:v hevc_rkmpp_decoder -user_agent FFmpeg Frigate/0.13.0-a822a51 -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1 -i rtsp://127.0.0.1:8554/Cam -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -c:a aac /tmp/cache/Camera@%Y%m%d%H%M%S%z.mp4"

Why exactly does the record process use hw acceleration? Does this improve performance? I thought, this process just saves the files.

Can I pass preset-rk-h264 to just one specific input (not camera, since this cam has two different codecs)?

@MarcA711
Copy link
Contributor Author

I looked it up in the docs, I can set the hwaccel args for each input. This would solve the problem, I will try it out later.

@NickM-27
Copy link
Collaborator

There's no particular reason, just that the record and detect can be done in the same ffmpeg process so it didn't make sense to force it out for record. In the future they may be forced to be separate in which case that would be easier to do.

@MarcA711
Copy link
Contributor Author

Apparently, the hardware presets were not the only issue. The problem still occurs. I will update the issue tomorrow with new log outputs etc.

@a2nt
Copy link

a2nt commented Nov 19, 2023

I have following config and it works
System 0.13.0-7d157df

detectors:
  armnn:
    type: rknn
    core_mask: 0b111 # 0b111 3 cores

model:
  # name of yolov8 model or path to your own .rknn model file
  # possible values are:
  # - default-yolov8n
  # - default-yolov8s
  # - default-yolov8m
  # - default-yolov8l
  # - default-yolov8x
  # - /config/model_cache/rknn/your_custom_model.rknn
  path: default-yolov8m
  width: 320
  height: 320
  input_pixel_format: bgr
  input_tensor: nhwc


ffmpeg:
  hwaccel_args: preset-rk-h264 # or preset-rk-h265
  output_args:
     record: preset-record-generic-audio-aac
     detect: -f rawvideo
     rtmp: -c copy -f flv -flvflags no_duration_filesize

ui:
  # Optional: Set the default live mode for cameras in the UI (default: shown below)
  live_mode: mse
  timezone: Etc/GMT-2
  use_experimental: True
  time_format: 24hour
  date_style: full
  time_style: medium
  strftime_fmt: "%a, %H:%M (%d.%m.%Y)"

# Re-stream
go2rtc:
  streams:
    camera_0:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.88.10:554
      - "ffmpeg:camera_0#audio=aac"
    camera_1:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.88.11:554
      - "ffmpeg:camera_1#audio=aac"
    camera_2:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.88.12:554
      - "ffmpeg:camera_2#audio=aac"
    camera_3:
      - rtsp://{FRIGATE_RTSP_USER}:{FRIGATE_RTSP_PASSWORD}@192.168.88.13:554
      - "ffmpeg:camera_3#audio=aac"

mqtt:
  host: 192.168.88.2 # Docker Host
  user: "{FRIGATE_MQTT_USER}"
  password: "{FRIGATE_MQTT_PASSWORD}"
  stats_interval: 60
  
birdseye:
  enabled: True
  mode: continuous

audio:
  enabled: True
  listen:
    - scream
    - speech
    - yell

database:
  path: /media/frigate/frigate.db

objects:
  track:
    - person
    - car
    - dog
    - cat

motion:
  threshold: 30
  contour_area: 30
  delta_alpha: 0.2
  frame_alpha: 0.2
  frame_height: 50
  mqtt_off_delay: 30

record:
  enabled: True
  retain:
    days: 2
    mode: motion

  events:
    #max_seconds: 300
    pre_capture: 10 # sec
    post_capture: 30 # sec
    objects:
      - person
      - car
      - dog
      - cat

    retain:
      default: 7
      mode: active_objects
      objects:
        person: 10
        car: 3
        dog: 1
        cat: 1

snapshots:
  enabled: True
  timestamp: True
  bounding_box: True
  crop: False
  height: 1080
  retain:
    default: 15
    objects:
      person: 15
      car: 3
      dog: 2
      cat: 2
rtmp:
  enabled: False

live:
  height: 1080
  quality: 1

timestamp_style:
  position: "br"
  format: "%d.%m.%Y %H:%M:%S"
  color:
    red: 255
    green: 255
    blue: 255
  thickness: 2
  effect: "solid"

detect:
  enabled: true
  fps: 5
  max_disappeared: 25
  stationary:
    interval: 100
    threshold: 50

  # Optional: Define a maximum number of frames for tracking a stationary object (default: not set, track forever)
  #max_frames:
  #    default: 3000 # 3000 frames / 5 fps = 10 mins

logger:
  default: info
  logs:
    frigate.mqtt: error

cameras:
  camera_0:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/camera_0
          roles:
            - detect
            - record
            - audio
    detect:
      width: 2880
      height: 1616
    zones:
      inner0:
        coordinates: 0,1616,2880,1616,2880,1301,2369,755,0,1044
      outside_gate0:
        coordinates: 2607,824,2880,975,2880,697,2625,673,2458,659,2396,673,2387,699,2451,755
      outside_house0:
        coordinates: 262,753,1614,504,1645,642,2149,617,2343,726,2767,688,2574,626,2380,544,2680,384,2880,395,2880,666,1559,824,313,959
    mqtt:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: True
      height: 1080
      quality: 100

  camera_1:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/camera_1 
          roles:
            - detect
            - record
            - audio
    detect:
      width: 2880
      height: 1616
    zones:
      inner1:
        coordinates: 2880,813,1807,275,1696,242,0,260,0,1616,2880,1616
      outside1:
        coordinates: 65,0,67,209,1177,185,1781,218,2880,697,2880,0
    mqtt:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: True
      height: 1080
      quality: 100

  camera_2:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/camera_2
          input_args: preset-rtsp-restream
          roles:
            - detect
            - record
            - audio
    detect:
      width: 2880
      height: 1616
    zones:
      inside2:
        coordinates: 413,693,460,0,0,0,0,1616,455,1616
      outside2:
        coordinates: 1434,1616,1414,1412,1377,1179,1124,462,984,193,406,0,404,506,429,1616
    mqtt:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: True
      height: 1080
      quality: 100

  camera_3:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/camera_3
          roles:
            - detect
            - record
            - audio
    detect:
      width: 2880
      height: 1616
    mqtt:
      enabled: True
      timestamp: True
      bounding_box: True
      crop: True
      height: 1080
      quality: 100
    zones:
      inner3:
        coordinates: 2880,895,2880,1616,2245,1616,2687,1157
      outside3:
        coordinates: 0,1616,2249,1616,2660,1204,2880,886,2669,724,1830,926,0,1201

I suppose you also need to following into docker-compose.yml

environment:
   FFMPEG_RKMPP_PIXFMT: "YUV420P"

Note: that some cameras work using H265X stream or so called H265+ and you can use H264 preset for frigate

@MarcA711
Copy link
Contributor Author

Thank you for sharing your config @a2nt!

  • This issue is about transcoding using go2rtc. As far as I can see, you don't transcode your stream, you just restream.
  • Why do you use the environment variable to set the pixel format? The problem with this environment variable is that it overwrites the preset. This means, if I add a preset in the future that requires a specific pixel format, you will force the pixel format with your environment variable and break that preset.

@NickM-27 Is there any pixel format that should be used when encoding? I didn't set the pixel format for encoding.

"preset-rk-h264": "ffmpeg -hide_banner {0} -c:v h264_rkmpp_encoder -profile high {1}",
"preset-rk-h265": "ffmpeg -hide_banner {0} -c:v hevc_rkmpp_encoder -profile high {1}",

I don't know if there is there is a default value or if the pixel format is left unchanged, the rk-ffmprg docs don't mention it.

@MarcA711
Copy link
Contributor Author

To my original transcoding issue:
Now, I used this config:

detectors:
  rknn:
    type: rknn
    core_mask: 0b111

model:
  input_pixel_format: bgr

objects:
  track:
    - person

mqtt:
  enabled: false

go2rtc:
  streams:
    Cam:
      - exec:ffmpeg -hide_banner -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v h264_rkmpp_encoder  -g 50 -bf 0 -pix_fmt:v yuv420p -c:a aac -rtsp_transport tcp -f rtsp {{output}}

detect:
  enabled: true
  width: 1280
  height: 720
  fps: 5
record:
  enabled: true
  events:
    retain:
      default: 10
      mode: motion

ffmpeg:
  output_args:
    record: "preset-record-generic-audio-copy"

cameras:
  Camera:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam
          input_args:  "preset-rtsp-restream" # "preset-rtsp-generic"
          hwaccel_args: ""
          roles:
            - record
        - path: rtsp://xxx.xxx.xxx.xxx:554/12
          input_args: "preset-rtsp-generic"
          hwaccel_args: preset-rk-h265
          roles:
            - detect
    live:
      stream_name: Cam
  • directly after starting frigate I noticed the following things:
    • no recordings created in /tmp/cache
    • this command on my host machine sudo ./ffmpeg -hide_banner -loglevel warning -threads 2 -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Cam -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy ./test/Camera@%Y%m%d%H%M%S%z.mp4 (essentially the same command that frigate uses in the container) fails with rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
    • this log output was generated:
2023-11-19 10:44:39.225472701  [INFO] Preparing Frigate...
2023-11-19 10:44:39.269108360  [INFO] Starting Frigate...
2023-11-19 10:44:41.123133434  [2023-11-19 10:44:41] frigate.app                    INFO    : Starting Frigate (0.13.0-a822a51)
2023-11-19 10:44:41.144934201  [2023-11-19 10:44:41] peewee_migrate.logs            INFO    : Starting migrations
2023-11-19 10:44:41.149845542  [2023-11-19 10:44:41] peewee_migrate.logs            INFO    : There is nothing to migrate
2023-11-19 10:44:41.166832967  [2023-11-19 10:44:41] frigate.app                    INFO    : Recording process started: 455
2023-11-19 10:44:41.169520074  [2023-11-19 10:44:41] frigate.app                    INFO    : go2rtc process pid: 89
2023-11-19 10:44:41.195250147  [2023-11-19 10:44:41] detector.rknn                  INFO    : Starting detection process: 464
2023-11-19 10:44:41.200880150  [2023-11-19 10:44:41] frigate.app                    INFO    : Output process started: 466
2023-11-19 10:44:41.213723020  [2023-11-19 10:44:41] frigate.app                    INFO    : Camera processor started for Camera: 470
2023-11-19 10:44:41.223541036  [2023-11-19 10:44:41] frigate.app                    INFO    : Capture process started for Camera: 472
2023-11-19 10:44:41.454031955  I RKNN: [10:44:41.453] RKNN Runtime Information: librknnrt version: 1.5.2 (c6b7b351a@2023-08-23T15:28:22)
2023-11-19 10:44:41.454109538  I RKNN: [10:44:41.454] RKNN Driver Information: version: 0.9.2
2023-11-19 10:44:41.454513202  I RKNN: [10:44:41.454] RKNN Model Information: version: 6, toolkit version: 1.5.2+b642f30c(compiler version: 1.5.2 (c6b7b351a@2023-08-23T07:39:01)), target: RKNPU v2, target platform: rk3588, framework name: ONNX, framework layout: NCHW, model inference type: static_shape
2023-11-19 10:45:01.290503522  [2023-11-19 10:45:01] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-19 10:45:01.303196476  [2023-11-19 10:45:01] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-19 10:45:01.303628723  [2023-11-19 10:45:01] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-19 10:45:11.330246793  [2023-11-19 10:45:11] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-19 10:45:11.331478785  [2023-11-19 10:45:11] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-19 10:45:11.331878365  [2023-11-19 10:45:11] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-19 10:45:21.349859560  [2023-11-19 10:45:21] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-19 10:45:21.349864519  [2023-11-19 10:45:21] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-19 10:45:21.349865977  [2023-11-19 10:45:21] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-19 10:45:31.382195806  [2023-11-19 10:45:31] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-19 10:45:31.394656304  [2023-11-19 10:45:31] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-19 10:45:31.395300007  [2023-11-19 10:45:31] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-19 10:45:41.415569697  [2023-11-19 10:45:41] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-19 10:45:41.428746814  [2023-11-19 10:45:41] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-19 10:45:41.428758189  [2023-11-19 10:45:41] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-19 10:45:51.453352878  [2023-11-19 10:45:51] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-19 10:45:51.453908791  [2023-11-19 10:45:51] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-19 10:45:51.453915207  [2023-11-19 10:45:51] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-19 10:46:01.471934515  [2023-11-19 10:46:01] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-19 10:46:01.484045889  [2023-11-19 10:46:01] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-19 10:46:01.484952091  [2023-11-19 10:46:01] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-19 10:46:11.493181874  [2023-11-19 10:46:11] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-19 10:46:11.508613267  [2023-11-19 10:46:11] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-19 10:46:11.509365762  [2023-11-19 10:46:11] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
2023-11-19 10:46:21.525982854  [2023-11-19 10:46:21] ffmpeg.Camera.record           ERROR   : rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input
2023-11-19 10:46:21.525994521  [2023-11-19 10:46:21] watchdog.Camera                INFO    : Terminating the existing ffmpeg process...
2023-11-19 10:46:21.525998312  [2023-11-19 10:46:21] watchdog.Camera                INFO    : Waiting for ffmpeg to exit gracefully...
  • after some time (usually 1 - 3 mins):
    • there are recording segments in /tmp/cache
    • The same ffmpeg command works on my host machine now and creates recording segments. However, it sometimes prints this warning after starting [rtsp @ 0xaaaaeeaeeab0] Stream #0: not enough frames to estimate rate; consider increasing probesize, sometimes it prints no warning. I guess, this is caused by the camera and no real issue.
    • If I encode the stream to aac (by replacing -c copy with -c:v copy -c:a aac) it always prints this warning: [aac @ 0xaaaaea07a860] Too many bits 8832.000000 > 6144 per frame requested, clamping to max. I guess this can be fixed by setting the bitrate in the go2rtc command. In any case, this is not related to video transcoding.
    • No new log output generated.
    • Recording works.

So, some minutes after starting it suddenly works. However, I am not really happy with this, because of the errors in the logs and I don't know, why there is this problem directly after starting.

@a2nt
Copy link

a2nt commented Nov 19, 2023

@MarcA711

This issue is about transcoding using go2rt

Agreed, but will it slow down detectors? I mean detectors work on raw video stream, but if u will transcode it should have some kind delay

Why do you use the environment variable to set the pixel format? The problem with this environment variable is that it overwrites the preset.

Sorry, just thought you had a warning about pixel format, but it was ffprobe results.

@MarcA711
Copy link
Contributor Author

@a2nt If you use transcoding on your detect stream, it will add a delay, yes. However, my config above uses the h265 stream from the camera and just decodes it for detection, so no transcoding on this stream. Only the stream used for recording is transcoded above.

@NickM-27
Copy link
Collaborator

I don't know why this would be, perhaps the data coming out of ffmpeg isn't usable for rtsp at first. I would suggest maybe creating a dummy stream in go2rtc, say ffmpeg:rtsp://nothing/stream#video=h264#hardware=v4l2m2m and make sure trace logs are enabled

go2rtc:
  log:
    exec: trace
  streams: ...

This will give you an ffmpeg command that go2rtc generates and hopefully provide some info. And also provide more info for what's going on with the command we expect to work.

@MarcA711
Copy link
Contributor Author

I am not sure what you mean, with creating a dummy stream in go2rtc. Should I replace the nothing/stream part with my camera stream? And should I replace the hardware=v4l2m2m with the rk preset?

However, I used this config to generate a trace output of my stream:

detectors:
  rknn:
    type: rknn
    core_mask: 0b111

model:
  input_pixel_format: bgr

objects:
  track:
    - person

mqtt:
  enabled: false

go2rtc:
  log:
    exec: trace
  streams:
    Cam:
      - exec:ffmpeg -hide_banner -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v h264_rkmpp_encoder -g 50 -bf 0 -pix_fmt:v yuv420p -c:a copy -rtsp_transport tcp -f rtsp {{output}}

detect:
  enabled: true
  width: 1280
  height: 720
  fps: 5
record:
  enabled: true
  events:
    retain:
      default: 10
      mode: motion

ffmpeg:
  output_args:
    record: "preset-record-generic-audio-copy"

cameras:
  Camera:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam
          input_args:  "preset-rtsp-restream" # "preset-rtsp-generic"
          hwaccel_args: ""
          roles:
            - record
        - path: rtsp://xxx.xxx.xxx.xxx:554/12
          input_args: "preset-rtsp-generic"
          hwaccel_args: preset-rk-h265
          roles:
            - detect
    live:
      stream_name: Cam

This is the go2rtc trace output (these messages repeat regularly)

2023-11-19 14:53:54.938832645  [INFO] Preparing go2rtc config...
2023-11-19 14:53:55.402228165  [INFO] Starting go2rtc...
2023-11-19 14:53:55.510073410  14:53:55.509 INF go2rtc version 1.8.3 linux/arm64
2023-11-19 14:53:55.510901736  14:53:55.510 INF [rtsp] listen addr=:8554
2023-11-19 14:53:55.510938194  14:53:55.510 INF [api] listen addr=:1984
2023-11-19 14:53:55.511434316  14:53:55.511 INF [webrtc] listen addr=:8555
2023-11-19 14:53:57.006195687  14:53:57.005 DBG [exec] run url="exec:ffmpeg -hide_banner -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v h264_rkmpp_encoder -g 50 -bf 0 -pix_fmt:v yuv420p -c:a copy -rtsp_transport tcp -f rtsp {output}"
2023-11-19 14:54:02.201827681  Input #0, rtsp, from 'rtsp://xxx.xxx.xxx.xxx:554/11':
2023-11-19 14:54:02.201838473    Metadata:
2023-11-19 14:54:02.201840806      title           : 10
2023-11-19 14:54:02.201843431    Duration: N/A, start: 0.000000, bitrate: N/A
2023-11-19 14:54:02.201900889    Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 5 fps, 5 tbr, 90k tbn
2023-11-19 14:54:02.201904389    Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
2023-11-19 14:54:02.202205387  [hevc_rkmpp_decoder @ 0xaaaaef89add0] Picture format is nv12.
2023-11-19 14:54:02.202910923  Stream mapping:
2023-11-19 14:54:02.202922006    Stream #0:0 -> #0:0 (hevc (hevc_rkmpp_decoder) -> h264 (h264_rkmpp_encoder))
2023-11-19 14:54:02.202924340    Stream #0:1 -> #0:1 (copy)
2023-11-19 14:54:02.202926673  Press [q] to stop, [?] for help
2023-11-19 14:54:02.208330922  [hevc_rkmpp_decoder @ 0xaaaaef89add0] Decoder noticed an info change
2023-11-19 14:54:02.285120617  [h264_rkmpp_encoder @ 0xaaaaef76d970] Picture format is yuv420p.
2023-11-19 14:54:02.285488697  [h264_rkmpp_encoder @ 0xaaaaef76d970] Rate Control mode is set to CBR
2023-11-19 14:54:02.285493072  [h264_rkmpp_encoder @ 0xaaaaef76d970] Bitrate Target/Min/Max is set to 6000000/5625000/6375000
2023-11-19 14:54:02.285494530  [h264_rkmpp_encoder @ 0xaaaaef76d970] Profile is set to HIGH
2023-11-19 14:54:02.285517864  [h264_rkmpp_encoder @ 0xaaaaef76d970] 8x8 Transform is enabled
2023-11-19 14:54:02.285519030  [h264_rkmpp_encoder @ 0xaaaaef76d970] Level is set to 0
2023-11-19 14:54:02.285520197  [h264_rkmpp_encoder @ 0xaaaaef76d970] Coder is set to CABAC
2023-11-19 14:54:02.285521655  [h264_rkmpp_encoder @ 0xaaaaef76d970] Quality Min/Max is set to 50%(Quant=30) / 100%(Quant=10)
2023-11-19 14:54:02.334303984  14:54:02.333 DBG [exec] run launch=5.327811966s
2023-11-19 14:54:02.334574649  Output #0, rtsp, to 'rtsp://127.0.0.1:8554/22eb31769b74fd6fd96d8d910780ece6':
2023-11-19 14:54:02.334602065    Metadata:
2023-11-19 14:54:02.334698606      title           : 10
2023-11-19 14:54:02.334759564      encoder         : Lavf60.3.100
2023-11-19 14:54:02.334952646    Stream #0:0: Video: h264 (High), yuv420p(tv, progressive), 3840x2160, q=2-31, 6000 kb/s, 5 fps, 90k tbn
2023-11-19 14:54:02.334989104      Metadata:
2023-11-19 14:54:02.335034020        encoder         : Lavc60.3.100 h264_rkmpp_encoder
2023-11-19 14:54:02.335111894    Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
2023-11-19 14:54:02.338205287  [h264_rkmpp_encoder @ 0xaaaaef76d970] Reconfigured with w=3840, h=2160, format=yuv420p.
2023-11-19 14:54:02.370205830  frame=    0 fps=0.0 q=-0.0 size=N/A time=00:00:01.10 bitrate=N/A speed= 220x    
av_interleaved_write_frame(): Broken pipe
2023-11-19 14:54:02.370263579  [out#0/rtsp @ 0xaaaaef7566c0] Error muxing a packet
2023-11-19 14:54:02.500845522  [out#0/rtsp @ 0xaaaaef7566c0] Error writing trailer: Broken pipe
2023-11-19 14:54:02.500902688  frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:01.16 bitrate=N/A speed=3.89x    
2023-11-19 14:54:02.500950521  video:1122kB audio:9kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
2023-11-19 14:54:02.506807725  Conversion failed!
2023-11-19 14:54:02.509850368  14:54:02.509 TRC [exec] close url="exec:ffmpeg -hide_banner -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v h264_rkmpp_encoder -g 50 -bf 0 -pix_fmt:v yuv420p -c:a copy -rtsp_transport tcp -f rtsp {output}"
2023-11-19 14:54:04.936909109  [INFO] Starting go2rtc healthcheck service...
2023-11-19 14:54:17.031184292  14:54:17.031 DBG [exec] run url="exec:ffmpeg -hide_banner -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v h264_rkmpp_encoder -g 50 -bf 0 -pix_fmt:v yuv420p -c:a copy -rtsp_transport tcp -f rtsp {output}"
2023-11-19 14:54:22.208684957  Input #0, rtsp, from 'rtsp://xxx.xxx.xxx.xxx:554/11':
2023-11-19 14:54:22.209175536    Metadata:
2023-11-19 14:54:22.209671366      title           : 10
2023-11-19 14:54:22.210401985    Duration: N/A, start: 0.000000, bitrate: N/A
2023-11-19 14:54:22.210859023    Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 5 fps, 5 tbr, 90k tbn
2023-11-19 14:54:22.211131146    Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
2023-11-19 14:54:22.212331929  [hevc_rkmpp_decoder @ 0xaaaaeeaa3260] Picture format is nv12.
2023-11-19 14:54:22.214495787  Stream mapping:
2023-11-19 14:54:22.214727952    Stream #0:0 -> #0:0 (hevc (hevc_rkmpp_decoder) -> h264 (h264_rkmpp_encoder))
2023-11-19 14:54:22.214959533    Stream #0:1 -> #0:1 (copy)
2023-11-19 14:54:22.215135698  Press [q] to stop, [?] for help
2023-11-19 14:54:22.222534349  [hevc_rkmpp_decoder @ 0xaaaaeeaa3260] Decoder noticed an info change
2023-11-19 14:54:22.300201953  [h264_rkmpp_encoder @ 0xaaaaee99f270] Picture format is yuv420p.
2023-11-19 14:54:22.300936072  [h264_rkmpp_encoder @ 0xaaaaee99f270] Rate Control mode is set to CBR
2023-11-19 14:54:22.300989155  [h264_rkmpp_encoder @ 0xaaaaee99f270] Bitrate Target/Min/Max is set to 6000000/5625000/6375000
2023-11-19 14:54:22.301227737  [h264_rkmpp_encoder @ 0xaaaaee99f270] Profile is set to HIGH
2023-11-19 14:54:22.301262736  [h264_rkmpp_encoder @ 0xaaaaee99f270] 8x8 Transform is enabled
2023-11-19 14:54:22.301448235  [h264_rkmpp_encoder @ 0xaaaaee99f270] Level is set to 0
2023-11-19 14:54:22.301488776  [h264_rkmpp_encoder @ 0xaaaaee99f270] Coder is set to CABAC
2023-11-19 14:54:22.301519401  [h264_rkmpp_encoder @ 0xaaaaee99f270] Quality Min/Max is set to 50%(Quant=30) / 100%(Quant=10)
2023-11-19 14:54:22.351042266  14:54:22.350 DBG [exec] run launch=5.319631643s
2023-11-19 14:54:22.351320805  Output #0, rtsp, to 'rtsp://127.0.0.1:8554/22eb31769b74fd6fd96d8d910780ece6':
2023-11-19 14:54:22.351346763    Metadata:
2023-11-19 14:54:22.351492304      title           : 10
2023-11-19 14:54:22.351540720      encoder         : Lavf60.3.100
2023-11-19 14:54:22.351780177    Stream #0:0: Video: h264 (High), yuv420p(tv, progressive), 3840x2160, q=2-31, 6000 kb/s, 5 fps, 90k tbn
2023-11-19 14:54:22.351801760      Metadata:
2023-11-19 14:54:22.351982008        encoder         : Lavc60.3.100 h264_rkmpp_encoder
2023-11-19 14:54:22.352171299    Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
2023-11-19 14:54:22.354830695  [h264_rkmpp_encoder @ 0xaaaaee99f270] Reconfigured with w=3840, h=2160, format=yuv420p.
2023-11-19 14:54:22.385377290  frame=    0 fps=0.0 q=-0.0 size=N/A time=00:00:01.10 bitrate=N/A speed= 276x    
av_interleaved_write_frame(): Broken pipe
2023-11-19 14:54:22.385649121  [out#0/rtsp @ 0xaaaaee9616c0] Error muxing a packet
2023-11-19 14:54:22.496166303  [out#0/rtsp @ 0xaaaaee9616c0] Error writing trailer: Broken pipe
2023-11-19 14:54:22.496217927  frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:01.28 bitrate=N/A speed=4.56x    
2023-11-19 14:54:22.496261385  video:1125kB audio:9kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
2023-11-19 14:54:22.506203350  Conversion failed!
2023-11-19 14:54:22.509603282  14:54:22.509 TRC [exec] close url="exec:ffmpeg -hide_banner -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v h264_rkmpp_encoder -g 50 -bf 0 -pix_fmt:v yuv420p -c:a copy -rtsp_transport tcp -f rtsp {output}"

This is the output after some minutes, when everything works suddenly (I left out repeating lines with ...)

2023-11-19 14:59:58.015148375  14:59:58.014 DBG [exec] run url="exec:ffmpeg -hide_banner -i rtsp://@xxx.xxx.xxx.xxx:554/11 -c:v h264_rkmpp_encoder -g 50 -bf 0 -pix_fmt:v yuv420p -c:a copy -rtsp_transport tcp -f rtsp {output}"
2023-11-19 15:00:02.960844682  Input #0, rtsp, from 'rtsp://@xxx.xxx.xxx.xxx:554/11':
2023-11-19 15:00:02.960883765    Metadata:
2023-11-19 15:00:02.960935682      title           : 10
2023-11-19 15:00:02.961018806    Duration: N/A, start: 0.000000, bitrate: N/A
2023-11-19 15:00:02.961137805    Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 5 fps, 5 tbr, 90k tbn
2023-11-19 15:00:02.961195555    Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
2023-11-19 15:00:02.961560135  [hevc_rkmpp_decoder @ 0xaaaadc783d70] Picture format is nv12.
2023-11-19 15:00:02.962293962  Stream mapping:
2023-11-19 15:00:02.962348795    Stream #0:0 -> #0:0 (hevc (hevc_rkmpp_decoder) -> h264 (h264_rkmpp_encoder))
2023-11-19 15:00:02.962438045    Stream #0:1 -> #0:1 (copy)
2023-11-19 15:00:02.962457003  Press [q] to stop, [?] for help
2023-11-19 15:00:02.970136233  [hevc_rkmpp_decoder @ 0xaaaadc783d70] Decoder noticed an info change
2023-11-19 15:00:03.044477683  [h264_rkmpp_encoder @ 0xaaaadc5e9460] Picture format is yuv420p.
2023-11-19 15:00:03.044916638  [h264_rkmpp_encoder @ 0xaaaadc5e9460] Rate Control mode is set to CBR
2023-11-19 15:00:03.044966805  [h264_rkmpp_encoder @ 0xaaaadc5e9460] Bitrate Target/Min/Max is set to 6000000/5625000/6375000
2023-11-19 15:00:03.044998596  [h264_rkmpp_encoder @ 0xaaaadc5e9460] Profile is set to HIGH
2023-11-19 15:00:03.045024846  [h264_rkmpp_encoder @ 0xaaaadc5e9460] 8x8 Transform is enabled
2023-11-19 15:00:03.045052262  [h264_rkmpp_encoder @ 0xaaaadc5e9460] Level is set to 0
2023-11-19 15:00:03.045078512  [h264_rkmpp_encoder @ 0xaaaadc5e9460] Coder is set to CABAC
2023-11-19 15:00:03.045106512  [h264_rkmpp_encoder @ 0xaaaadc5e9460] Quality Min/Max is set to 50%(Quant=30) / 100%(Quant=10)
2023-11-19 15:00:03.095284154  15:00:03.095 DBG [exec] run launch=5.080222694s
2023-11-19 15:00:03.096428937  Output #0, rtsp, to 'rtsp://127.0.0.1:8554/22eb31769b74fd6fd96d8d910780ece6':
2023-11-19 15:00:03.096842808    Metadata:
2023-11-19 15:00:03.097413304      title           : 10
2023-11-19 15:00:03.097914383      encoder         : Lavf60.3.100
2023-11-19 15:00:03.098754085    Stream #0:0: Video: h264 (High), yuv420p(tv, progressive), 3840x2160, q=2-31, 6000 kb/s, 5 fps, 90k tbn
2023-11-19 15:00:03.098908959      Metadata:
2023-11-19 15:00:03.099704327        encoder         : Lavc60.3.100 h264_rkmpp_encoder
2023-11-19 15:00:03.100365822    Stream #0:1: Audio: pcm_alaw, 8000 Hz, mono, s16, 64 kb/s
2023-11-19 15:00:03.113522509  [h264_rkmpp_encoder @ 0xaaaadc5e9460] Reconfigured with w=3840, h=2160, format=yuv420p.
2023-11-19 15:00:03.954373318  frame=    0 fps=0.0 q=-0.0 size=N/A time=00:00:00.88 bitrate=N/A speed= 477x    
frame=    8 fps=0.0 q=-0.0 size=N/A time=00:00:03.30 bitrate=N/A speed=5.51x    
[rtsp @ 0xaaaadc561910] max delay reached. need to consume packet
2023-11-19 15:00:03.955119103  [rtsp @ 0xaaaadc561910] RTP: missed 39 packets
2023-11-19 15:01:04.143726615  frame=   17 fps= 14 q=-0.0 size=N/A time=00:00:05.30 bitrate=N/A speed=4.48x    
frame=   20 fps= 12 q=-0.0 size=N/A time=00:00:06.00 bitrate=N/A speed=3.53x    
frame=   23 fps= 10 q=-0.0 size=N/A time=00:00:06.40 bitrate=N/A speed=2.83x    
frame=   26 fps=9.2 q=-0.0 size=N/A time=00:00:07.00 bitrate=N/A speed=2.47x    
...
[rtsp @ 0xaaaadc561910] max delay reached. need to consume packet
2023-11-19 15:06:54.911820202  [rtsp @ 0xaaaadc561910] RTP: missed 4 packets
2023-11-19 15:07:55.926753983  frame= 2072 fps=5.0 q=-0.0 size=N/A time=00:06:56.26 bitrate=N/A speed=1.01x    
frame= 2076 fps=5.0 q=-0.0 size=N/A time=00:06:57.08 bitrate=N/A speed=1.01x    
frame= 2079 fps=5.0 q=-0.0 size=N/A time=00:06:57.66 bitrate=N/A speed=1.01x    
frame= 2081 fps=5.0 q=-0.0 size=N/A time=00:06:58.28 bitrate=N/A speed=1.01x    
...
frame= 2806 fps=5.0 q=-0.0 size=N/A time=00:09:22.90 bitrate=N/A dup=1 drop=0 speed=1.01x    
frame= 2809 fps=5.0 q=-0.0 size=N/A time=00:09:23.50 bitrate=N/A dup=1 drop=0 speed=1.01x    
frame= 2812 fps=5.0 q=-0.0 size=N/A time=00:09:24.20 bitrate=N/A dup=1 drop=0 speed=1.01x    

The debug output of ffmpeg:

sudo ./ffmpeg -hide_banner -loglevel debug -threads 2 -rtsp_transport tcp -timeout 5000000 -i rtsp://127.0.0.1:8554/Cam -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy ./test/Camera@%Y%m%d%H%M%S%z.mp4
Splitting the commandline.
Reading option '-hide_banner' ... matched as option 'hide_banner' (do not show program banner) with argument '1'.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-threads' ... matched as AVOption 'threads' with argument '2'.
Reading option '-rtsp_transport' ... matched as AVOption 'rtsp_transport' with argument 'tcp'.
Reading option '-timeout' ... matched as AVOption 'timeout' with argument '5000000'.
Reading option '-i' ... matched as input url with argument 'rtsp://127.0.0.1:8554/Cam'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'segment'.
Reading option '-segment_time' ... matched as AVOption 'segment_time' with argument '10'.
Reading option '-segment_format' ... matched as AVOption 'segment_format' with argument 'mp4'.
Reading option '-reset_timestamps' ... matched as AVOption 'reset_timestamps' with argument '1'.
Reading option '-strftime' ... matched as AVOption 'strftime' with argument '1'.
Reading option '-c' ... matched as option 'c' (codec name) with argument 'copy'.
Reading option './test/Camera@%Y%m%d%H%M%S%z.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option hide_banner (do not show program banner) with argument 1.
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url rtsp://127.0.0.1:8554/Cam.
Successfully parsed a group of options.
Opening an input file: rtsp://127.0.0.1:8554/Cam.
[tcp @ 0xaaaaffa665c0] No default whitelist set
[tcp @ 0xaaaaffa665c0] Original list of addresses:
[tcp @ 0xaaaaffa665c0] Address 127.0.0.1 port 8554
[tcp @ 0xaaaaffa665c0] Interleaved list of addresses:
[tcp @ 0xaaaaffa665c0] Address 127.0.0.1 port 8554
[tcp @ 0xaaaaffa665c0] Starting connection attempt to 127.0.0.1 port 8554
[tcp @ 0xaaaaffa665c0] Successfully connected to 127.0.0.1 port 8554
rtsp://127.0.0.1:8554/Cam: Invalid data found when processing input

@NickM-27
Copy link
Collaborator

is the stream really only 5 fps?

@MarcA711
Copy link
Contributor Author

yes

@MarcA711
Copy link
Contributor Author

I tried it without hw transcoding using libx264 and the same errors occurred. So, this is not directly related to the hardware transcoding part. Maybe the ffmpeg binary is faulty.

@MarcA711
Copy link
Contributor Author

Well, now i am puzzled.

I used the latest static ffmpeg build from btbn and the error still exists. Perhaps one of the dependent libraries that btbn uses broke. I used FFmpeg 6.0 in both cases.
This is my config.yml:

#detectors:
#  rknn:
#    type: rknn
#    core_mask: 0b111

#model:
#  input_pixel_format: bgr

objects:
  track:
    - person

mqtt:
  enabled: false

go2rtc:
  log:
    exec: trace
  streams:
    Cam:
      - ffmpeg:rtsp://xxx.xxx.xxx.xxx:544/11#video=h264
      #- exec:ffmpeg -c:v hevc -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v libx264 -g 50 -bf 0 -pix_fmt:v yuv420p -c:a copy -rtsp_transport tcp -f rtsp {{output}}

detect:
  enabled: true
  width: 1280
  height: 720
  fps: 5
record:
  enabled: true
  events:
    retain:
      default: 10
      mode: motion

ffmpeg:
  output_args:
    record: "preset-record-generic-audio-copy"

cameras:
  Camera:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam
          input_args:  "preset-rtsp-restream" # "preset-rtsp-generic"
          hwaccel_args: ""
          roles:
            - record
        - path: rtsp://xxx.xxx.xxx.xxx:554/12
          input_args: "preset-rtsp-generic"
          hwaccel_args: preset-rk-h265
          roles:
            - detect
    live:
      stream_name: Cam

Here is the go2rtc trace log:

2023-11-19 18:01:22.108338309  [INFO] Preparing go2rtc config...
2023-11-19 18:01:22.539400617  [INFO] Starting go2rtc...
2023-11-19 18:01:22.647108810  18:01:22.647 INF go2rtc version 1.8.3 linux/arm64
2023-11-19 18:01:22.647936843  18:01:22.647 INF [rtsp] listen addr=:8554
2023-11-19 18:01:22.647999259  18:01:22.647 INF [api] listen addr=:1984
2023-11-19 18:01:22.648449296  18:01:22.648 INF [webrtc] listen addr=:8555
2023-11-19 18:01:24.121885549  18:01:24.121 DBG [exec] run url="exec:ffmpeg -hide_banner -allowed_media_types video -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v libx264 -g 50 -profile:v high -level:v 4.1 -preset:v superfast -tune:v zerolatency -pix_fmt:v yuv420p -an -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"
2023-11-19 18:01:29.130330955  Input #0, rtsp, from 'rtsp://xxx.xxx.xxx.xxx:554/11':
2023-11-19 18:01:29.130418162    Metadata:
2023-11-19 18:01:29.130768742      title           : 10
2023-11-19 18:01:29.131151988    Duration: N/A, start: 0.200000, bitrate: N/A
2023-11-19 18:01:29.131585109    Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 5 fps, 5 tbr, 90k tbn
2023-11-19 18:01:29.133434840  Stream mapping:
2023-11-19 18:01:29.133860377    Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
2023-11-19 18:01:29.133864460  Press [q] to stop, [?] for help
2023-11-19 18:01:29.343341116  [hevc @ 0xaaab0c0cf1c0] Could not find ref with POC 21
2023-11-19 18:01:29.447903716  [libx264 @ 0xaaab0c0c6380] frame MB size (240x135) > level limit (8192)
2023-11-19 18:01:29.448465461  [libx264 @ 0xaaab0c0c6380] using cpu capabilities: ARMv8 NEON
2023-11-19 18:01:29.450636605  [libx264 @ 0xaaab0c0c6380] profile High, level 4.1, 4:2:0, 8-bit
2023-11-19 18:01:29.450673355  [libx264 @ 0xaaab0c0c6380] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=8 lookahead_threads=8 sliced_threads=1 slices=8 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=1 keyint=50 keyint_min=5 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
2023-11-19 18:01:29.492818175  Output #0, rtsp, to 'rtsp://127.0.0.1:8554/9d9ec313b5f90824bc1c67b5f4dcf92a':
2023-11-19 18:01:29.492820508    Metadata:
2023-11-19 18:01:29.492830133      title           : 10
2023-11-19 18:01:29.492831300      encoder         : Lavf60.3.100
2023-11-19 18:01:29.492859883    Stream #0:018:01:29.492 DBG [exec] run launch=5.370936709s
2023-11-19 18:01:29.492866883  : Video: h264, yuv420p(tv, progressive), 3840x2160, q=2-31, 5 fps, 90k tbn
2023-11-19 18:01:29.492872133      Metadata:
2023-11-19 18:01:29.492873008        encoder         : Lavc60.3.100 libx264
2023-11-19 18:01:29.492873883      Side data:
2023-11-19 18:01:29.492879716        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
2023-11-19 18:01:29.584348950  frame=    0 fps=0.0 q=20.0 size=       0kB time=00:00:04.20 bitrate=   0.0kbits/s speed=4.2e+06x    
av_interleaved_write_frame(): Broken pipe
2023-11-19 18:01:29.584374616  [out#0/rtsp @ 0xaaab0c08b180] Error muxing a packet
2023-11-19 18:01:29.748978229  frame=    1 fps=0.0 q=16.0 Lsize=N/A time=00:00:04.40 bitrate=N/A speed=7.15x    
2023-11-19 18:01:29.748994563  video:168kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
2023-11-19 18:01:29.752391570  [libx264 @ 0xaaab0c0c6380] frame I:1     Avg QP: 8.55  size:171181
2023-11-19 18:01:29.752439402  [libx264 @ 0xaaab0c0c6380] frame P:1     Avg QP: 7.81  size:270573
2023-11-19 18:01:29.752473235  [libx264 @ 0xaaab0c0c6380] mb I  I16..4: 86.7%  9.1%  4.3%
2023-11-19 18:01:29.752507943  [libx264 @ 0xaaab0c0c6380] mb P  I16..4: 19.7%  6.3%  3.4%  P16..4: 40.9%  0.0%  0.0%  0.0%  0.0%    skip:29.6%
2023-11-19 18:01:29.752573859  [libx264 @ 0xaaab0c0c6380] 8x8 transform intra:11.9% inter:30.3%
2023-11-19 18:01:29.752609151  [libx264 @ 0xaaab0c0c6380] coded y,uvDC,uvAC intra: 47.3% 5.5% 2.8% inter: 27.6% 4.2% 1.2%
2023-11-19 18:01:29.752642400  [libx264 @ 0xaaab0c0c6380] i16 v,h,dc,p: 77% 13% 10%  0%
2023-11-19 18:01:29.752675358  [libx264 @ 0xaaab0c0c6380] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 14% 15% 47%  3%  6%  4%  4%  3%  5%
2023-11-19 18:01:29.752736608  [libx264 @ 0xaaab0c0c6380] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 19% 37%  2%  6%  3%  4%  2%  4%
2023-11-19 18:01:29.752779482  [libx264 @ 0xaaab0c0c6380] i8c dc,h,v,p: 94%  3%  3%  0%
2023-11-19 18:01:29.752831398  [libx264 @ 0xaaab0c0c6380] Weighted P-Frames: Y:0.0% UV:0.0%
2023-11-19 18:01:29.752864356  [libx264 @ 0xaaab0c0c6380] kb/s:8835.08
2023-11-19 18:01:29.930808667  Conversion failed!
2023-11-19 18:01:29.933720345  18:01:29.933 TRC [exec] close url="exec:ffmpeg -hide_banner -allowed_media_types video -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v libx264 -g 50 -profile:v high -level:v 4.1 -preset:v superfast -tune:v zerolatency -pix_fmt:v yuv420p -an -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"
2023-11-19 18:01:32.105271455  [INFO] Starting go2rtc healthcheck service...
2023-11-19 18:01:44.193686903  18:01:44.193 DBG [exec] run url="exec:ffmpeg -hide_banner -allowed_media_types video -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v libx264 -g 50 -profile:v high -level:v 4.1 -preset:v superfast -tune:v zerolatency -pix_fmt:v yuv420p -an -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"
2023-11-19 18:01:49.332000080  Input #0, rtsp, from 'rtsp://xxx.xxx.xxx.xxx:554/11':
2023-11-19 18:01:49.332010580    Metadata:
2023-11-19 18:01:49.332013205      title           : 10
2023-11-19 18:01:49.332015830    Duration: N/A, start: 0.201000, bitrate: N/A
2023-11-19 18:01:49.332059872    Stream #0:0: Video: hevc (Main), yuv420p(tv), 3840x2160, 5 fps, 5 tbr, 90k tbn
2023-11-19 18:01:49.332862530  Stream mapping:
2023-11-19 18:01:49.332866613    Stream #0:0 -> #0:0 (hevc (native) -> h264 (libx264))
2023-11-19 18:01:49.332867780  Press [q] to stop, [?] for help
2023-11-19 18:01:49.543985924  [hevc @ 0xaaaabf8246d0] Could not find ref with POC 21
2023-11-19 18:01:49.602134041  [libx264 @ 0xaaaabf817d70] frame MB size (240x135) > level limit (8192)
2023-11-19 18:01:49.602708619  [libx264 @ 0xaaaabf817d70] using cpu capabilities: ARMv8 NEON
2023-11-19 18:01:49.604883555  [libx264 @ 0xaaaabf817d70] profile High, level 4.1, 4:2:0, 8-bit
2023-11-19 18:01:49.604916805  [libx264 @ 0xaaaabf817d70] 264 - core 164 - H.264/MPEG-4 AVC codec - Copyleft 2003-2023 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=8 lookahead_threads=8 sliced_threads=1 slices=8 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=1 keyint=50 keyint_min=5 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
2023-11-19 18:01:49.649336894  Output #0, rtsp, to 'rtsp://127.0.0.1:8554/9d9ec313b5f90824bc1c67b5f4dcf92a':
2023-11-19 18:01:49.649368977    Metadata:
2023-11-19 18:01:49.649411269      title           : 10
2023-11-19 18:01:49.649450643      encoder         : Lavf60.3.100
2023-11-19 18:01:49.649564100    Stream #0:0: Video: h264, yuv420p(tv, progressive), 3840x2160, q=2-31, 5 fps, 90k tbn
2023-11-19 18:01:49.649584517      Metadata:
2023-11-19 18:01:49.649620975        encoder         : Lavc60.3.100 libx264
2023-11-19 18:01:49.649639058      Side data:
2023-11-19 18:01:49.649942680        cpb: 18:01:49.649 DBG [exec] run launch=5.455801365s
2023-11-19 18:01:49.649947930  bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
2023-11-19 18:01:49.730146989  frame=    0 fps=0.0 q=20.0 size=       0kB time=00:00:04.20 bitrate=   0.0kbits/s speed=N/A    
av_interleaved_write_frame(): Broken pipe
2023-11-19 18:01:49.730152822  [out#0/rtsp @ 0xaaaabf824320] Error muxing a packet
2023-11-19 18:01:49.951709902  frame=    1 fps=0.0 q=17.0 Lsize=N/A time=00:00:04.40 bitrate=N/A speed=7.11x    
2023-11-19 18:01:49.951715152  video:86kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
2023-11-19 18:01:49.967949154  [libx264 @ 0xaaaabf817d70] frame I:1     Avg QP: 6.69  size: 87366
2023-11-19 18:01:49.967952945  [libx264 @ 0xaaaabf817d70] frame P:1     Avg QP:10.90  size:447036
2023-11-19 18:01:49.967954404  [libx264 @ 0xaaaabf817d70] mb I  I16..4: 91.3%  5.0%  3.7%
2023-11-19 18:01:49.967955862  [libx264 @ 0xaaaabf817d70] mb P  I16..4: 15.1% 15.5% 14.7%  P16..4: 27.0%  0.0%  0.0%  0.0%  0.0%    skip:27.8%
2023-11-19 18:01:49.967957029  [libx264 @ 0xaaaabf817d70] 8x8 transform intra:14.1% inter:45.3%
2023-11-19 18:01:49.967958487  [libx264 @ 0xaaaabf817d70] coded y,uvDC,uvAC intra: 37.0% 12.3% 6.7% inter: 27.3% 14.9% 3.4%
2023-11-19 18:01:49.967959654  [libx264 @ 0xaaaabf817d70] i16 v,h,dc,p: 87%  5%  8%  0%
2023-11-19 18:01:49.967962279  [libx264 @ 0xaaaabf817d70] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 16% 41%  4%  7%  4%  5%  4%  7%
2023-11-19 18:01:49.967964904  [libx264 @ 0xaaaabf817d70] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19% 19% 29%  4%  8%  4%  6%  4%  7%
2023-11-19 18:01:49.967992612  [libx264 @ 0xaaaabf817d70] i8c dc,h,v,p: 87%  6%  6%  1%
2023-11-19 18:01:49.967994070  [libx264 @ 0xaaaabf817d70] Weighted P-Frames: Y:0.0% UV:0.0%
2023-11-19 18:01:49.967995237  [libx264 @ 0xaaaabf817d70] kb/s:10688.04
2023-11-19 18:01:50.136984018  Conversion failed!
2023-11-19 18:01:50.140069820  18:01:50.139 TRC [exec] close url="exec:ffmpeg -hide_banner -allowed_media_types video -fflags nobuffer -flags low_delay -timeout 5000000 -user_agent go2rtc/ffmpeg -rtsp_flags prefer_tcp -i rtsp://xxx.xxx.xxx.xxx:554/11 -c:v libx264 -g 50 -profile:v high -level:v 4.1 -preset:v superfast -tune:v zerolatency -pix_fmt:v yuv420p -an -user_agent ffmpeg/go2rtc -rtsp_transport tcp -f rtsp {output}"

@NickM-27
Copy link
Collaborator

I would definitely recommend using the btbn build that frigate uses, many users are encoding without issues

@MarcA711
Copy link
Contributor Author

Yes, I will try this next.

@MarcA711
Copy link
Contributor Author

I have the same error with the frigate:stable release from ghcr. Does this mean my cam sends invalid data?

@NickM-27
Copy link
Collaborator

Yes that seems to be the case, I've not had this issue. You could try using some MP4 file and transcoding that.

@MarcA711
Copy link
Contributor Author

MarcA711 commented Nov 19, 2023

This is my latest docker-compose.yml:

version: "3.9"
services:
  frigate:
    container_name: frigate
    privileged: true
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable
    shm_size: "400mb"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/ubuntu/frigate-rknn/data/config:/config
      - /home/ubuntu/frigate-rknn/data/media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935"
      - "8554:8554"
      - "8555:8555/tcp"
      - "8555:8555/udp"

this is my config.yml:

mqtt:
  enabled: false

go2rtc:
  log:
    exec: trace
  streams:
    Cam:
      - ffmpeg:rtsp://xxx.xxx.xxx.xxx:554/11#video=h264

detect:
  enabled: true
  width: 1280
  height: 720
  fps: 5
record:
  enabled: true
  events:
    retain:
      default: 10
      mode: motion

ffmpeg:
  output_args:
    record: "preset-record-generic-audio-copy"

cameras:
  Camera:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/Cam
          input_args:  "preset-rtsp-restream" # "preset-rtsp-generic"
          hwaccel_args: ""
          roles:
            - record
        - path: rtsp://xxx.xxx.xxx.xxx:554/12
          input_args: "preset-rtsp-generic"
          hwaccel_args: preset-rk-h265
          roles:
            - detect
    live:
      stream_name: Cam

@MarcA711
Copy link
Contributor Author

Since the issue is on my side, I will do the troubleshooting later. Consider this issue closed for now. When I have more time, I will look at it again and maybe open a new issue with my new insights. Thanks for the help, I really appreciate it!

But my main focus is still to enable the rockchip platform for the next release:

  • My next goal is to merge the go2rtc presets (I will use valid mp4 files for testing rather than my cam).
  • Next, I will try to collect information, which systems are compatible. I thought about creating a docker image that users can pull. This docker image automatically tries to do video transcoding using hardware acceleration, do inferences on the npu and select system information like the used sbc, chip kernel version etc. The image then generates an output.log with all relevant data that users can share.
  • Finally, I will start a discussion for rockchip users. Other users can share their test outputs from above, I will collect this data in a "supported systems" table. I will also include a trobuleshooting section with common problems. Maybe users can share tips, for example how they increased performance on their platform, which kernel works best etc.

Is this the right approach or should I continue differently?

@MarcA711 MarcA711 closed this as not planned Won't fix, can't repro, duplicate, stale Nov 19, 2023
@NickM-27
Copy link
Collaborator

I think go2rtc presets sounds good, at that point I think we'll be ready to push beta 6 and at that point we'd only want to merge bug fixes or docs changes.

I understand the want to add more features or other things but I think what's here now (potentially including go2rtc presets) is a great start and pushing to stable will provide more opportunity for exposure to get more users and info. It's always possible to do a 0.13.1 with more fixes after user feedback is collected, we usually done one or two post releases

@MarcA711
Copy link
Contributor Author

I absolutely agree 👍🏻
I will be looking forward to adding the presets within the next few days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beta Related to the current beta version of frigate support triage
Projects
None yet
Development

No branches or pull requests

3 participants