Home Assistant iOS App and Frigate #8328

janstadt · 2023-10-26T03:59:57Z

janstadt
Oct 26, 2023

Im looking for a second set of eyes on my config. On all devices we get nice clear live streams except our iphones. This all happened after updating my config to support the webrtc stuff. I understand iOS doesnt support MSE but it seems like its downgrading to jpg when at one time I had clear streams.

Below are my frigate and custom card configs:

environment_vars:
  LIBVA_DRIVER_NAME: i965 
ffmpeg:
  # Optional: global ffmpeg args (default: shown below)
  global_args: -hide_banner -loglevel warning
  # Optional: global hwaccel args (default: shown below)
  # NOTE: See hardware acceleration docs for your specific device
#   hwaccel_args:
#     - -hwaccel
#     - qsv
#     - -qsv_device
#     - /dev/dri/renderD128
  hwaccel_args: preset-intel-qsv-h264 #preset-vaapi
  #hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
  # Optional: global output args
  #output_args:
  #  clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac -filter:a volume=1.5
  output_args:
    # Optional: output args for detect streams (default: shown below)
    detect: -threads 2 -f rawvideo -pix_fmt yuv420p #-f rawvideo -pix_fmt yuv420p
    record: preset-record-generic-audio-aac
    rtmp: preset-record-generic-audio-aac
    # Optional: output args for record streams (default: shown below)
    #record: -f segment -segment_time 60 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    # Optional: output args for clips streams (default: shown below)
    #clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -c:a aac
    # Optional: output args for rtmp streams (default: shown below)
    #rtmp: -c copy -f flv -c:a aac
detectors:
  coral:
    type: edgetpu
    device: usb
#   cpu1:
#     type: cpu
#   cpu2:
#     type: cpu
#   cpu3:
#     type: cpu
#   cpu4:
#     type: cpu
mqtt: 
  host: [redacted]
  password: '[redacted]'
  user: [redacted]
cameras: 
  tucker: 
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/tucker # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record
            # - rtmp
        - path: 'rtsp://[redacted]@192.168.1.141:554/cam/realmonitor?channel=1&subtype=1'
          roles: 
            - detect
        # - path: 'rtsp://[redacted]@192.168.1.198:554/cam/realmonitor?channel=1&subtype=0'
        #   roles:
        #     - record
        #     - rtmp
    detect:
      fps: 5
      height: 480
      width: 704
      enabled: true
    objects:
      track:
        - person
    motion:
      mask:
        - 0,480,0,0,189,0,185,480
    # zones:
    #   back_door:
    #     coordinates: 539,480,525,366,467,373,428,338,364,339,358,390,196,401,196,480
    #     filters:
    #       person:
    #         min_area: 5000
    #         max_area: 100000
    #         threshold: 0.7
    #   main_stall:
    #     coordinates: 400,48,406,276,124,262,120,63
    #     filters:
    #       car:
    #         min_area: 5000
    #         max_area: 100000
    #         threshold: 0.7
    #   third_stall:
    #     coordinates: 603,76,583,187,462,175,473,51
    #     filters:
    #       car:
    #         min_area: 5000
    #         max_area: 100000
    #         threshold: 0.7
    snapshots:
      enabled: false
    record:
      enabled: true
      retain:
        days: 3
  garage: 
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/garage # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record
            # - rtmp
        - path: 'rtsp://[redacted]@192.168.1.198:554/cam/realmonitor?channel=1&subtype=1'
          roles: 
            - detect
        # - path: 'rtsp://[redacted]@192.168.1.198:554/cam/realmonitor?channel=1&subtype=0'
        #   roles:
        #     - record
        #     - rtmp
    detect:
      fps: 5
      height: 480
      width: 704
      enabled: true
    objects:
      track:
        - person
        - car
    motion:
      mask:
        - 704,0,704,43,505,40,503,0
        - 516,0,528,75,211,76,200,0
    zones:
      back_door:
        coordinates: 539,480,525,366,467,373,428,338,364,339,358,390,196,401,196,480
        filters:
          person:
            min_area: 5000
            max_area: 100000
            threshold: 0.7
      main_stall:
        coordinates: 400,48,406,276,124,262,120,63
        filters:
          car:
            min_area: 5000
            max_area: 100000
            threshold: 0.7
      third_stall:
        coordinates: 603,76,583,187,462,175,473,51
        filters:
          car:
            min_area: 5000
            max_area: 100000
            threshold: 0.7
    snapshots:
      enabled: false
    record:
      enabled: true
      retain:
        days: 3
  porch: 
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/porch # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record
        - path: 'rtsp://[redacted]@192.168.1.195:554/cam/realmonitor?channel=1&subtype=1'
          roles: 
            - detect
        # - path: 'rtsp://[redacted]@192.168.1.195:554/cam/realmonitor?channel=1&subtype=0'
        #   roles: 
        #     - record
        #     - rtmp
    detect:
      fps: 5
      height: 480
      width: 704
      enabled: true
    motion:
      mask:
        - 0,480,231,480,320,400,334,341,253,240,0,334
        - 0,0,704,0,704,153,591,106,478,75,422,64,151,63,182,0,125,142,0,176
    objects:
      track:
        - person
        - bus
        # - sports ball
      filters:
        person:
          mask:
            - 0,0,482,0,471,66,164,64,125,142,0,176
        bus:
          mask:
            - 704,306,704,342,704,381,704,429,704,480,467,480,308,480,197,480,96,480,0,480,0,139,0,44,102,37,116,138
            - 224,0,704,0,704,292,154,141,267,116
    zones:
      entryway:
        coordinates: 581,325,546,398,331,348,212,261,340,198
        filters:
          person:
            min_area: 5000
            max_area: 100000
            threshold: 0.7
      driveway_entry:
        coordinates: 295,114,244,79,34,118,77,156
        filters:
          bus:
            min_area: 5000
            max_area: 100000
            threshold: 0.7
    #   basketball_backboard:
    #     coordinates: 427,66,468,74,461,97,427,90
    #     filters:
    #       "sports ball":
    #         min_area: 5000
    #         max_area: 100000
    #         threshold: 0.7
    #   basketball_hoop:
    #     coordinates: 431,94,449,94,452,80,436,78
    #     filters:
    #       "sports ball":
    #         min_area: 5000
    #         max_area: 100000
    #         threshold: 0.7
    snapshots:
      enabled: true
      required_zones:
        - entryway
      retain:
        objects:
          person: 10
    record:
      enabled: true
      retain:
        days: 5
  deck: 
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/deck # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record 
        - path: 'rtsp://[redacted]@192.168.1.196:554/cam/realmonitor?channel=1&subtype=1'
          roles: 
            - detect
        # - path: 'rtsp://[redacted]@192.168.1.196:554/cam/realmonitor?channel=1&subtype=0'
        #   roles: 
        #     # - clips
        #     - record
        #     - rtmp
    detect:
      fps: 5
      height: 480
      width: 704
      enabled: true
    motion:
      mask:
        - 0,0,226,0,229,25,133,40,161,137,0,179
        - 490,0,455,104,417,85,431,0
        - 552,0,704,0,657,107,431,0
    objects:
      track:
        - person
      mask:
        - 0,0,226,0,229,25,133,40,153,118,0,149
        - 468,0,414,96,383,77,392,0
        - 552,0,640,57,611,121,431,0
  kitchen: 
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/kitchen # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record  
        - path: 'rtsp://[redacted]@192.168.1.205:554/stream2'
          roles: 
            - detect
        # - path: 'rtsp://[redacted]@192.168.1.205:554/stream1'
        #   roles: 
        #     # - clips
        #     - record
        #     - rtmp
    detect:
      fps: 5
      height: 480
      width: 704
      enabled: true
    motion:
      mask:
        - 257,96,458,81,442,0,264,0
    objects:
      track:
        - person
      filters:
        person:
          mask:
            - 257,96,458,81,442,0,264,0
    record:
      retain:
        days: 1
  loft: 
    ffmpeg:
      output_args:
        record: preset-record-generic-audio-copy
      inputs:
        - path: rtsp://127.0.0.1:8554/loft # <--- the name here must match the name of the camera in restream
          input_args: preset-rtsp-restream
          roles:
            - record   
        - path: 'rtsp://[redacted]@192.168.1.209:554/stream2'
          roles: 
            - detect
        # - path: 'rtsp://[redacted]@192.168.1.209:554/stream1'
        #   roles: 
        #     # - clips
        #     - record
        #     - rtmp
    detect:
      fps: 5
      height: 480
      width: 704
      enabled: true
    motion:
      mask:
        - 262,74,555,55,577,0,264,0
    objects:
      track:
        - person
    record:
      retain:
        days: 1
go2rtc:
  streams:
    tucker:
      - rtsp://[redacted]@192.168.1.141:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:tucker#video=h264#audio=aac"
    garage:
      - rtsp://[redacted]@192.168.1.198:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:garage#video=h264#audio=aac"      
    porch:
      - rtsp://[redacted]@192.168.1.195:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:porch#video=h264#audio=aac"
    deck:
      - rtsp://[redacted]@192.168.1.196:554/cam/realmonitor?channel=1&subtype=0
      - "ffmpeg:deck#video=h264#audio=aac"
    kitchen:
      - rtsp://[redacted]@192.168.1.205:554/stream1
      - "ffmpeg:kitchen#video=h264#audio=aac"
      # - tapo://[redacted]@192.168.1.205#:554/stream1
      # - "ffmpeg:rtsp://[redacted]@192.168.1.205:554/stream1#video=copy#audio=copy#audio=opus"
    loft:
      # - "ffmpeg:rtsp://[redacted]@192.168.1.209:554/stream1#video=copy#audio=copy#audio=opus"
      - rtsp://[redacted]@192.168.1.209:554/stream1
      - "ffmpeg:loft#video=h264#audio=aac"
      # - tapo://[redacted]@192.168.1.205#:554/stream1
objects: 
  filters: 
    person: 
      max_area: 100000
      min_area: 5000
      min_score: 0.5
      threshold: 0.6
record: 
  enabled: true
  retain:
    days: 0
  events:
    objects:
      - person
    retain: 
      default: 10
      objects: 
        person: 10

type: horizontal-stack
cards:
  - type: custom:frigate-card
    cameras:
      - camera_entity: camera.porch
        webrtc_card:
          entity: camera.porch
        live_provider: webrtc-card
        go2rtc:
          modes: []
    view:
      default: live
    menu:
      style: hover
      position: right
      button_size: 35
      alignment: top
    live:
      preload: true

also tried just this but this just shows a black page and nothing else:

type: 'custom:webrtc-camera'
entity: camera.generic_stream

If anyone sees anything that'd be super helpful. Just want the streams i see on my computers and other devices display as well on my iphones.

Answered by NickM-27

Oct 30, 2023

The stun server is not something you host, go2rtc uses Google stun server

Can read more about this in the go2rtc docs https://github.com/AlexxIT/go2rtc#module-webrtc

View full answer

NickM-27 · 2023-10-26T08:31:26Z

NickM-27
Oct 26, 2023
Collaborator Sponsor

Wanted to note that with frigate 0.13 and iOS 17.1 mse is supported.

You haven't defined WebRTC candidates in the config.

Also, "ffmpeg:tucker#video=h264#audio=aac" there is likely no reason to have the video=h264 here unless the mains streams are h265

12 replies

janstadt Oct 30, 2023
Author

Thanks @NickM-27. So i switched the frigate camera live view settings to HA Video Stream instead of WebRTC Card and now im getting a clear stream with a couple seconds of latency. I noticed that when i am off my network, the WebRTC card was trying to him my local frigate IP address with the webrtc card and thats why it was failing. Is that expected? Or should i list my public domain in the frigate configs for rtc?

NickM-27 Oct 30, 2023
Collaborator Sponsor

For WebRTC it will try all candidates until it finds one that works. Stun:8555 is where it will use a stun server to find the public ip address of your instance and then point there if the local candidate did not work.

janstadt Oct 30, 2023
Author

Does that mean i have to expose a stun server on my WAN in order for that to work or what exactly would i need to do to get that to work as expected?

NickM-27 Oct 30, 2023
Collaborator Sponsor

The stun server is not something you host, go2rtc uses Google stun server

Can read more about this in the go2rtc docs https://github.com/AlexxIT/go2rtc#module-webrtc

Answer selected by janstadt

janstadt Oct 30, 2023
Author

Much appreciated. Things seem to be working now. I am not sure what all was going on but i think some combination of changes to things have got me going. Thanks again.

f1d094 Apr 2, 2024

The stun server is not something you host, go2rtc uses Google stun server

Can read more about this in the go2rtc docs https://github.com/AlexxIT/go2rtc#module-webrtc

*Not something many people host themselves, but some do where they don't like the idea of internal services using external resources.

NOTE: go2rtc is hardcoded to use google STUN server (among other externally-facing public technologies) and you will need to build your own go2rtc to avoid this. Frigate allows the use of custom go2rtc binary.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Home Assistant iOS App and Frigate #8328

{{title}}

Replies: 1 comment 12 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Home Assistant iOS App and Frigate #8328

janstadt Oct 26, 2023

Replies: 1 comment · 12 replies

NickM-27 Oct 26, 2023 Collaborator Sponsor

janstadt Oct 30, 2023 Author

NickM-27 Oct 30, 2023 Collaborator Sponsor

janstadt Oct 30, 2023 Author

NickM-27 Oct 30, 2023 Collaborator Sponsor

janstadt Oct 30, 2023 Author

f1d094 Apr 2, 2024

janstadt
Oct 26, 2023

Replies: 1 comment 12 replies

NickM-27
Oct 26, 2023
Collaborator Sponsor

janstadt Oct 30, 2023
Author

NickM-27 Oct 30, 2023
Collaborator Sponsor

janstadt Oct 30, 2023
Author

NickM-27 Oct 30, 2023
Collaborator Sponsor

janstadt Oct 30, 2023
Author