Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'No space left on device' errors because tmpfs full #1755

Closed
jasonpstokes opened this issue Sep 10, 2021 · 66 comments
Closed

'No space left on device' errors because tmpfs full #1755

jasonpstokes opened this issue Sep 10, 2021 · 66 comments

Comments

@jasonpstokes
Copy link
Contributor

jasonpstokes commented Sep 10, 2021

Describe the bug
Frigate has crashed again after a couple days running, throwing these errors for all cameras every 10 seconds.

[2021-09-10 16:17:52] ffmpeg.Deck.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device,
[2021-09-10 16:17:52] ffmpeg.Deck.record             ERROR   : ,
[2021-09-10 16:17:52] watchdog.Deck                  INFO    : Terminating the existing ffmpeg process...,
[2021-09-10 16:17:52] watchdog.Deck                  INFO    : Waiting for ffmpeg to exit gracefully...,
[2021-09-10 16:17:52] ffmpeg.Laundry.record          ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device,
[2021-09-10 16:17:52] ffmpeg.Laundry.record          ERROR   : ,
[2021-09-10 16:17:52] watchdog.Laundry               INFO    : Terminating the existing ffmpeg process...,
[2021-09-10 16:17:52] watchdog.Laundry               INFO    : Waiting for ffmpeg to exit gracefully...,
[2021-09-10 16:18:02] ffmpeg.Entrance.record         ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device,
[2021-09-10 16:18:02] ffmpeg.Entrance.record         ERROR   : ,
[2021-09-10 16:18:02] watchdog.Entrance              INFO    : Terminating the existing ffmpeg process...,
[2021-09-10 16:18:02] watchdog.Entrance              INFO    : Waiting for ffmpeg to exit gracefully...,
[2021-09-10 16:18:02] ffmpeg.Driveway.record         ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device,
[2021-09-10 16:18:02] ffmpeg.Driveway.record         ERROR   : ,
[2021-09-10 16:18:02] watchdog.Driveway              INFO    : Terminating the existing ffmpeg process...,
[2021-09-10 16:18:02] watchdog.Driveway              INFO    : Waiting for ffmpeg to exit gracefully...,
[2021-09-10 16:18:02] ffmpeg.Rumpus.record           ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device,
[2021-09-10 16:18:02] ffmpeg.Rumpus.record           ERROR   : ,
[2021-09-10 16:18:02] watchdog.Rumpus                INFO    : Terminating the existing ffmpeg process...,
[2021-09-10 16:18:02] watchdog.Rumpus                INFO    : Waiting for ffmpeg to exit gracefully...,
[2021-09-10 16:18:02] ffmpeg.Balcony.record          ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device,
[2021-09-10 16:18:02] ffmpeg.Balcony.record          ERROR   : ,
[2021-09-10 16:18:02] watchdog.Balcony               INFO    : Terminating the existing ffmpeg process...,
[2021-09-10 16:18:02] watchdog.Balcony               INFO    : Waiting for ffmpeg to exit gracefully...,
[2021-09-10 16:18:02] ffmpeg.Deck.record             ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device,
[2021-09-10 16:18:02] ffmpeg.Deck.record             ERROR   : ,
[2021-09-10 16:18:02] watchdog.Deck                  INFO    : Terminating the existing ffmpeg process...,
[2021-09-10 16:18:02] watchdog.Deck                  INFO    : Waiting for ffmpeg to exit gracefully...,
root@frigate:/opt/frigate# df -h
Filesystem                     Size  Used Avail Use% Mounted on
overlay                        233G   20G  201G   9% /
tmpfs                           64M     0   64M   0% /dev
shm                            300M   26M  275M   9% /dev/shm
/dev/sda2                      233G   20G  201G   9% /labelmap.txt
192.168.30.8:/storage/frigate  1.8T  449G  1.4T  26% /media/frigate
tmpfs                          2.0G  2.0G     0 100% /tmp/cache
tmpfs                          7.7G     0  7.7G   0% /proc/asound
tmpfs                          7.7G     0  7.7G   0% /proc/acpi
tmpfs                          7.7G     0  7.7G   0% /sys/firmware

Version of frigate
0.9.0-6c28613

Config file

database:
  path: /media/frigate/database/frigate.db

detectors:
  coral:
    type: edgetpu
    device: usb

mqtt:
  host: 192.168.1.5
  port: 1883
  user: mqtt
  password: "{FRIGATE_MQTT_PASSWORD}"

birdseye:
  mode: continuous
  width: 1792
  height: 1344

detect:
  width: 896
  height: 672
  fps: 5

ffmpeg:
  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
  input_args: -avoid_negative_ts make_zero -flags low_delay -fflags nobuffer -fflags +genpts+discardcorrupt -rw_timeout 5000000 -strict experimental -use_wallclock_as_timestamps 1 -f live_flv
  output_args:
    record: -f segment -segment_time 30 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy # -an

motion:
  mask:
    - 265,650,265,625,630,625,630,650

record:
  enabled: true
  retain_days: 1
  events:
    enabled: true
    pre_capture: 3
    retain:
      default: 15

snapshots:
  enabled: true
  clean_copy: false
  bounding_box: true
  retain:
    default: 15

cameras:
  Driveway:
    ffmpeg:
      inputs:
        - path: http://192.168.1.71/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.71/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      filters:
        person:
          mask: 510,120,540,120,540,160,510,160
      track:
        - person
        - vehicle
        - dog
    rtmp:
      enabled: false
    timestamp_style:
      format: "%d/%m/%Y %H:%M:%S"

  Entrance:
    ffmpeg:
      inputs:
        - path: http://192.168.1.72/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.72/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
        - dog
    rtmp:
      enabled: false
    timestamp_style:
      format: "%d/%m/%Y %H:%M:%S"

  Rumpus:
    ffmpeg:
      inputs:
        - path: http://192.168.1.73/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.73/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
        - dog
    rtmp:
      enabled: false
    timestamp_style:
      format: "%d/%m/%Y %H:%M:%S"

  Deck:
    ffmpeg:
      inputs:
        - path: http://192.168.1.74/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.74/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      mask: 
        - 265,650,265,625,630,625,630,650
        - 0,585,29,592,60,617,74,672,0,672
      track:
        - person
    rtmp:
      enabled: false
    timestamp_style:
      format: "%d/%m/%Y %H:%M:%S"

  Laundry:
    ffmpeg:
      inputs:
        - path: http://192.168.1.75/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.75/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
    rtmp:
      enabled: false
    timestamp_style:
      format: "%d/%m/%Y %H:%M:%S"

  Balcony:
    ffmpeg:
      inputs:
        - path: http://192.168.1.76/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.76/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
        - vehicle
    rtmp:
      enabled: false
    timestamp_style:
      format: "%d/%m/%Y %H:%M:%S"

Frigate stats
(After restart, as Frigate's web ui was not responding)

{"Balcony":{"camera_fps":5.1,"capture_pid":256,"detection_fps":0.0,"pid":235,"process_fps":5.1,"skipped_fps":0.0},"Deck":{"camera_fps":5.1,"capture_pid":247,"detection_fps":3.2,"pid":233,"process_fps":5.1,"skipped_fps":0.0},"Driveway":{"camera_fps":5.1,"capture_pid":237,"detection_fps":0.0,"pid":228,"process_fps":5.1,"skipped_fps":0.0},"Entrance":{"camera_fps":5.1,"capture_pid":241,"detection_fps":0.0,"pid":230,"process_fps":5.1,"skipped_fps":0.0},"Laundry":{"camera_fps":5.1,"capture_pid":252,"detection_fps":0.0,"pid":234,"process_fps":5.1,"skipped_fps":0.0},"Rumpus":{"camera_fps":5.0,"capture_pid":243,"detection_fps":0.0,"pid":231,"process_fps":5.0,"skipped_fps":0.0},"detection_fps":3.2,"detectors":{"coral":{"detection_start":0.0,"inference_speed":13.15,"pid":219}},"service":{"storage":{"/dev/shm":{"free":306.5,"mount_type":"tmpfs","total":314.6,"used":8.1},"/media/frigate/clips":{"free":1447499.7,"mount_type":"nfs4","total":1921292.6,"used":473256.0},"/media/frigate/recordings":{"free":1447499.8,"mount_type":"nfs4","total":1921292.6,"used":473256.0},"/tmp/cache":{"free":1993.8,"mount_type":"tmpfs","total":2147.5,"used":153.6}},"uptime":98,"version":"0.9.0-6c28613"}}

Computer Hardware

  • OS: Debian 11
  • Install method: Docker Compose
  • Coral Version: USB
  • Network Setup: Wired

Camera Info:

  • Manufacturer: Reolink
  • Model: RLC-520A
  • Resolution: 5Mp
  • FPS: 5

docker-compose.yml

services:
  frigate:
    container_name: frigate-0.9
    image: blakeblackshear/frigate:0.9.0-6c28613-amd64
    hostname: frigate
    restart: always
    shm_size: '300m'
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/bus/usb:/dev/bus/usb
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /usr/docker/frigate-0.9/config.yml:/config/config.yml:ro
      - /usr/docker/frigate-0.9/labelmap.txt:/labelmap.txt:ro
      - /usr/docker/frigate-0.9/database:/media/frigate/database
      - /mnt/nas/frigate:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: '2gb'
    network_mode: bridge
    ports:
      - 0.0.0.0:5000:5000
    environment:
      FRIGATE_MQTT_PASSWORD: ""
      FRIGATE_USER: ""
      FRIGATE_PASSWORD: ""
      LIBVA_DRIVER_NAME: "radeonsi"
@rhatguy
Copy link

rhatguy commented Sep 10, 2021 via email

@nic0dk
Copy link

nic0dk commented Sep 10, 2021

Hi,
maybe it doesn't clean up in the tmp video clips after the "recording" is done? can you ls -lh your tmp/cache directory?

just a side question about your RLC-520A (im having one myself) i do not use record but instead clips im getting alot of

frigate | frigate.events WARNING : Unable to create clip for indkorsel and event 1631278700.555235-ht2a0j. There were no cache files for this event.

the same goes for the RLC-420 and i dont run into that problem with my hikvision, have you experience something similar?

@jasonpstokes
Copy link
Contributor Author

@rhatguy Yes I was running with 1Gb tmpfs but when this happened the first time, I increased it to 2Gb. Hadn't thought to check if it was a problem with one camera in particular, so if/when it happens again I'll do that. Usually there are only 1 or 2 files for each camera (totalling <200Mb), and so they're being written out/deleted as expected.

@nic0dk Hi, I don't think I've seen that error (all mine are RLC-520A cameras) so can't comment sorry.

@stamandr
Copy link

I'm having the same issue. It doesn't clean up in the tmp video clips after the "recording" is done. They just keep building up until 2 GB limit, then I get the same error "Could not write header for output file #0 (incorrect codec parameters ?): No space left on device,"

Is there a setting to clean up the/tmp/cache that can be set in frigate.yml ?

@blakeblackshear
Copy link
Owner

This almost certainly means your recording cleanup errored out. You should have a related error message in the logs.

@stamandr
Copy link

This almost certainly means your recording cleanup errored out. You should have a related error message in the logs.

The only errors are below and I receive dozens of these for each camera. I'm at a loss as to what to do to resolve this....

ffmpeg.FrontDoor.clips ERROR : ERROR : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
watchdog.FrontDoor INFO : Terminating the existing ffmpeg process...
watchdog.FrontDoor INFO : Waiting for ffmpeg to exit gracefully...
ffmpeg.Driveway.clips ERROR : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device

If I rm /tmp/cache/* the files all come back quickly and df -h shows full again.

root@ccab4aaf-frigate:/opt/frigate# df -h
Filesystem Size Used Avail Use% Mounted on
overlay 55G 18G 35G 35% /
tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup
/dev/root 55G 18G 35G 35% /usr/sbin/docker-init
devtmpfs 1.7G 0 1.7G 0% /dev
tmpfs 1.9G 3.7M 1.9G 1% /dev/shm
tmpfs 1.9G 1.9G 0 100% /tmp
tmpfs 1.9G 0 1.9G 0% /proc/asound
tmpfs 1.9G 0 1.9G 0% /sys/firmware

@blakeblackshear
Copy link
Owner

Can you list the contents of /tmp/cache? You should have small 10s video segments from your cameras. These should get moved out and into the recordings folder after 5 seconds.

@stamandr
Copy link

Here is part of the list...
root@ccab4aaf-frigate:/opt/frigate# ls /tmp/cache/
Driveway-20210923133542.mp4 FrontDoor-20210923133704.mp4 FrontYard-20210923133708.mp4
Driveway-20210923133552.mp4 FrontDoor-20210923133709.mp4 FrontYard-20210923133710.mp4
Driveway-20210923133601.mp4 FrontDoor-20210923133716.mp4 FrontYard-20210923133712.mp4
Driveway-20210923133612.mp4 FrontDoor-20210923133721.mp4 FrontYard-20210923133735.mp4
Driveway-20210923133622.mp4 FrontDoor-20210923133728.mp4 Roof-20210923133543.mp4
Driveway-20210923133632.mp4 FrontDoor-20210923133729.mp4 Roof-20210923133551.mp4
Driveway-20210923133642.mp4 FrontDoor-20210923133731.mp4 Roof-20210923133601.mp4
Driveway-20210923133652.mp4 FrontDoor-20210923133733.mp4 Roof-20210923133613.mp4
Driveway-20210923133702.mp4 FrontDoor-20210923133734.mp4 Roof-20210923133621.mp4
Driveway-20210923133712.mp4 FrontDoor-20210923133738.mp4 Roof-20210923133631.mp4
Driveway-20210923133722.mp4 FrontDoor-20210923133744.mp4 Roof-20210923133641.mp4
Driveway-20210923133732.mp4 FrontYard-20210923133418.mp4 Roof-20210923133651.mp4
Driveway-20210923133742.mp4 FrontYard-20210923133703.mp4 Roof-20210923133701.mp4
Driveway-20210923133752.mp4 FrontYard-20210923133705.mp4 Roof-20210923133719.mp4
FrontDoor-20210923133511.mp4 FrontYard-20210923133707.mp4

@blakeblackshear
Copy link
Owner

You need to look further back in the logs. This is still almost certainly the right answer: #1755 (comment)

@stamandr
Copy link

What specific logs do I look at ? I have this below....
logger:
default: info
logs:
custom_components.frigate: debug
frigate.mqtt: error
frigate.app: error
What should I be looking for.

Also, keep up the great work. It is much appreciated......

@blakeblackshear
Copy link
Owner

It should be a clear error message, but it will happen several minutes before the cache fills up enough to cause an error. I would remove the log config from your frigate config file and use the defaults. All the logs you have posted so far are too far after the error message.

@jasonpstokes
Copy link
Contributor Author

Jumped back to 0.8.4 today and had this issue within a few hours; have truncated a LOT of similar lines with "...".

Frigate logs
* Starting nginx nginx
 ...done.
Starting migrations
peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
peewee_migrate                 INFO    : There is nothing to migrate
frigate.mqtt                   INFO    : MQTT connected
frigate.app                    INFO    : Camera processor started for Driveway: 37
frigate.app                    INFO    : Camera processor started for Entrance: 38
frigate.app                    INFO    : Camera processor started for Rumpus: 39
frigate.app                    INFO    : Camera processor started for Deck: 41
frigate.app                    INFO    : Camera processor started for Laundry: 43
frigate.app                    INFO    : Camera processor started for Balcony: 45
frigate.app                    INFO    : Capture process started for Driveway: 46
frigate.app                    INFO    : Capture process started for Entrance: 49
frigate.app                    INFO    : Capture process started for Rumpus: 53
frigate.app                    INFO    : Capture process started for Deck: 59
frigate.app                    INFO    : Capture process started for Laundry: 60
frigate.app                    INFO    : Capture process started for Balcony: 65
detector.coral                 INFO    : Starting detection process: 34
frigate.edgetpu                INFO    : Attempting to load TPU as usb
frigate.edgetpu                INFO    : TPU found
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55d24476b140] moov atom not found
/tmp/cache/Balcony-20210927092857.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Balcony-20210927092857.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x556b05e53140] moov atom not found
/tmp/cache/Driveway-20210927092856.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Driveway-20210927092856.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55781d843140] moov atom not found
/tmp/cache/Deck-20210927092856.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Deck-20210927092856.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x560d3150c140] moov atom not found
/tmp/cache/Laundry-20210927092856.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Laundry-20210927092856.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55a33e651140] moov atom not found
/tmp/cache/Entrance-20210927092856.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Entrance-20210927092856.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55c3fa4cc140] moov atom not found
/tmp/cache/Rumpus-20210927092855.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Rumpus-20210927092855.mp4
ffmpeg.Driveway.clips          ERROR   : [segment @ 0x556e36cff140] Non-monotonous DTS in output stream 0:0; previous: 31606400, current: 31606400; changing to 31606401. This may result in incorrect timestamps in the output file.
...
ffmpeg.Driveway.clips          ERROR   : [segment @ 0x556e36cff140] Non-monotonous DTS in output stream 0:0; previous: 45778000, current: 45778000; changing to 45778001. This may result in incorrect timestamps in the output file.
ffmpeg.Driveway.clips          ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Driveway.clips          ERROR   : [segment @ 0x556e36cff140] Failure occurred when ending segment '/tmp/cache/Driveway-20210927092856.mp4'
ffmpeg.Driveway.clips          ERROR   : Error writing trailer of /tmp/cache/Driveway-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Driveway              INFO    : Terminating the existing ffmpeg process...
watchdog.Driveway              INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Rumpus.clips            ERROR   : [segment @ 0x556e78ac0440] Non-monotonous DTS in output stream 0:0; previous: 25815984, current: 25815984; changing to 25815985. This may result in incorrect timestamps in the output file.
...
ffmpeg.Rumpus.clips            ERROR   : [segment @ 0x556e78ac0440] Non-monotonous DTS in output stream 0:0; previous: 45745920, current: 45745920; changing to 45745921. This may result in incorrect timestamps in the output file.
ffmpeg.Rumpus.clips            ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Balcony.clips           ERROR   : [segment @ 0x55af69171840] Non-monotonous DTS in output stream 0:0; previous: 40211760, current: 40211760; changing to 40211761. This may result in incorrect timestamps in the output file.
...
ffmpeg.Deck.clips              ERROR   : [segment @ 0x563aee317c00] Non-monotonous DTS in output stream 0:0; previous: 38643968, current: 38643968; changing to 38643969. This may result in incorrect timestamps in the output file.
ffmpeg.Rumpus.clips            ERROR   : [segment @ 0x556e78ac0440] Failure occurred when ending segment '/tmp/cache/Rumpus-20210927092855.mp4'
ffmpeg.Entrance.clips          ERROR   : [segment @ 0x55dfa900e140] Non-monotonous DTS in output stream 0:0; previous: 37172432, current: 37172432; changing to 37172433. This may result in incorrect timestamps in the output file.
ffmpeg.Rumpus.clips            ERROR   : Error writing trailer of /tmp/cache/Rumpus-%Y%m%d%H%M%S.mp4: No space left on device
ffmpeg.Entrance.clips          ERROR   : [segment @ 0x55dfa900e140] Non-monotonous DTS in output stream 0:0; previous: 37301056, current: 37301056; changing to 37301057. This may result in incorrect timestamps in the output file.
...
ffmpeg.Balcony.clips           ERROR   : [segment @ 0x55af69171840] Non-monotonous DTS in output stream 0:0; previous: 40531632, current: 40531632; changing to 40531633. This may result in incorrect timestamps in the output file.
watchdog.Rumpus                INFO    : Terminating the existing ffmpeg process...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x55590b638180] Non-monotonous DTS in output stream 0:0; previous: 35892656, current: 35892656; changing to 35892657. This may result in incorrect timestamps in the output file.
...
ffmpeg.Balcony.clips           ERROR   : [segment @ 0x55af69171840] Non-monotonous DTS in output stream 0:0; previous: 41171344, current: 41171344; changing to 41171345. This may result in incorrect timestamps in the output file.
watchdog.Rumpus                INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x55590b638180] Non-monotonous DTS in output stream 0:0; previous: 36436768, current: 36436768; changing to 36436769. This may result in incorrect timestamps in the output file.
...
ffmpeg.Entrance.clips          ERROR   : [segment @ 0x55dfa900e140] Non-monotonous DTS in output stream 0:0; previous: 45681760, current: 45681760; changing to 45681761. This may result in incorrect timestamps in the output file.
ffmpeg.Entrance.clips          ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Entrance.clips          ERROR   : [segment @ 0x55dfa900e140] Failure occurred when ending segment '/tmp/cache/Entrance-20210927092856.mp4'
ffmpeg.Entrance.clips          ERROR   : Error writing trailer of /tmp/cache/Entrance-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Entrance              INFO    : Terminating the existing ffmpeg process...
watchdog.Entrance              INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Deck.clips              ERROR   : [segment @ 0x563aee317c00] Non-monotonous DTS in output stream 0:0; previous: 42770928, current: 42770928; changing to 42770929. This may result in incorrect timestamps in the output file.
...
ffmpeg.Deck.clips              ERROR   : [segment @ 0x563aee317c00] Non-monotonous DTS in output stream 0:0; previous: 45777728, current: 45777728; changing to 45777729. This may result in incorrect timestamps in the output file.
ffmpeg.Deck.clips              ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Deck.clips              ERROR   : [segment @ 0x563aee317c00] Failure occurred when ending segment '/tmp/cache/Deck-20210927092856.mp4'
ffmpeg.Deck.clips              ERROR   : Error writing trailer of /tmp/cache/Deck-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Deck                  INFO    : Terminating the existing ffmpeg process...
watchdog.Deck                  INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Balcony.clips           ERROR   : [segment @ 0x55af69171840] Non-monotonous DTS in output stream 0:0; previous: 44114624, current: 44114624; changing to 44114625. This may result in incorrect timestamps in the output file.
...
ffmpeg.Balcony.clips           ERROR   : [segment @ 0x55af69171840] Non-monotonous DTS in output stream 0:0; previous: 45649936, current: 45649936; changing to 45649937. This may result in incorrect timestamps in the output file.
ffmpeg.Balcony.clips           ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Balcony.clips           ERROR   : [segment @ 0x55af69171840] Failure occurred when ending segment '/tmp/cache/Balcony-20210927092857.mp4'
ffmpeg.Balcony.clips           ERROR   : Error writing trailer of /tmp/cache/Balcony-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Balcony               INFO    : Terminating the existing ffmpeg process...
watchdog.Balcony               INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x55590b638180] Non-monotonous DTS in output stream 0:0; previous: 45585712, current: 45585712; changing to 45585713. This may result in incorrect timestamps in the output file.
ffmpeg.Laundry.clips           ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x55590b638180] Failure occurred when ending segment '/tmp/cache/Laundry-20210927092856.mp4'
ffmpeg.Laundry.clips           ERROR   : Error writing trailer of /tmp/cache/Laundry-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Laundry               INFO    : Terminating the existing ffmpeg process...
watchdog.Laundry               INFO    : Waiting for ffmpeg to exit gracefully...
frigate.mqtt                   INFO    : Turning off detection for Driveway via mqtt
frigate.mqtt                   INFO    : Turning off detection for Laundry via mqtt
frigate.mqtt                   INFO    : Turning off detection for Rumpus via mqtt
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
ffmpeg.Driveway.clips          ERROR   : [segment @ 0x5561979f5480] Non-monotonous DTS in output stream 0:0; previous: 64076896, current: 64076896; changing to 64076897. This may result in incorrect timestamps in the output file.
...
ffmpeg.Driveway.clips          ERROR   : [segment @ 0x5561979f5480] Non-monotonous DTS in output stream 0:0; previous: 69547072, current: 69547072; changing to 69547073. This may result in incorrect timestamps in the output file.
ffmpeg.Driveway.clips          ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Driveway.clips          ERROR   : [segment @ 0x5561979f5480] Failure occurred when ending segment '/tmp/cache/Driveway-20210927104149.mp4'
ffmpeg.Driveway.clips          ERROR   : Error writing trailer of /tmp/cache/Driveway-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Driveway              INFO    : Terminating the existing ffmpeg process...
watchdog.Driveway              INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x55cf90b39140] Non-monotonous DTS in output stream 0:0; previous: 53839872, current: 53839872; changing to 53839873. This may result in incorrect timestamps in the output file.
...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x55cf90b39140] Non-monotonous DTS in output stream 0:0; previous: 69707440, current: 69707440; changing to 69707441. This may result in incorrect timestamps in the output file.
ffmpeg.Laundry.clips           ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x55cf90b39140] Failure occurred when ending segment '/tmp/cache/Laundry-20210927104149.mp4'
ffmpeg.Laundry.clips           ERROR   : Error writing trailer of /tmp/cache/Laundry-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Laundry               INFO    : Terminating the existing ffmpeg process...
watchdog.Laundry               INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Rumpus.clips            ERROR   : [segment @ 0x562f29144540] Non-monotonous DTS in output stream 0:0; previous: 69323120, current: 69323120; changing to 69323121. This may result in incorrect timestamps in the output file.
...
ffmpeg.Rumpus.clips            ERROR   : [segment @ 0x562f29144540] Non-monotonous DTS in output stream 0:0; previous: 69867120, current: 69867120; changing to 69867121. This may result in incorrect timestamps in the output file.
ffmpeg.Rumpus.clips            ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Rumpus.clips            ERROR   : [segment @ 0x562f29144540] Failure occurred when ending segment '/tmp/cache/Rumpus-20210927104148.mp4'
ffmpeg.Rumpus.clips            ERROR   : Error writing trailer of /tmp/cache/Rumpus-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Rumpus                INFO    : Terminating the existing ffmpeg process...
watchdog.Rumpus                INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Balcony.clips           ERROR   : [segment @ 0x56394ae37f40] Non-monotonous DTS in output stream 0:0; previous: 63660832, current: 63660832; changing to 63660833. This may result in incorrect timestamps in the output file.
...
ffmpeg.Balcony.clips           ERROR   : [segment @ 0x56394ae37f40] Non-monotonous DTS in output stream 0:0; previous: 69866944, current: 69866944; changing to 69866945. This may result in incorrect timestamps in the output file.
ffmpeg.Balcony.clips           ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Balcony.clips           ERROR   : [segment @ 0x56394ae37f40] Failure occurred when ending segment '/tmp/cache/Balcony-20210927104150.mp4'
ffmpeg.Balcony.clips           ERROR   : Error writing trailer of /tmp/cache/Balcony-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Balcony               INFO    : Terminating the existing ffmpeg process...
watchdog.Balcony               INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Deck.clips              ERROR   : [segment @ 0x56316d328140] Non-monotonous DTS in output stream 0:0; previous: 53199888, current: 53199888; changing to 53199889. This may result in incorrect timestamps in the output file.
...
ffmpeg.Deck.clips              ERROR   : [segment @ 0x56316d328140] Non-monotonous DTS in output stream 0:0; previous: 69867024, current: 69867024; changing to 69867025. This may result in incorrect timestamps in the output file.
ffmpeg.Deck.clips              ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Deck.clips              ERROR   : [segment @ 0x56316d328140] Failure occurred when ending segment '/tmp/cache/Deck-20210927104149.mp4'
ffmpeg.Deck.clips              ERROR   : Error writing trailer of /tmp/cache/Deck-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Deck                  INFO    : Terminating the existing ffmpeg process...
watchdog.Deck                  INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Entrance.clips          ERROR   : [segment @ 0x55b5d0773140] Non-monotonous DTS in output stream 0:0; previous: 51183856, current: 51183856; changing to 51183857. This may result in incorrect timestamps in the output file.
...
ffmpeg.Entrance.clips          ERROR   : [segment @ 0x55b5d0773140] Non-monotonous DTS in output stream 0:0; previous: 69834112, current: 69834112; changing to 69834113. This may result in incorrect timestamps in the output file.
ffmpeg.Entrance.clips          ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Entrance.clips          ERROR   : [segment @ 0x55b5d0773140] Failure occurred when ending segment '/tmp/cache/Entrance-20210927104148.mp4'
ffmpeg.Entrance.clips          ERROR   : Error writing trailer of /tmp/cache/Entrance-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Entrance              INFO    : Terminating the existing ffmpeg process...
watchdog.Entrance              INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Driveway.clips          ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
ffmpeg.Driveway.clips          ERROR   : 
watchdog.Driveway              INFO    : Terminating the existing ffmpeg process...
watchdog.Driveway              INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Laundry.clips           ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
ffmpeg.Laundry.clips           ERROR   : 
watchdog.Laundry               INFO    : Terminating the existing ffmpeg process...
watchdog.Laundry               INFO    : Waiting for ffmpeg to exit gracefully...
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x562594993140] moov atom not found
/tmp/cache/Entrance-20210927104204.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Entrance-20210927104204.mp4
ffmpeg.Rumpus.clips            ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
ffmpeg.Rumpus.clips            ERROR   : 
watchdog.Rumpus                INFO    : Terminating the existing ffmpeg process...
watchdog.Rumpus                INFO    : Waiting for ffmpeg to exit gracefully...
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x564f93deb140] moov atom not found
/tmp/cache/Rumpus-20210927104204.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Rumpus-20210927104204.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x555890b16140] moov atom not found
/tmp/cache/Balcony-20210927104203.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Balcony-20210927104203.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x560aeeec4140] moov atom not found
/tmp/cache/Deck-20210927104202.mp4: Invalid data found when processing input
ffmpeg.Balcony.clips           ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
ffmpeg.Balcony.clips           ERROR   : 
watchdog.Balcony               INFO    : Terminating the existing ffmpeg process...
watchdog.Balcony               INFO    : Waiting for ffmpeg to exit gracefully...
frigate.events                 INFO    : bad file: Deck-20210927104202.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55f819873140] moov atom not found
/tmp/cache/Driveway-20210927104202.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Driveway-20210927104202.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x562fdc67a140] moov atom not found
/tmp/cache/Laundry-20210927104202.mp4: Invalid data found when processing input
ffmpeg.Deck.clips              ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
ffmpeg.Deck.clips              ERROR   : 
watchdog.Deck                  INFO    : Terminating the existing ffmpeg process...
watchdog.Deck                  INFO    : Waiting for ffmpeg to exit gracefully...
frigate.events                 INFO    : bad file: Laundry-20210927104202.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55aad56a1140] moov atom not found
/tmp/cache/Balcony-20210927104150.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Balcony-20210927104150.mp4
ffmpeg.Entrance.clips          ERROR   : Could not write header for output file #0 (incorrect codec parameters ?): No space left on device
ffmpeg.Entrance.clips          ERROR   : 
watchdog.Entrance              INFO    : Terminating the existing ffmpeg process...
watchdog.Entrance              INFO    : Waiting for ffmpeg to exit gracefully...
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5586ecce7140] moov atom not found
/tmp/cache/Driveway-20210927104149.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Driveway-20210927104149.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55e6ba4b0140] moov atom not found
/tmp/cache/Deck-20210927104149.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Deck-20210927104149.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5565c264e140] moov atom not found
/tmp/cache/Laundry-20210927104149.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Laundry-20210927104149.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x5560eb259140] moov atom not found
/tmp/cache/Entrance-20210927104148.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Entrance-20210927104148.mp4
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x557f5698a140] moov atom not found
/tmp/cache/Rumpus-20210927104148.mp4: Invalid data found when processing input
frigate.events                 INFO    : bad file: Rumpus-20210927104148.mp4
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
frigate.events                 WARNING : More than 90% of the cache is used.
frigate.events                 WARNING : Consider increasing space available at /tmp/cache or reducing max_seconds in your clips config.
frigate.events                 WARNING : Proactively cleaning up the cache...
ffmpeg.Driveway.clips          ERROR   : [segment @ 0x55f383784800] Non-monotonous DTS in output stream 0:0; previous: 29878976, current: 29878976; changing to 29878977. This may result in incorrect timestamps in the output file.
...
ffmpeg.Driveway.clips          ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Driveway.clips          ERROR   : [segment @ 0x55f383784800] Failure occurred when ending segment '/tmp/cache/Driveway-20210927111912.mp4'
ffmpeg.Driveway.clips          ERROR   : Error writing trailer of /tmp/cache/Driveway-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Driveway              INFO    : Terminating the existing ffmpeg process...
watchdog.Driveway              INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x559e14f8d140] Non-monotonous DTS in output stream 0:0; previous: 26615616, current: 26615616; changing to 26615617. This may result in incorrect timestamps in the output file.
...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x559e14f8d140] Non-monotonous DTS in output stream 0:0; previous: 28887184, current: 28887184; changing to 28887185. This may result in incorrect timestamps in the output file.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x55595d7ed140] moov atom not found
/tmp/cache/Entrance-20210927111914.mp4: Invalid data found when processing input
ffmpeg.Rumpus.clips            ERROR   : [segment @ 0x557215599040] Non-monotonous DTS in output stream 0:0; previous: 18778464, current: 18778464; changing to 18778465. This may result in incorrect timestamps in the output file.
...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x559e14f8d140] Non-monotonous DTS in output stream 0:0; previous: 32086048, current: 32086048; changing to 32086049. This may result in incorrect timestamps in the output file.
frigate.events                 INFO    : bad file: Entrance-20210927111914.mp4
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x559e14f8d140] Non-monotonous DTS in output stream 0:0; previous: 32117936, current: 32117936; changing to 32117937. This may result in incorrect timestamps in the output file.
...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x559e14f8d140] Non-monotonous DTS in output stream 0:0; previous: 34037600, current: 34037600; changing to 34037601. This may result in incorrect timestamps in the output file.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x56460dcf0140] moov atom not found
/tmp/cache/Rumpus-20210927111913.mp4: Invalid data found when processing input
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x559e14f8d140] Non-monotonous DTS in output stream 0:0; previous: 34069648, current: 34069648; changing to 34069649. This may result in incorrect timestamps in the output file.
...
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x559e14f8d140] Non-monotonous DTS in output stream 0:0; previous: 35477216, current: 35477216; changing to 35477217. This may result in incorrect timestamps in the output file.
ffmpeg.Laundry.clips           ERROR   : av_interleaved_write_frame(): No space left on device
ffmpeg.Laundry.clips           ERROR   : [segment @ 0x559e14f8d140] Failure occurred when ending segment '/tmp/cache/Laundry-20210927111912.mp4'
ffmpeg.Laundry.clips           ERROR   : Error writing trailer of /tmp/cache/Laundry-%Y%m%d%H%M%S.mp4: No space left on device
watchdog.Laundry               INFO    : Terminating the existing ffmpeg process...
watchdog.Laundry               INFO    : Waiting for ffmpeg to exit gracefully...
ffmpeg.Rumpus.clips            ERROR   : [segment @ 0x557215599040] Non-monotonous DTS in output stream 0:0; previous: 18809984, current: 18809984; changing to 18809985. This may result in incorrect timestamps in the output file.
...
contents of /tmp/cache
Balcony-20210927185507.mp4
Balcony-20210927185516.mp4
Balcony-20210927185528.mp4
Balcony-20210927185538.mp4
Balcony-20210927185547.mp4
Balcony-20210927185557.mp4
Balcony-20210927185608.mp4
Balcony-20210927185618.mp4
Balcony-20210927185627.mp4
Balcony-20210927185637.mp4
Balcony-20210927185647.mp4
Balcony-20210927185658.mp4
Balcony-20210927185708.mp4
Balcony-20210927185717.mp4
Balcony-20210927185727.mp4
Balcony-20210927185737.mp4
Balcony-20210927185748.mp4
Balcony-20210927185758.mp4
Balcony-20210927185807.mp4
Balcony-20210927185817.mp4
Balcony-20210927185826.mp4
Balcony-20210927185838.mp4
Balcony-20210927185848.mp4
Balcony-20210927185857.mp4
Balcony-20210927185907.mp4
Balcony-20210927185918.mp4
Balcony-20210927185928.mp4
Balcony-20210927185937.mp4
Balcony-20210927185947.mp4
Balcony-20210927185957.mp4
Balcony-20210927190008.mp4
Balcony-20210927190018.mp4
Balcony-20210927190027.mp4
Balcony-20210927190037.mp4
Balcony-20210927190047.mp4
Balcony-20210927190058.mp4
Balcony-20210927190108.mp4
Balcony-20210927190117.mp4
Balcony-20210927190127.mp4
Balcony-20210927190136.mp4
Balcony-20210927190148.mp4
Balcony-20210927190158.mp4
Balcony-20210927190207.mp4
Balcony-20210927190217.mp4
Balcony-20210927190228.mp4
Balcony-20210927190238.mp4
Balcony-20210927190247.mp4
Balcony-20210927190257.mp4
Balcony-20210927190307.mp4
Balcony-20210927190318.mp4
Balcony-20210927190328.mp4
Balcony-20210927190337.mp4
Balcony-20210927190347.mp4
Balcony-20210927190357.mp4
Balcony-20210927190409.mp4
Balcony-20210927190421.mp4
Balcony-20210927190430.mp4
Balcony-20210927190440.mp4
Balcony-20210927190450.mp4
Balcony-20210927190501.mp4
Balcony-20210927190511.mp4
Balcony-20210927190520.mp4
Balcony-20210927190530.mp4
Balcony-20210927190540.mp4
Balcony-20210927190551.mp4
Balcony-20210927190601.mp4
Balcony-20210927190610.mp4
Balcony-20210927190620.mp4
Balcony-20210927190631.mp4
Balcony-20210927190641.mp4
Balcony-20210927190651.mp4
Balcony-20210927190700.mp4
Balcony-20210927190710.mp4
Balcony-20210927190721.mp4
Balcony-20210927190731.mp4
Balcony-20210927190740.mp4
Balcony-20210927190750.mp4
Balcony-20210927190802.mp4
Balcony-20210927190811.mp4
Balcony-20210927190821.mp4
Balcony-20210927190830.mp4
Balcony-20210927190840.mp4
Balcony-20210927190851.mp4
Balcony-20210927190901.mp4
Balcony-20210927190911.mp4
Balcony-20210927190920.mp4
Balcony-20210927190932.mp4
Balcony-20210927190941.mp4
Balcony-20210927190951.mp4
Balcony-20210927191001.mp4
Balcony-20210927191010.mp4
Balcony-20210927191022.mp4
Balcony-20210927191031.mp4
Balcony-20210927191041.mp4
Balcony-20210927191050.mp4
Balcony-20210927191102.mp4
Balcony-20210927191112.mp4
Balcony-20210927191121.mp4
Balcony-20210927191131.mp4
Balcony-20210927191142.mp4
Balcony-20210927191152.mp4
Balcony-20210927191201.mp4
Balcony-20210927191211.mp4
Balcony-20210927191221.mp4
Balcony-20210927191232.mp4
Balcony-20210927191242.mp4
Balcony-20210927191251.mp4
Balcony-20210927191301.mp4
Balcony-20210927191312.mp4
Balcony-20210927191322.mp4
Balcony-20210927191332.mp4
Balcony-20210927191341.mp4
Balcony-20210927191351.mp4
Balcony-20210927191402.mp4
Balcony-20210927191412.mp4
Balcony-20210927191422.mp4
Balcony-20210927191431.mp4
Balcony-20210927191443.mp4
Balcony-20210927191452.mp4
Balcony-20210927191502.mp4
Balcony-20210927191511.mp4
Balcony-20210927191521.mp4
Balcony-20210927191533.mp4
Balcony-20210927191542.mp4
Balcony-20210927191552.mp4
Balcony-20210927191601.mp4
Balcony-20210927191613.mp4
Balcony-20210927191622.mp4
Balcony-20210927191632.mp4
Balcony-20210927191642.mp4
Balcony-20210927191653.mp4
Balcony-20210927191703.mp4
Balcony-20210927191712.mp4
Balcony-20210927191722.mp4
Balcony-20210927191732.mp4
Balcony-20210927191743.mp4
Balcony-20210927191753.mp4
Balcony-20210927191802.mp4
Balcony-20210927191812.mp4
Balcony-20210927191823.mp4
Balcony-20210927191833.mp4
Balcony-20210927191843.mp4
Balcony-20210927191852.mp4
Balcony-20210927191902.mp4
Balcony-20210927191913.mp4
Balcony-20210927191923.mp4
Balcony-20210927191932.mp4
Balcony-20210927191942.mp4
Balcony-20210927191954.mp4
Balcony-20210927192003.mp4
Balcony-20210927192013.mp4
Balcony-20210927192022.mp4
Balcony-20210927192034.mp4
Balcony-20210927192043.mp4
Balcony-20210927192053.mp4
Balcony-20210927192103.mp4
Balcony-20210927192112.mp4
Balcony-20210927192124.mp4
Balcony-20210927192133.mp4
Balcony-20210927192143.mp4
Balcony-20210927192153.mp4
Balcony-20210927192204.mp4
Balcony-20210927192214.mp4
Balcony-20210927192223.mp4
Balcony-20210927192233.mp4
Balcony-20210927192242.mp4
Balcony-20210927192254.mp4
Balcony-20210927192304.mp4
Balcony-20210927192313.mp4
Balcony-20210927192323.mp4
Balcony-20210927192334.mp4
Balcony-20210927192344.mp4
Balcony-20210927192353.mp4
Balcony-20210927192403.mp4
Balcony-20210927192413.mp4
Balcony-20210927192424.mp4
Balcony-20210927192434.mp4
Balcony-20210927192443.mp4
Balcony-20210927192453.mp4
Balcony-20210927192504.mp4
Balcony-20210927192514.mp4
Balcony-20210927192524.mp4
Balcony-20210927192533.mp4
Balcony-20210927192545.mp4
Balcony-20210927192554.mp4
Balcony-20210927192604.mp4
Balcony-20210927192614.mp4
Balcony-20210927192623.mp4
Balcony-20210927192635.mp4
Balcony-20210927192644.mp4
Balcony-20210927192654.mp4
Balcony-20210927192703.mp4
Balcony-20210927192715.mp4
Balcony-20210927192725.mp4
Balcony-20210927192734.mp4
Balcony-20210927192744.mp4
Balcony-20210927192753.mp4
Balcony-20210927192805.mp4
Balcony-20210927192814.mp4
Balcony-20210927192824.mp4
Balcony-20210927192834.mp4
Balcony-20210927192845.mp4
Balcony-20210927192855.mp4
Balcony-20210927192904.mp4
Balcony-20210927192914.mp4
Balcony-20210927192924.mp4
Balcony-20210927192935.mp4
Balcony-20210927192945.mp4
Balcony-20210927192954.mp4
Balcony-20210927193004.mp4
Balcony-20210927193015.mp4
Balcony-20210927193025.mp4
Balcony-20210927193035.mp4
Balcony-20210927193044.mp4
Balcony-20210927193054.mp4
Balcony-20210927193105.mp4
Balcony-20210927193115.mp4
Balcony-20210927193124.mp4
Balcony-20210927193134.mp4
Balcony-20210927193146.mp4
Balcony-20210927193155.mp4
Balcony-20210927193205.mp4
Balcony-20210927193214.mp4
Balcony-20210927193226.mp4
Balcony-20210927193235.mp4
Balcony-20210927193245.mp4
Balcony-20210927193255.mp4
Balcony-20210927193304.mp4
Balcony-20210927193316.mp4
Balcony-20210927193325.mp4
Balcony-20210927193335.mp4
Balcony-20210927193345.mp4
Balcony-20210927193356.mp4
Balcony-20210927193406.mp4
Balcony-20210927193415.mp4
Balcony-20210927193425.mp4
Balcony-20210927193434.mp4
Balcony-20210927193446.mp4
Balcony-20210927193456.mp4
Balcony-20210927193505.mp4
Balcony-20210927193515.mp4
Balcony-20210927193526.mp4
Balcony-20210927193536.mp4
Balcony-20210927193545.mp4
Balcony-20210927193555.mp4
Balcony-20210927193605.mp4
Balcony-20210927193616.mp4
Balcony-20210927193626.mp4
Balcony-20210927193635.mp4
Balcony-20210927193645.mp4
Balcony-20210927193656.mp4
Balcony-20210927193706.mp4
Balcony-20210927193716.mp4
Balcony-20210927193725.mp4
Balcony-20210927193737.mp4
Balcony-20210927193746.mp4
Balcony-20210927193756.mp4
Balcony-20210927193806.mp4
Balcony-20210927193815.mp4
Balcony-20210927193827.mp4
Balcony-20210927193836.mp4
Balcony-20210927193846.mp4
Balcony-20210927193855.mp4
Balcony-20210927193907.mp4
Balcony-20210927193917.mp4
Balcony-20210927193926.mp4
Balcony-20210927193936.mp4
Balcony-20210927193945.mp4
Balcony-20210927193957.mp4
Balcony-20210927194006.mp4
Balcony-20210927194016.mp4
Balcony-20210927194026.mp4
Balcony-20210927194037.mp4
Balcony-20210927194047.mp4
Balcony-20210927194056.mp4
Balcony-20210927194106.mp4
Balcony-20210927194116.mp4
Balcony-20210927194127.mp4
Balcony-20210927194137.mp4
Balcony-20210927194146.mp4
Balcony-20210927194156.mp4
Balcony-20210927194207.mp4
Balcony-20210927194217.mp4
Balcony-20210927194227.mp4
Balcony-20210927194236.mp4
Balcony-20210927194248.mp4
Balcony-20210927194257.mp4
Balcony-20210927194307.mp4
Balcony-20210927194316.mp4
Balcony-20210927194326.mp4
Balcony-20210927194338.mp4
Balcony-20210927194347.mp4
Balcony-20210927194357.mp4
Balcony-20210927194406.mp4
Balcony-20210927194418.mp4
Balcony-20210927194427.mp4
Balcony-20210927194437.mp4
Balcony-20210927194447.mp4
Balcony-20210927194456.mp4
Balcony-20210927194508.mp4
Balcony-20210927194517.mp4
Balcony-20210927194527.mp4
Balcony-20210927194537.mp4
Balcony-20210927194548.mp4
Balcony-20210927194558.mp4
Balcony-20210927194607.mp4
Balcony-20210927194617.mp4
Balcony-20210927194628.mp4
Balcony-20210927194638.mp4
Balcony-20210927194648.mp4
Balcony-20210927194657.mp4
Balcony-20210927194707.mp4
Balcony-20210927194718.mp4
Balcony-20210927194728.mp4
Balcony-20210927194737.mp4
Balcony-20210927194747.mp4
Balcony-20210927194759.mp4
Balcony-20210927194808.mp4
Balcony-20210927194818.mp4
Balcony-20210927194827.mp4
Balcony-20210927194837.mp4
Balcony-20210927194848.mp4
Balcony-20210927194858.mp4
Balcony-20210927194908.mp4
Balcony-20210927194917.mp4
Balcony-20210927194929.mp4
Balcony-20210927194938.mp4
Balcony-20210927194948.mp4
Balcony-20210927194958.mp4
Balcony-20210927195007.mp4
Balcony-20210927195019.mp4
Balcony-20210927195028.mp4
Balcony-20210927195038.mp4
Balcony-20210927195047.mp4
Balcony-20210927195059.mp4
Balcony-20210927195109.mp4
Balcony-20210927195118.mp4
Balcony-20210927195128.mp4
Balcony-20210927195137.mp4
Balcony-20210927195149.mp4
Balcony-20210927195158.mp4
Balcony-20210927195208.mp4
Balcony-20210927195218.mp4
Balcony-20210927195229.mp4
Balcony-20210927195239.mp4
Balcony-20210927195248.mp4
Balcony-20210927195258.mp4
Balcony-20210927195309.mp4
Balcony-20210927195319.mp4
Balcony-20210927195329.mp4
Balcony-20210927195338.mp4
Balcony-20210927195348.mp4
Balcony-20210927195359.mp4
Balcony-20210927195409.mp4
Balcony-20210927195419.mp4
Balcony-20210927195428.mp4
Balcony-20210927195440.mp4
Balcony-20210927195449.mp4
Balcony-20210927195459.mp4
Balcony-20210927195508.mp4
Balcony-20210927195518.mp4
Balcony-20210927195530.mp4
Balcony-20210927195539.mp4
Balcony-20210927195549.mp4
Balcony-20210927195558.mp4
Balcony-20210927195610.mp4
Balcony-20210927195619.mp4
Balcony-20210927195629.mp4
Balcony-20210927195639.mp4
Balcony-20210927195648.mp4
Balcony-20210927195700.mp4
Balcony-20210927195709.mp4
Balcony-20210927195719.mp4
Balcony-20210927195729.mp4
Balcony-20210927195740.mp4
Balcony-20210927195750.mp4
Balcony-20210927195759.mp4
Balcony-20210927195809.mp4
Balcony-20210927195820.mp4
Balcony-20210927195830.mp4
Balcony-20210927195840.mp4
Balcony-20210927195849.mp4
Balcony-20210927195859.mp4
Balcony-20210927195910.mp4
Balcony-20210927195920.mp4
Balcony-20210927195929.mp4
Balcony-20210927195939.mp4
Balcony-20210927195951.mp4
Balcony-20210927200000.mp4
Balcony-20210927200010.mp4
Balcony-20210927200019.mp4
Balcony-20210927200029.mp4
Balcony-20210927200040.mp4
Balcony-20210927200050.mp4
Balcony-20210927200100.mp4
Balcony-20210927200109.mp4
Balcony-20210927200121.mp4
Balcony-20210927200130.mp4
Balcony-20210927200140.mp4
Balcony-20210927200150.mp4
Balcony-20210927200201.mp4
Balcony-20210927200211.mp4
Balcony-20210927200220.mp4
Balcony-20210927200230.mp4
Balcony-20210927200239.mp4
Balcony-20210927200251.mp4
Balcony-20210927200301.mp4
Balcony-20210927200310.mp4
Balcony-20210927200320.mp4
Balcony-20210927200331.mp4
Balcony-20210927200341.mp4
Balcony-20210927200350.mp4
Balcony-20210927200400.mp4
Balcony-20210927200410.mp4
Balcony-20210927200421.mp4
Balcony-20210927200431.mp4
Balcony-20210927200440.mp4
Balcony-20210927200450.mp4
Balcony-20210927200501.mp4
Balcony-20210927200511.mp4
Balcony-20210927200521.mp4
Balcony-20210927200530.mp4
Balcony-20210927200540.mp4
Balcony-20210927200551.mp4
Balcony-20210927200601.mp4
Balcony-20210927200611.mp4
Balcony-20210927200620.mp4
Balcony-20210927200632.mp4
Balcony-20210927200641.mp4
Balcony-20210927200651.mp4
Balcony-20210927200700.mp4
Balcony-20210927200712.mp4
Balcony-20210927200722.mp4
Balcony-20210927200731.mp4
Balcony-20210927200741.mp4
Balcony-20210927200750.mp4
Balcony-20210927200802.mp4
Balcony-20210927200811.mp4
Balcony-20210927200821.mp4
Balcony-20210927200831.mp4
Balcony-20210927200842.mp4
Balcony-20210927200852.mp4
Balcony-20210927200901.mp4
Balcony-20210927200911.mp4
Balcony-20210927200921.mp4
Balcony-20210927200932.mp4
Balcony-20210927200942.mp4
Balcony-20210927200951.mp4
Balcony-20210927201001.mp4
Balcony-20210927201012.mp4
Balcony-20210927201022.mp4
Balcony-20210927201032.mp4
Balcony-20210927201041.mp4
Balcony-20210927201051.mp4
Balcony-20210927201102.mp4
Balcony-20210927201112.mp4
Balcony-20210927201121.mp4
Balcony-20210927201131.mp4
Balcony-20210927201143.mp4
Balcony-20210927201152.mp4
Balcony-20210927201202.mp4
Balcony-20210927201211.mp4
Balcony-20210927201223.mp4
Balcony-20210927201232.mp4
Balcony-20210927201242.mp4
Balcony-20210927201252.mp4
Balcony-20210927201301.mp4
Balcony-20210927201313.mp4
Balcony-20210927201322.mp4
Balcony-20210927201332.mp4
Balcony-20210927201342.mp4
Balcony-20210927201353.mp4
Balcony-20210927201403.mp4
Balcony-20210927201412.mp4
Balcony-20210927201422.mp4
Balcony-20210927201431.mp4
Balcony-20210927201443.mp4
Balcony-20210927201453.mp4
Balcony-20210927201502.mp4
Balcony-20210927201512.mp4
Balcony-20210927201523.mp4
Balcony-20210927201533.mp4
Balcony-20210927201542.mp4
Balcony-20210927201552.mp4
Balcony-20210927201602.mp4
Balcony-20210927201613.mp4
Balcony-20210927201623.mp4
Balcony-20210927201632.mp4
Balcony-20210927201642.mp4
Balcony-20210927201653.mp4
Balcony-20210927201703.mp4
Balcony-20210927201713.mp4
Balcony-20210927201722.mp4
Balcony-20210927201734.mp4
Balcony-20210927201743.mp4
Balcony-20210927201753.mp4
Balcony-20210927201803.mp4
Balcony-20210927201812.mp4
Balcony-20210927201824.mp4
Balcony-20210927201833.mp4
Balcony-20210927201843.mp4
Balcony-20210927201852.mp4
Balcony-20210927201904.mp4
Balcony-20210927201914.mp4
Balcony-20210927201923.mp4
Balcony-20210927201933.mp4
Balcony-20210927201942.mp4
Balcony-20210927201954.mp4
Balcony-20210927202003.mp4
Balcony-20210927202013.mp4
Balcony-20210927202023.mp4
Balcony-20210927202034.mp4
Balcony-20210927202044.mp4
Balcony-20210927202053.mp4
Balcony-20210927202103.mp4
Balcony-20210927202113.mp4
Balcony-20210927202124.mp4
Balcony-20210927202134.mp4
Balcony-20210927202143.mp4
Balcony-20210927202153.mp4
Balcony-20210927202204.mp4
Balcony-20210927202214.mp4
Balcony-20210927202224.mp4
Balcony-20210927202233.mp4
Balcony-20210927202245.mp4
Balcony-20210927202254.mp4
Balcony-20210927202304.mp4
Balcony-20210927202313.mp4
Balcony-20210927202323.mp4
Balcony-20210927202335.mp4
Balcony-20210927202344.mp4
Balcony-20210927202354.mp4
Balcony-20210927202403.mp4
Balcony-20210927202415.mp4
Balcony-20210927202424.mp4
Balcony-20210927202434.mp4
Balcony-20210927202444.mp4
Balcony-20210927202453.mp4
Balcony-20210927202505.mp4
Balcony-20210927202514.mp4
Balcony-20210927202524.mp4
Balcony-20210927202534.mp4
Balcony-20210927202545.mp4
Balcony-20210927202555.mp4
Balcony-20210927202604.mp4
Balcony-20210927202614.mp4
Deck-20210927185508.mp4
Deck-20210927185518.mp4
Deck-20210927185529.mp4
Deck-20210927185539.mp4
Deck-20210927185548.mp4
Deck-20210927185558.mp4
Deck-20210927185608.mp4
Deck-20210927185619.mp4
Deck-20210927185629.mp4
Deck-20210927185638.mp4
Deck-20210927185648.mp4
Deck-20210927185659.mp4
Deck-20210927185709.mp4
Deck-20210927185719.mp4
Deck-20210927185728.mp4
Deck-20210927185738.mp4
Deck-20210927185749.mp4
Deck-20210927185759.mp4
Deck-20210927185808.mp4
Deck-20210927185818.mp4
Deck-20210927185828.mp4
Deck-20210927185839.mp4
Deck-20210927185849.mp4
Deck-20210927185858.mp4
Deck-20210927185908.mp4
Deck-20210927185918.mp4
Deck-20210927185929.mp4
Deck-20210927185939.mp4
Deck-20210927185948.mp4
Deck-20210927185958.mp4
Deck-20210927190009.mp4
Deck-20210927190019.mp4
Deck-20210927190029.mp4
Deck-20210927190038.mp4
Deck-20210927190048.mp4
Deck-20210927190059.mp4
Deck-20210927190109.mp4
Deck-20210927190118.mp4
Deck-20210927190128.mp4
Deck-20210927190138.mp4
Deck-20210927190149.mp4
Deck-20210927190159.mp4
Deck-20210927190208.mp4
Deck-20210927190218.mp4
Deck-20210927190227.mp4
Deck-20210927190239.mp4
Deck-20210927190249.mp4
Deck-20210927190258.mp4
Deck-20210927190308.mp4
Deck-20210927190319.mp4
Deck-20210927190329.mp4
Deck-20210927190339.mp4
Deck-20210927190348.mp4
Deck-20210927190410.mp4
Deck-20210927190420.mp4
Deck-20210927190430.mp4
Deck-20210927190439.mp4
Deck-20210927190451.mp4
Deck-20210927190500.mp4
Deck-20210927190510.mp4
Deck-20210927190520.mp4
Deck-20210927190531.mp4
Deck-20210927190541.mp4
Deck-20210927190550.mp4
Deck-20210927190600.mp4
Deck-20210927190609.mp4
Deck-20210927190621.mp4
Deck-20210927190631.mp4
Deck-20210927190640.mp4
Deck-20210927190650.mp4
Deck-20210927190701.mp4
Deck-20210927190711.mp4
Deck-20210927190720.mp4
Deck-20210927190730.mp4
Deck-20210927190740.mp4
Deck-20210927190751.mp4
Deck-20210927190801.mp4
Deck-20210927190810.mp4
Deck-20210927190820.mp4
Deck-20210927190831.mp4
Deck-20210927190841.mp4
Deck-20210927190851.mp4
Deck-20210927190900.mp4
Deck-20210927190910.mp4
Deck-20210927190921.mp4
Deck-20210927190931.mp4
Deck-20210927190941.mp4
Deck-20210927190950.mp4
Deck-20210927191002.mp4
Deck-20210927191011.mp4
Deck-20210927191021.mp4
Deck-20210927191030.mp4
Deck-20210927191042.mp4
Deck-20210927191052.mp4
Deck-20210927191101.mp4
Deck-20210927191111.mp4
Deck-20210927191120.mp4
Deck-20210927191132.mp4
Deck-20210927191141.mp4
Deck-20210927191151.mp4
Deck-20210927191201.mp4
Deck-20210927191212.mp4
Deck-20210927191222.mp4
Deck-20210927191231.mp4
Deck-20210927191241.mp4
Deck-20210927191251.mp4
Deck-20210927191302.mp4
Deck-20210927191312.mp4
Deck-20210927191321.mp4
Deck-20210927191331.mp4
Deck-20210927191342.mp4
Deck-20210927191352.mp4
Deck-20210927191402.mp4
Deck-20210927191411.mp4
Deck-20210927191421.mp4
Deck-20210927191432.mp4
Deck-20210927191442.mp4
Deck-20210927191451.mp4
Deck-20210927191501.mp4
Deck-20210927191513.mp4
Deck-20210927191522.mp4
Deck-20210927191532.mp4
Deck-20210927191541.mp4
Deck-20210927191553.mp4
Deck-20210927191602.mp4
Deck-20210927191612.mp4
Deck-20210927191622.mp4
Deck-20210927191631.mp4
Deck-20210927191643.mp4
Deck-20210927191652.mp4
Deck-20210927191702.mp4
Deck-20210927191712.mp4
Deck-20210927191723.mp4
Deck-20210927191733.mp4
Deck-20210927191742.mp4
Deck-20210927191752.mp4
Deck-20210927191801.mp4
Deck-20210927191813.mp4
Deck-20210927191823.mp4
Deck-20210927191832.mp4
Deck-20210927191842.mp4
Deck-20210927191853.mp4
Deck-20210927191903.mp4
Deck-20210927191912.mp4
Deck-20210927191922.mp4
Deck-20210927191934.mp4
Deck-20210927191943.mp4
Deck-20210927191953.mp4
Deck-20210927192002.mp4
Deck-20210927192012.mp4
Deck-20210927192023.mp4
Deck-20210927192033.mp4
Deck-20210927192043.mp4
Deck-20210927192052.mp4
Deck-20210927192104.mp4
Deck-20210927192113.mp4
Deck-20210927192123.mp4
Deck-20210927192133.mp4
Deck-20210927192142.mp4
Deck-20210927192154.mp4
Deck-20210927192203.mp4
Deck-20210927192213.mp4
Deck-20210927192222.mp4
Deck-20210927192234.mp4
Deck-20210927192244.mp4
Deck-20210927192253.mp4
Deck-20210927192303.mp4
Deck-20210927192312.mp4
Deck-20210927192324.mp4
Deck-20210927192333.mp4
Deck-20210927192343.mp4
Deck-20210927192353.mp4
Deck-20210927192404.mp4
Deck-20210927192414.mp4
Deck-20210927192423.mp4
Deck-20210927192433.mp4
Deck-20210927192444.mp4
Deck-20210927192454.mp4
Deck-20210927192504.mp4
Deck-20210927192513.mp4
Deck-20210927192523.mp4
Deck-20210927192534.mp4
Deck-20210927192544.mp4
Deck-20210927192554.mp4
Deck-20210927192603.mp4
Deck-20210927192615.mp4
Deck-20210927192624.mp4
Deck-20210927192634.mp4
Deck-20210927192643.mp4
Deck-20210927192653.mp4
Deck-20210927192705.mp4
Deck-20210927192714.mp4
Deck-20210927192724.mp4
Deck-20210927192733.mp4
Deck-20210927192745.mp4
Deck-20210927192754.mp4
Deck-20210927192804.mp4
Deck-20210927192814.mp4
Deck-20210927192823.mp4
Deck-20210927192835.mp4
Deck-20210927192844.mp4
Deck-20210927192854.mp4
Deck-20210927192904.mp4
Deck-20210927192915.mp4
Deck-20210927192925.mp4
Deck-20210927192934.mp4
Deck-20210927192944.mp4
Deck-20210927192955.mp4
Deck-20210927193005.mp4
Deck-20210927193015.mp4
Deck-20210927193024.mp4
Deck-20210927193034.mp4
Deck-20210927193045.mp4
Deck-20210927193055.mp4
Deck-20210927193104.mp4
Deck-20210927193114.mp4
Deck-20210927193126.mp4
Deck-20210927193135.mp4
Deck-20210927193145.mp4
Deck-20210927193154.mp4
Deck-20210927193204.mp4
Deck-20210927193215.mp4
Deck-20210927193225.mp4
Deck-20210927193235.mp4
Deck-20210927193244.mp4
Deck-20210927193256.mp4
Deck-20210927193305.mp4
Deck-20210927193315.mp4
Deck-20210927193325.mp4
Deck-20210927193334.mp4
Deck-20210927193346.mp4
Deck-20210927193355.mp4
Deck-20210927193405.mp4
Deck-20210927193414.mp4
Deck-20210927193426.mp4
Deck-20210927193436.mp4
Deck-20210927193445.mp4
Deck-20210927193455.mp4
Deck-20210927193506.mp4
Deck-20210927193516.mp4
Deck-20210927193525.mp4
Deck-20210927193535.mp4
Deck-20210927193545.mp4
Deck-20210927193556.mp4
Deck-20210927193606.mp4
Deck-20210927193615.mp4
Deck-20210927193625.mp4
Deck-20210927193636.mp4
Deck-20210927193646.mp4
Deck-20210927193656.mp4
Deck-20210927193705.mp4
Deck-20210927193715.mp4
Deck-20210927193726.mp4
Deck-20210927193736.mp4
Deck-20210927193746.mp4
Deck-20210927193755.mp4
Deck-20210927193807.mp4
Deck-20210927193816.mp4
Deck-20210927193826.mp4
Deck-20210927193835.mp4
Deck-20210927193847.mp4
Deck-20210927193857.mp4
Deck-20210927193906.mp4
Deck-20210927193916.mp4
Deck-20210927193925.mp4
Deck-20210927193937.mp4
Deck-20210927193946.mp4
Deck-20210927193956.mp4
Deck-20210927194006.mp4
Deck-20210927194017.mp4
Deck-20210927194027.mp4
Deck-20210927194036.mp4
Deck-20210927194046.mp4
Deck-20210927194056.mp4
Deck-20210927194107.mp4
Deck-20210927194117.mp4
Deck-20210927194126.mp4
Deck-20210927194136.mp4
Deck-20210927194147.mp4
Deck-20210927194157.mp4
Deck-20210927194207.mp4
Deck-20210927194216.mp4
Deck-20210927194226.mp4
Deck-20210927194237.mp4
Deck-20210927194247.mp4
Deck-20210927194256.mp4
Deck-20210927194306.mp4
Deck-20210927194318.mp4
Deck-20210927194327.mp4
Deck-20210927194337.mp4
Deck-20210927194346.mp4
Deck-20210927194356.mp4
Deck-20210927194407.mp4
Deck-20210927194417.mp4
Deck-20210927194427.mp4
Deck-20210927194436.mp4
Deck-20210927194448.mp4
Deck-20210927194457.mp4
Deck-20210927194507.mp4
Deck-20210927194517.mp4
Deck-20210927194528.mp4
Deck-20210927194538.mp4
Deck-20210927194547.mp4
Deck-20210927194557.mp4
Deck-20210927194606.mp4
Deck-20210927194618.mp4
Deck-20210927194628.mp4
Deck-20210927194637.mp4
Deck-20210927194647.mp4
Deck-20210927194658.mp4
Deck-20210927194708.mp4
Deck-20210927194717.mp4
Deck-20210927194727.mp4
Deck-20210927194737.mp4
Deck-20210927194748.mp4
Deck-20210927194758.mp4
Deck-20210927194807.mp4
Deck-20210927194817.mp4
Deck-20210927194828.mp4
Deck-20210927194838.mp4
Deck-20210927194848.mp4
Deck-20210927194857.mp4
Deck-20210927194907.mp4
Deck-20210927194918.mp4
Deck-20210927194928.mp4
Deck-20210927194938.mp4
Deck-20210927194947.mp4
Deck-20210927194959.mp4
Deck-20210927195008.mp4
Deck-20210927195018.mp4
Deck-20210927195027.mp4
Deck-20210927195039.mp4
Deck-20210927195049.mp4
Deck-20210927195058.mp4
Deck-20210927195108.mp4
Deck-20210927195117.mp4
Deck-20210927195129.mp4
Deck-20210927195138.mp4
Deck-20210927195148.mp4
Deck-20210927195158.mp4
Deck-20210927195209.mp4
Deck-20210927195219.mp4
Deck-20210927195228.mp4
Deck-20210927195238.mp4
Deck-20210927195248.mp4
Deck-20210927195259.mp4
Deck-20210927195309.mp4
Deck-20210927195318.mp4
Deck-20210927195328.mp4
Deck-20210927195339.mp4
Deck-20210927195349.mp4
Deck-20210927195359.mp4
Deck-20210927195408.mp4
Deck-20210927195420.mp4
Deck-20210927195429.mp4
Deck-20210927195439.mp4
Deck-20210927195448.mp4
Deck-20210927195458.mp4
Deck-20210927195510.mp4
Deck-20210927195519.mp4
Deck-20210927195529.mp4
Deck-20210927195538.mp4
Deck-20210927195550.mp4
Deck-20210927195559.mp4
Deck-20210927195609.mp4
Deck-20210927195619.mp4
Deck-20210927195628.mp4
Deck-20210927195640.mp4
Deck-20210927195649.mp4
Deck-20210927195659.mp4
Deck-20210927195709.mp4
Deck-20210927195720.mp4
Deck-20210927195730.mp4
Deck-20210927195739.mp4
Deck-20210927195749.mp4
Deck-20210927195758.mp4
Deck-20210927195810.mp4
Deck-20210927195820.mp4
Deck-20210927195829.mp4
Deck-20210927195839.mp4
Deck-20210927195850.mp4
Deck-20210927195900.mp4
Deck-20210927195909.mp4
Deck-20210927195919.mp4
Deck-20210927195929.mp4
Deck-20210927195940.mp4
Deck-20210927195950.mp4
Deck-20210927195959.mp4
Deck-20210927200009.mp4
Deck-20210927200020.mp4
Deck-20210927200030.mp4
Deck-20210927200040.mp4
Deck-20210927200049.mp4
Deck-20210927200101.mp4
Deck-20210927200110.mp4
Deck-20210927200120.mp4
Deck-20210927200130.mp4
Deck-20210927200139.mp4
Deck-20210927200151.mp4
Deck-20210927200200.mp4
Deck-20210927200210.mp4
Deck-20210927200219.mp4
Deck-20210927200231.mp4
Deck-20210927200241.mp4
Deck-20210927200250.mp4
Deck-20210927200300.mp4
Deck-20210927200309.mp4
Deck-20210927200321.mp4
Deck-20210927200330.mp4
Deck-20210927200340.mp4
Deck-20210927200350.mp4
Deck-20210927200401.mp4
Deck-20210927200411.mp4
Deck-20210927200420.mp4
Deck-20210927200430.mp4
Deck-20210927200441.mp4
Deck-20210927200451.mp4
Deck-20210927200501.mp4
Deck-20210927200510.mp4
Deck-20210927200520.mp4
Deck-20210927200531.mp4
Deck-20210927200541.mp4
Deck-20210927200551.mp4
Deck-20210927200600.mp4
Deck-20210927200612.mp4
Deck-20210927200621.mp4
Deck-20210927200631.mp4
Deck-20210927200640.mp4
Deck-20210927200650.mp4
Deck-20210927200702.mp4
Deck-20210927200711.mp4
Deck-20210927200721.mp4
Deck-20210927200730.mp4
Deck-20210927200742.mp4
Deck-20210927200751.mp4
Deck-20210927200801.mp4
Deck-20210927200811.mp4
Deck-20210927200820.mp4
Deck-20210927200832.mp4
Deck-20210927200841.mp4
Deck-20210927200851.mp4
Deck-20210927200901.mp4
Deck-20210927200912.mp4
Deck-20210927200922.mp4
Deck-20210927200931.mp4
Deck-20210927200941.mp4
Deck-20210927200952.mp4
Deck-20210927201002.mp4
Deck-20210927201012.mp4
Deck-20210927201021.mp4
Deck-20210927201031.mp4
Deck-20210927201042.mp4
Deck-20210927201052.mp4
Deck-20210927201101.mp4
Deck-20210927201111.mp4
Deck-20210927201123.mp4
Deck-20210927201132.mp4
Deck-20210927201142.mp4
Deck-20210927201151.mp4
Deck-20210927201201.mp4
Deck-20210927201212.mp4
Deck-20210927201222.mp4
Deck-20210927201232.mp4
Deck-20210927201241.mp4
Deck-20210927201253.mp4
Deck-20210927201302.mp4
Deck-20210927201312.mp4
Deck-20210927201322.mp4
Deck-20210927201331.mp4
Deck-20210927201343.mp4
Deck-20210927201352.mp4
Deck-20210927201402.mp4
Deck-20210927201411.mp4
Deck-20210927201423.mp4
Deck-20210927201433.mp4
Deck-20210927201442.mp4
Deck-20210927201452.mp4
Deck-20210927201503.mp4
Deck-20210927201513.mp4
Deck-20210927201523.mp4
Deck-20210927201532.mp4
Deck-20210927201542.mp4
Deck-20210927201553.mp4
Deck-20210927201603.mp4
Deck-20210927201612.mp4
Deck-20210927201622.mp4
Deck-20210927201633.mp4
Deck-20210927201643.mp4
Deck-20210927201653.mp4
Deck-20210927201702.mp4
Deck-20210927201712.mp4
Deck-20210927201723.mp4
Deck-20210927201733.mp4
Deck-20210927201743.mp4
Deck-20210927201752.mp4
Deck-20210927201804.mp4
Deck-20210927201813.mp4
Deck-20210927201823.mp4
Deck-20210927201832.mp4
Deck-20210927201842.mp4
Deck-20210927201854.mp4
Deck-20210927201903.mp4
Deck-20210927201913.mp4
Deck-20210927201922.mp4
Deck-20210927201934.mp4
Deck-20210927201943.mp4
Deck-20210927201953.mp4
Deck-20210927202003.mp4
Deck-20210927202014.mp4
Deck-20210927202024.mp4
Deck-20210927202033.mp4
Deck-20210927202043.mp4
Deck-20210927202053.mp4
Deck-20210927202104.mp4
Deck-20210927202114.mp4
Deck-20210927202123.mp4
Deck-20210927202133.mp4
Deck-20210927202144.mp4
Deck-20210927202154.mp4
Deck-20210927202204.mp4
Deck-20210927202213.mp4
Deck-20210927202223.mp4
Deck-20210927202234.mp4
Deck-20210927202244.mp4
Deck-20210927202253.mp4
Deck-20210927202303.mp4
Deck-20210927202315.mp4
Deck-20210927202324.mp4
Deck-20210927202334.mp4
Deck-20210927202343.mp4
Deck-20210927202353.mp4
Deck-20210927202405.mp4
Deck-20210927202414.mp4
Deck-20210927202424.mp4
Deck-20210927202433.mp4
Deck-20210927202445.mp4
Deck-20210927202454.mp4
Deck-20210927202504.mp4
Deck-20210927202514.mp4
Deck-20210927202525.mp4
Deck-20210927202535.mp4
Deck-20210927202544.mp4
Deck-20210927202554.mp4
Deck-20210927202603.mp4
Deck-20210927202615.mp4
Driveway-20210927185507.mp4
Driveway-20210927185516.mp4
Driveway-20210927185526.mp4
Driveway-20210927185537.mp4
Driveway-20210927185547.mp4
Driveway-20210927185556.mp4
Driveway-20210927185606.mp4
Driveway-20210927185616.mp4
Driveway-20210927185627.mp4
Driveway-20210927185637.mp4
Driveway-20210927185646.mp4
Driveway-20210927185656.mp4
Driveway-20210927185706.mp4
Driveway-20210927185717.mp4
Driveway-20210927185727.mp4
Driveway-20210927185736.mp4
Driveway-20210927185746.mp4
Driveway-20210927185757.mp4
Driveway-20210927185807.mp4
Driveway-20210927185817.mp4
Driveway-20210927185826.mp4
Driveway-20210927185836.mp4
Driveway-20210927185847.mp4
Driveway-20210927185857.mp4
Driveway-20210927185906.mp4
Driveway-20210927185916.mp4
Driveway-20210927185926.mp4
Driveway-20210927185937.mp4
Driveway-20210927185947.mp4
Driveway-20210927185956.mp4
Driveway-20210927190006.mp4
Driveway-20210927190016.mp4
Driveway-20210927190027.mp4
Driveway-20210927190037.mp4
Driveway-20210927190046.mp4
Driveway-20210927190056.mp4
Driveway-20210927190107.mp4
Driveway-20210927190117.mp4
Driveway-20210927190127.mp4
Driveway-20210927190136.mp4
Driveway-20210927190146.mp4
Driveway-20210927190157.mp4
Driveway-20210927190207.mp4
Driveway-20210927190216.mp4
Driveway-20210927190226.mp4
Driveway-20210927190236.mp4
Driveway-20210927190247.mp4
Driveway-20210927190257.mp4
Driveway-20210927190306.mp4
Driveway-20210927190316.mp4
Driveway-20210927190326.mp4
Driveway-20210927190337.mp4
Driveway-20210927190347.mp4
Driveway-20210927190356.mp4
Driveway-20210927190409.mp4
Driveway-20210927190419.mp4
Driveway-20210927190430.mp4
Driveway-20210927190440.mp4
Driveway-20210927190449.mp4
Driveway-20210927190459.mp4
Driveway-20210927190510.mp4
Driveway-20210927190520.mp4
Driveway-20210927190530.mp4
Driveway-20210927190539.mp4
Driveway-20210927190551.mp4
Driveway-20210927190600.mp4
Driveway-20210927190610.mp4
Driveway-20210927190619.mp4
Driveway-20210927190629.mp4
Driveway-20210927190641.mp4
Driveway-20210927190650.mp4
Driveway-20210927190700.mp4
Driveway-20210927190709.mp4
Driveway-20210927190721.mp4
Driveway-20210927190731.mp4
Driveway-20210927190740.mp4
Driveway-20210927190750.mp4
Driveway-20210927190759.mp4
Driveway-20210927190811.mp4
Driveway-20210927190820.mp4
Driveway-20210927190830.mp4
Driveway-20210927190840.mp4
Driveway-20210927190851.mp4
Driveway-20210927190901.mp4
Driveway-20210927190910.mp4
Driveway-20210927190920.mp4
Driveway-20210927190929.mp4
Driveway-20210927190941.mp4
Driveway-20210927190951.mp4
Driveway-20210927191000.mp4
Driveway-20210927191010.mp4
Driveway-20210927191021.mp4
Driveway-20210927191031.mp4
Driveway-20210927191040.mp4
Driveway-20210927191050.mp4
Driveway-20210927191102.mp4
Driveway-20210927191111.mp4
Driveway-20210927191121.mp4
Driveway-20210927191130.mp4
Driveway-20210927191140.mp4
Driveway-20210927191152.mp4
Driveway-20210927191201.mp4
Driveway-20210927191211.mp4
Driveway-20210927191220.mp4
Driveway-20210927191232.mp4
Driveway-20210927191241.mp4
Driveway-20210927191251.mp4
Driveway-20210927191301.mp4
Driveway-20210927191310.mp4
Driveway-20210927191322.mp4
Driveway-20210927191331.mp4
Driveway-20210927191341.mp4
Driveway-20210927191351.mp4
Driveway-20210927191402.mp4
Driveway-20210927191412.mp4
Driveway-20210927191421.mp4
Driveway-20210927191431.mp4
Driveway-20210927191440.mp4
Driveway-20210927191452.mp4
Driveway-20210927191502.mp4
Driveway-20210927191511.mp4
Driveway-20210927191521.mp4
Driveway-20210927191532.mp4
Driveway-20210927191542.mp4
Driveway-20210927191551.mp4
Driveway-20210927191601.mp4
Driveway-20210927191613.mp4
Driveway-20210927191622.mp4
Driveway-20210927191632.mp4
Driveway-20210927191641.mp4
Driveway-20210927191651.mp4
Driveway-20210927191702.mp4
Driveway-20210927191712.mp4
Driveway-20210927191722.mp4
Driveway-20210927191731.mp4
Driveway-20210927191743.mp4
Driveway-20210927191752.mp4
Driveway-20210927191802.mp4
Driveway-20210927191812.mp4
Driveway-20210927191821.mp4
Driveway-20210927191833.mp4
Driveway-20210927191842.mp4
Driveway-20210927191852.mp4
Driveway-20210927191901.mp4
Driveway-20210927191913.mp4
Driveway-20210927191923.mp4
Driveway-20210927191932.mp4
Driveway-20210927191942.mp4
Driveway-20210927191951.mp4
Driveway-20210927192003.mp4
Driveway-20210927192012.mp4
Driveway-20210927192022.mp4
Driveway-20210927192032.mp4
Driveway-20210927192043.mp4
Driveway-20210927192053.mp4
Driveway-20210927192102.mp4
Driveway-20210927192112.mp4
Driveway-20210927192123.mp4
Driveway-20210927192133.mp4
Driveway-20210927192143.mp4
Driveway-20210927192152.mp4
Driveway-20210927192202.mp4
Driveway-20210927192213.mp4
Driveway-20210927192223.mp4
Driveway-20210927192233.mp4
Driveway-20210927192242.mp4
Driveway-20210927192254.mp4
Driveway-20210927192303.mp4
Driveway-20210927192313.mp4
Driveway-20210927192322.mp4
Driveway-20210927192332.mp4
Driveway-20210927192343.mp4
Driveway-20210927192353.mp4
Driveway-20210927192403.mp4
Driveway-20210927192412.mp4
Driveway-20210927192424.mp4
Driveway-20210927192433.mp4
Driveway-20210927192443.mp4
Driveway-20210927192453.mp4
Driveway-20210927192502.mp4
Driveway-20210927192514.mp4
Driveway-20210927192523.mp4
Driveway-20210927192533.mp4
Driveway-20210927192542.mp4
Driveway-20210927192554.mp4
Driveway-20210927192604.mp4
Driveway-20210927192613.mp4
Driveway-20210927192623.mp4
Driveway-20210927192634.mp4
Driveway-20210927192644.mp4
Driveway-20210927192654.mp4
Driveway-20210927192703.mp4
Driveway-20210927192713.mp4
Driveway-20210927192724.mp4
Driveway-20210927192734.mp4
Driveway-20210927192743.mp4
Driveway-20210927192753.mp4
Driveway-20210927192805.mp4
Driveway-20210927192814.mp4
Driveway-20210927192824.mp4
Driveway-20210927192833.mp4
Driveway-20210927192843.mp4
Driveway-20210927192854.mp4
Driveway-20210927192904.mp4
Driveway-20210927192914.mp4
Driveway-20210927192923.mp4
Driveway-20210927192935.mp4
Driveway-20210927192944.mp4
Driveway-20210927192954.mp4
Driveway-20210927193004.mp4
Driveway-20210927193015.mp4
Driveway-20210927193025.mp4
Driveway-20210927193034.mp4
Driveway-20210927193044.mp4
Driveway-20210927193053.mp4
Driveway-20210927193105.mp4
Driveway-20210927193115.mp4
Driveway-20210927193124.mp4
Driveway-20210927193134.mp4
Driveway-20210927193145.mp4
Driveway-20210927193155.mp4
Driveway-20210927193204.mp4
Driveway-20210927193214.mp4
Driveway-20210927193224.mp4
Driveway-20210927193235.mp4
Driveway-20210927193245.mp4
Driveway-20210927193254.mp4
Driveway-20210927193304.mp4
Driveway-20210927193315.mp4
Driveway-20210927193325.mp4
Driveway-20210927193335.mp4
Driveway-20210927193344.mp4
Driveway-20210927193354.mp4
Driveway-20210927193405.mp4
Driveway-20210927193415.mp4
Driveway-20210927193424.mp4
Driveway-20210927193434.mp4
Driveway-20210927193446.mp4
Driveway-20210927193455.mp4
Driveway-20210927193505.mp4
Driveway-20210927193514.mp4
Driveway-20210927193524.mp4
Driveway-20210927193536.mp4
Driveway-20210927193545.mp4
Driveway-20210927193555.mp4
Driveway-20210927193604.mp4
Driveway-20210927193616.mp4
Driveway-20210927193625.mp4
Driveway-20210927193635.mp4
Driveway-20210927193645.mp4
Driveway-20210927193656.mp4
Driveway-20210927193706.mp4
Driveway-20210927193715.mp4
Driveway-20210927193725.mp4
Driveway-20210927193735.mp4
Driveway-20210927193746.mp4
Driveway-20210927193756.mp4
Driveway-20210927193805.mp4
Driveway-20210927193815.mp4
Driveway-20210927193826.mp4
Driveway-20210927193836.mp4
Driveway-20210927193846.mp4
Driveway-20210927193855.mp4
Driveway-20210927193905.mp4
Driveway-20210927193916.mp4
Driveway-20210927193926.mp4
Driveway-20210927193935.mp4
Driveway-20210927193945.mp4
Driveway-20210927193957.mp4
Driveway-20210927194006.mp4
Driveway-20210927194016.mp4
Driveway-20210927194025.mp4
Driveway-20210927194037.mp4
Driveway-20210927194046.mp4
Driveway-20210927194056.mp4
Driveway-20210927194106.mp4
Driveway-20210927194115.mp4
Driveway-20210927194127.mp4
Driveway-20210927194136.mp4
Driveway-20210927194146.mp4
Driveway-20210927194156.mp4
Driveway-20210927194207.mp4
Driveway-20210927194217.mp4
Driveway-20210927194226.mp4
Driveway-20210927194236.mp4
Driveway-20210927194245.mp4
Driveway-20210927194257.mp4
Driveway-20210927194307.mp4
Driveway-20210927194316.mp4
Driveway-20210927194326.mp4
Driveway-20210927194337.mp4
Driveway-20210927194347.mp4
Driveway-20210927194356.mp4
Driveway-20210927194406.mp4
Driveway-20210927194416.mp4
Driveway-20210927194427.mp4
Driveway-20210927194437.mp4
Driveway-20210927194446.mp4
Driveway-20210927194456.mp4
Driveway-20210927194507.mp4
Driveway-20210927194517.mp4
Driveway-20210927194527.mp4
Driveway-20210927194536.mp4
Driveway-20210927194548.mp4
Driveway-20210927194557.mp4
Driveway-20210927194607.mp4
Driveway-20210927194617.mp4
Driveway-20210927194626.mp4
Driveway-20210927194638.mp4
Driveway-20210927194647.mp4
Driveway-20210927194657.mp4
Driveway-20210927194706.mp4
Driveway-20210927194718.mp4
Driveway-20210927194728.mp4
Driveway-20210927194737.mp4
Driveway-20210927194747.mp4
Driveway-20210927194756.mp4
Driveway-20210927194808.mp4
Driveway-20210927194817.mp4
Driveway-20210927194827.mp4
Driveway-20210927194837.mp4
Driveway-20210927194848.mp4
Driveway-20210927194858.mp4
Driveway-20210927194907.mp4
Driveway-20210927194917.mp4
Driveway-20210927194927.mp4
Driveway-20210927194938.mp4
Driveway-20210927194948.mp4
Driveway-20210927194957.mp4
Driveway-20210927195007.mp4
Driveway-20210927195018.mp4
Driveway-20210927195028.mp4
Driveway-20210927195038.mp4
Driveway-20210927195047.mp4
Driveway-20210927195059.mp4
Driveway-20210927195108.mp4
Driveway-20210927195118.mp4
Driveway-20210927195127.mp4
Driveway-20210927195137.mp4
Driveway-20210927195149.mp4
Driveway-20210927195158.mp4
Driveway-20210927195208.mp4
Driveway-20210927195217.mp4
Driveway-20210927195229.mp4
Driveway-20210927195238.mp4
Driveway-20210927195248.mp4
Driveway-20210927195258.mp4
Driveway-20210927195307.mp4
Driveway-20210927195319.mp4
Driveway-20210927195328.mp4
Driveway-20210927195338.mp4
Driveway-20210927195348.mp4
Driveway-20210927195359.mp4
Driveway-20210927195409.mp4
Driveway-20210927195418.mp4
Driveway-20210927195428.mp4
Driveway-20210927195437.mp4
Driveway-20210927195449.mp4
Driveway-20210927195459.mp4
Driveway-20210927195508.mp4
Driveway-20210927195518.mp4
Driveway-20210927195529.mp4
Driveway-20210927195539.mp4
Driveway-20210927195548.mp4
Driveway-20210927195558.mp4
Driveway-20210927195610.mp4
Driveway-20210927195619.mp4
Driveway-20210927195629.mp4
Driveway-20210927195638.mp4
Driveway-20210927195648.mp4
Driveway-20210927195659.mp4
Driveway-20210927195709.mp4
Driveway-20210927195719.mp4
Driveway-20210927195728.mp4
Driveway-20210927195740.mp4
Driveway-20210927195749.mp4
Driveway-20210927195759.mp4
Driveway-20210927195809.mp4
Driveway-20210927195818.mp4
Driveway-20210927195830.mp4
Driveway-20210927195839.mp4
Driveway-20210927195849.mp4
Driveway-20210927195858.mp4
Driveway-20210927195910.mp4
Driveway-20210927195920.mp4
Driveway-20210927195929.mp4
Driveway-20210927195939.mp4
Driveway-20210927195948.mp4
Driveway-20210927200000.mp4
Driveway-20210927200009.mp4
Driveway-20210927200019.mp4
Driveway-20210927200029.mp4
Driveway-20210927200040.mp4
Driveway-20210927200050.mp4
Driveway-20210927200059.mp4
Driveway-20210927200109.mp4
Driveway-20210927200120.mp4
Driveway-20210927200130.mp4
Driveway-20210927200140.mp4
Driveway-20210927200149.mp4
Driveway-20210927200159.mp4
Driveway-20210927200210.mp4
Driveway-20210927200220.mp4
Driveway-20210927200230.mp4
Driveway-20210927200239.mp4
Driveway-20210927200251.mp4
Driveway-20210927200300.mp4
Driveway-20210927200310.mp4
Driveway-20210927200319.mp4
Driveway-20210927200329.mp4
Driveway-20210927200341.mp4
Driveway-20210927200350.mp4
Driveway-20210927200400.mp4
Driveway-20210927200409.mp4
Driveway-20210927200421.mp4
Driveway-20210927200430.mp4
Driveway-20210927200440.mp4
Driveway-20210927200450.mp4
Driveway-20210927200459.mp4
Driveway-20210927200511.mp4
Driveway-20210927200520.mp4
Driveway-20210927200530.mp4
Driveway-20210927200540.mp4
Driveway-20210927200551.mp4
Driveway-20210927200601.mp4
Driveway-20210927200610.mp4
Driveway-20210927200620.mp4
Driveway-20210927200631.mp4
Driveway-20210927200641.mp4
Driveway-20210927200651.mp4
Driveway-20210927200700.mp4
Driveway-20210927200710.mp4
Driveway-20210927200721.mp4
Driveway-20210927200731.mp4
Driveway-20210927200740.mp4
Driveway-20210927200750.mp4
Driveway-20210927200802.mp4
Driveway-20210927200811.mp4
Driveway-20210927200821.mp4
Driveway-20210927200830.mp4
Driveway-20210927200840.mp4
Driveway-20210927200851.mp4
Driveway-20210927200901.mp4
Driveway-20210927200911.mp4
Driveway-20210927200920.mp4
Driveway-20210927200932.mp4
Driveway-20210927200941.mp4
Driveway-20210927200951.mp4
Driveway-20210927201001.mp4
Driveway-20210927201012.mp4
Driveway-20210927201022.mp4
Driveway-20210927201031.mp4
Driveway-20210927201041.mp4
Driveway-20210927201050.mp4
Driveway-20210927201102.mp4
Driveway-20210927201112.mp4
Driveway-20210927201121.mp4
Driveway-20210927201131.mp4
Driveway-20210927201142.mp4
Driveway-20210927201152.mp4
Driveway-20210927201201.mp4
Driveway-20210927201211.mp4
Driveway-20210927201221.mp4
Driveway-20210927201232.mp4
Driveway-20210927201242.mp4
Driveway-20210927201251.mp4
Driveway-20210927201301.mp4
Driveway-20210927201312.mp4
Driveway-20210927201322.mp4
Driveway-20210927201332.mp4
Driveway-20210927201341.mp4
Driveway-20210927201351.mp4
Driveway-20210927201402.mp4
Driveway-20210927201412.mp4
Driveway-20210927201422.mp4
Driveway-20210927201431.mp4
Driveway-20210927201443.mp4
Driveway-20210927201452.mp4
Driveway-20210927201502.mp4
Driveway-20210927201511.mp4
Driveway-20210927201523.mp4
Driveway-20210927201533.mp4
Driveway-20210927201542.mp4
Driveway-20210927201552.mp4
Driveway-20210927201601.mp4
Driveway-20210927201613.mp4
Driveway-20210927201622.mp4
Driveway-20210927201632.mp4
Driveway-20210927201642.mp4
Driveway-20210927201653.mp4
Driveway-20210927201703.mp4
Driveway-20210927201712.mp4
Driveway-20210927201722.mp4
Driveway-20210927201732.mp4
Driveway-20210927201743.mp4
Driveway-20210927201753.mp4
Driveway-20210927201802.mp4
Driveway-20210927201812.mp4
Driveway-20210927201823.mp4
Driveway-20210927201833.mp4
Driveway-20210927201843.mp4
Driveway-20210927201852.mp4
Driveway-20210927201902.mp4
Driveway-20210927201913.mp4
Driveway-20210927201923.mp4
Driveway-20210927201932.mp4
Driveway-20210927201942.mp4
Driveway-20210927201954.mp4
Driveway-20210927202003.mp4
Driveway-20210927202013.mp4
Driveway-20210927202022.mp4
Driveway-20210927202034.mp4
Driveway-20210927202043.mp4
Driveway-20210927202053.mp4
Driveway-20210927202103.mp4
Driveway-20210927202112.mp4
Driveway-20210927202124.mp4
Driveway-20210927202133.mp4
Driveway-20210927202143.mp4
Driveway-20210927202153.mp4
Driveway-20210927202204.mp4
Driveway-20210927202214.mp4
Driveway-20210927202223.mp4
Driveway-20210927202233.mp4
Driveway-20210927202242.mp4
Driveway-20210927202254.mp4
Driveway-20210927202304.mp4
Driveway-20210927202313.mp4
Driveway-20210927202323.mp4
Driveway-20210927202334.mp4
Driveway-20210927202344.mp4
Driveway-20210927202353.mp4
Driveway-20210927202403.mp4
Driveway-20210927202415.mp4
Driveway-20210927202424.mp4
Driveway-20210927202434.mp4
Driveway-20210927202443.mp4
Driveway-20210927202453.mp4
Driveway-20210927202504.mp4
Driveway-20210927202514.mp4
Driveway-20210927202524.mp4
Driveway-20210927202533.mp4
Driveway-20210927202545.mp4
Driveway-20210927202554.mp4
Driveway-20210927202604.mp4
Driveway-20210927202614.mp4
Entrance-20210927185507.mp4
Entrance-20210927185519.mp4
Entrance-20210927185528.mp4
Entrance-20210927185538.mp4
Entrance-20210927185547.mp4
Entrance-20210927185557.mp4
Entrance-20210927185608.mp4
Entrance-20210927185618.mp4
Entrance-20210927185628.mp4
Entrance-20210927185637.mp4
Entrance-20210927185649.mp4
Entrance-20210927185658.mp4
Entrance-20210927185708.mp4
Entrance-20210927185718.mp4
Entrance-20210927185727.mp4
Entrance-20210927185739.mp4
Entrance-20210927185748.mp4
Entrance-20210927185758.mp4
Entrance-20210927185807.mp4
Entrance-20210927185817.mp4
Entrance-20210927185829.mp4
Entrance-20210927185838.mp4
Entrance-20210927185848.mp4
Entrance-20210927185857.mp4
Entrance-20210927185907.mp4
Entrance-20210927185918.mp4
Entrance-20210927185928.mp4
Entrance-20210927185938.mp4
Entrance-20210927185947.mp4
Entrance-20210927185959.mp4
Entrance-20210927190008.mp4
Entrance-20210927190018.mp4
Entrance-20210927190028.mp4
Entrance-20210927190037.mp4
Entrance-20210927190049.mp4
Entrance-20210927190058.mp4
Entrance-20210927190108.mp4
Entrance-20210927190117.mp4
Entrance-20210927190127.mp4
Entrance-20210927190139.mp4
Entrance-20210927190148.mp4
Entrance-20210927190158.mp4
Entrance-20210927190207.mp4
Entrance-20210927190217.mp4
Entrance-20210927190228.mp4
Entrance-20210927190238.mp4
Entrance-20210927190248.mp4
Entrance-20210927190257.mp4
Entrance-20210927190309.mp4
Entrance-20210927190318.mp4
Entrance-20210927190328.mp4
Entrance-20210927190338.mp4
Entrance-20210927190347.mp4
Entrance-20210927190410.mp4
Entrance-20210927190421.mp4
Entrance-20210927190431.mp4
Entrance-20210927190440.mp4
Entrance-20210927190450.mp4
Entrance-20210927190459.mp4
Entrance-20210927190511.mp4
Entrance-20210927190521.mp4
Entrance-20210927190530.mp4
Entrance-20210927190540.mp4
Entrance-20210927190551.mp4
Entrance-20210927190601.mp4
Entrance-20210927190610.mp4
Entrance-20210927190620.mp4
Entrance-20210927190630.mp4
Entrance-20210927190641.mp4
Entrance-20210927190651.mp4
Entrance-20210927190700.mp4
Entrance-20210927190710.mp4
Entrance-20210927190721.mp4
Entrance-20210927190731.mp4
Entrance-20210927190741.mp4
Entrance-20210927190750.mp4
Entrance-20210927190802.mp4
Entrance-20210927190811.mp4
Entrance-20210927190821.mp4
Entrance-20210927190831.mp4
Entrance-20210927190840.mp4
Entrance-20210927190852.mp4
Entrance-20210927190901.mp4
Entrance-20210927190911.mp4
Entrance-20210927190920.mp4
Entrance-20210927190932.mp4
Entrance-20210927190942.mp4
Entrance-20210927190951.mp4
Entrance-20210927191001.mp4
Entrance-20210927191010.mp4
Entrance-20210927191022.mp4
Entrance-20210927191031.mp4
Entrance-20210927191041.mp4
Entrance-20210927191051.mp4
Entrance-20210927191102.mp4
Entrance-20210927191112.mp4
Entrance-20210927191121.mp4
Entrance-20210927191131.mp4
Entrance-20210927191141.mp4
Entrance-20210927191152.mp4
Entrance-20210927191202.mp4
Entrance-20210927191211.mp4
Entrance-20210927191221.mp4
Entrance-20210927191232.mp4
Entrance-20210927191242.mp4
Entrance-20210927191252.mp4
Entrance-20210927191301.mp4
Entrance-20210927191311.mp4
Entrance-20210927191322.mp4
Entrance-20210927191332.mp4
Entrance-20210927191341.mp4
Entrance-20210927191351.mp4
Entrance-20210927191403.mp4
Entrance-20210927191412.mp4
Entrance-20210927191422.mp4
Entrance-20210927191431.mp4
Entrance-20210927191443.mp4
Entrance-20210927191452.mp4
Entrance-20210927191502.mp4
Entrance-20210927191512.mp4
Entrance-20210927191521.mp4
Entrance-20210927191533.mp4
Entrance-20210927191542.mp4
Entrance-20210927191552.mp4
Entrance-20210927191602.mp4
Entrance-20210927191613.mp4
Entrance-20210927191623.mp4
Entrance-20210927191632.mp4
Entrance-20210927191642.mp4
Entrance-20210927191651.mp4
Entrance-20210927191703.mp4
Entrance-20210927191713.mp4
Entrance-20210927191722.mp4
Entrance-20210927191732.mp4
Entrance-20210927191743.mp4
Entrance-20210927191753.mp4
Entrance-20210927191802.mp4
Entrance-20210927191812.mp4
Entrance-20210927191822.mp4
Entrance-20210927191833.mp4
Entrance-20210927191843.mp4
Entrance-20210927191852.mp4
Entrance-20210927191902.mp4
Entrance-20210927191913.mp4
Entrance-20210927191923.mp4
Entrance-20210927191933.mp4
Entrance-20210927191942.mp4
Entrance-20210927191954.mp4
Entrance-20210927192003.mp4
Entrance-20210927192013.mp4
Entrance-20210927192023.mp4
Entrance-20210927192032.mp4
Entrance-20210927192044.mp4
Entrance-20210927192053.mp4
Entrance-20210927192103.mp4
Entrance-20210927192112.mp4
Entrance-20210927192124.mp4
Entrance-20210927192134.mp4
Entrance-20210927192143.mp4
Entrance-20210927192153.mp4
Entrance-20210927192202.mp4
Entrance-20210927192214.mp4
Entrance-20210927192223.mp4
Entrance-20210927192233.mp4
Entrance-20210927192243.mp4
Entrance-20210927192254.mp4
Entrance-20210927192304.mp4
Entrance-20210927192313.mp4
Entrance-20210927192323.mp4
Entrance-20210927192333.mp4
Entrance-20210927192344.mp4
Entrance-20210927192354.mp4
Entrance-20210927192403.mp4
Entrance-20210927192413.mp4
Entrance-20210927192424.mp4
Entrance-20210927192434.mp4
Entrance-20210927192444.mp4
Entrance-20210927192453.mp4
Entrance-20210927192503.mp4
Entrance-20210927192514.mp4
Entrance-20210927192524.mp4
Entrance-20210927192533.mp4
Entrance-20210927192543.mp4
Entrance-20210927192555.mp4
Entrance-20210927192604.mp4
Entrance-20210927192614.mp4
Entrance-20210927192623.mp4
Entrance-20210927192635.mp4
Entrance-20210927192644.mp4
Entrance-20210927192654.mp4
Entrance-20210927192704.mp4
Entrance-20210927192713.mp4
Entrance-20210927192725.mp4
Entrance-20210927192734.mp4
Entrance-20210927192744.mp4
Entrance-20210927192754.mp4
Entrance-20210927192805.mp4
Entrance-20210927192815.mp4
Entrance-20210927192824.mp4
Entrance-20210927192834.mp4
Entrance-20210927192843.mp4
Entrance-20210927192855.mp4
Entrance-20210927192905.mp4
Entrance-20210927192914.mp4
Entrance-20210927192924.mp4
Entrance-20210927192935.mp4
Entrance-20210927192945.mp4
Entrance-20210927192954.mp4
Entrance-20210927193004.mp4
Entrance-20210927193016.mp4
Entrance-20210927193025.mp4
Entrance-20210927193035.mp4
Entrance-20210927193044.mp4
Entrance-20210927193054.mp4
Entrance-20210927193105.mp4
Entrance-20210927193115.mp4
Entrance-20210927193125.mp4
Entrance-20210927193134.mp4
Entrance-20210927193146.mp4
Entrance-20210927193155.mp4
Entrance-20210927193205.mp4
Entrance-20210927193215.mp4
Entrance-20210927193224.mp4
Entrance-20210927193236.mp4
Entrance-20210927193245.mp4
Entrance-20210927193255.mp4
Entrance-20210927193304.mp4
Entrance-20210927193316.mp4
Entrance-20210927193326.mp4
Entrance-20210927193335.mp4
Entrance-20210927193345.mp4
Entrance-20210927193354.mp4
Entrance-20210927193406.mp4
Entrance-20210927193415.mp4
Entrance-20210927193425.mp4
Entrance-20210927193435.mp4
Entrance-20210927193446.mp4
Entrance-20210927193456.mp4
Entrance-20210927193505.mp4
Entrance-20210927193515.mp4
Entrance-20210927193526.mp4
Entrance-20210927193536.mp4
Entrance-20210927193546.mp4
Entrance-20210927193555.mp4
Entrance-20210927193605.mp4
Entrance-20210927193616.mp4
Entrance-20210927193626.mp4
Entrance-20210927193636.mp4
Entrance-20210927193645.mp4
Entrance-20210927193657.mp4
Entrance-20210927193706.mp4
Entrance-20210927193716.mp4
Entrance-20210927193725.mp4
Entrance-20210927193735.mp4
Entrance-20210927193747.mp4
Entrance-20210927193756.mp4
Entrance-20210927193806.mp4
Entrance-20210927193815.mp4
Entrance-20210927193827.mp4
Entrance-20210927193836.mp4
Entrance-20210927193846.mp4
Entrance-20210927193856.mp4
Entrance-20210927193905.mp4
Entrance-20210927193917.mp4
Entrance-20210927193926.mp4
Entrance-20210927193936.mp4
Entrance-20210927193946.mp4
Entrance-20210927193957.mp4
Entrance-20210927194007.mp4
Entrance-20210927194016.mp4
Entrance-20210927194026.mp4
Entrance-20210927194037.mp4
Entrance-20210927194047.mp4
Entrance-20210927194057.mp4
Entrance-20210927194106.mp4
Entrance-20210927194116.mp4
Entrance-20210927194127.mp4
Entrance-20210927194137.mp4
Entrance-20210927194146.mp4
Entrance-20210927194156.mp4
Entrance-20210927194208.mp4
Entrance-20210927194217.mp4
Entrance-20210927194227.mp4
Entrance-20210927194236.mp4
Entrance-20210927194246.mp4
Entrance-20210927194257.mp4
Entrance-20210927194307.mp4
Entrance-20210927194317.mp4
Entrance-20210927194326.mp4
Entrance-20210927194338.mp4
Entrance-20210927194347.mp4
Entrance-20210927194357.mp4
Entrance-20210927194407.mp4
Entrance-20210927194416.mp4
Entrance-20210927194428.mp4
Entrance-20210927194437.mp4
Entrance-20210927194447.mp4
Entrance-20210927194456.mp4
Entrance-20210927194508.mp4
Entrance-20210927194518.mp4
Entrance-20210927194527.mp4
Entrance-20210927194537.mp4
Entrance-20210927194546.mp4
Entrance-20210927194558.mp4
Entrance-20210927194607.mp4
Entrance-20210927194617.mp4
Entrance-20210927194627.mp4
Entrance-20210927194638.mp4
Entrance-20210927194648.mp4
Entrance-20210927194657.mp4
Entrance-20210927194707.mp4
Entrance-20210927194718.mp4
Entrance-20210927194728.mp4
Entrance-20210927194738.mp4
Entrance-20210927194747.mp4
Entrance-20210927194757.mp4
Entrance-20210927194808.mp4
Entrance-20210927194818.mp4
Entrance-20210927194828.mp4
Entrance-20210927194837.mp4
Entrance-20210927194849.mp4
Entrance-20210927194858.mp4
Entrance-20210927194908.mp4
Entrance-20210927194917.mp4
Entrance-20210927194927.mp4
Entrance-20210927194939.mp4
Entrance-20210927194948.mp4
Entrance-20210927194958.mp4
Entrance-20210927195007.mp4
Entrance-20210927195019.mp4
Entrance-20210927195028.mp4
Entrance-20210927195038.mp4
Entrance-20210927195048.mp4
Entrance-20210927195057.mp4
Entrance-20210927195109.mp4
Entrance-20210927195118.mp4
Entrance-20210927195128.mp4
Entrance-20210927195138.mp4
Entrance-20210927195149.mp4
Entrance-20210927195159.mp4
Entrance-20210927195208.mp4
Entrance-20210927195218.mp4
Entrance-20210927195229.mp4
Entrance-20210927195239.mp4
Entrance-20210927195249.mp4
Entrance-20210927195258.mp4
Entrance-20210927195308.mp4
Entrance-20210927195319.mp4
Entrance-20210927195329.mp4
Entrance-20210927195338.mp4
Entrance-20210927195348.mp4
Entrance-20210927195400.mp4
Entrance-20210927195409.mp4
Entrance-20210927195419.mp4
Entrance-20210927195428.mp4
Entrance-20210927195438.mp4
Entrance-20210927195449.mp4
Entrance-20210927195459.mp4
Entrance-20210927195509.mp4
Entrance-20210927195518.mp4
Entrance-20210927195530.mp4
Entrance-20210927195539.mp4
Entrance-20210927195549.mp4
Entrance-20210927195559.mp4
Entrance-20210927195610.mp4
Entrance-20210927195620.mp4
Entrance-20210927195629.mp4
Entrance-20210927195639.mp4
Entrance-20210927195648.mp4
Entrance-20210927195700.mp4
Entrance-20210927195710.mp4
Entrance-20210927195719.mp4
Entrance-20210927195729.mp4
Entrance-20210927195740.mp4
Entrance-20210927195750.mp4
Entrance-20210927195759.mp4
Entrance-20210927195809.mp4
Entrance-20210927195819.mp4
Entrance-20210927195830.mp4
Entrance-20210927195840.mp4
Entrance-20210927195849.mp4
Entrance-20210927195859.mp4
Entrance-20210927195910.mp4
Entrance-20210927195920.mp4
Entrance-20210927195930.mp4
Entrance-20210927195939.mp4
Entrance-20210927195949.mp4
Entrance-20210927200000.mp4
Entrance-20210927200010.mp4
Entrance-20210927200020.mp4
Entrance-20210927200029.mp4
Entrance-20210927200041.mp4
Entrance-20210927200050.mp4
Entrance-20210927200100.mp4
Entrance-20210927200109.mp4
Entrance-20210927200121.mp4
Entrance-20210927200131.mp4
Entrance-20210927200140.mp4
Entrance-20210927200150.mp4
Entrance-20210927200159.mp4
Entrance-20210927200211.mp4
Entrance-20210927200220.mp4
Entrance-20210927200230.mp4
Entrance-20210927200240.mp4
Entrance-20210927200251.mp4
Entrance-20210927200301.mp4
Entrance-20210927200310.mp4
Entrance-20210927200320.mp4
Entrance-20210927200330.mp4
Entrance-20210927200341.mp4
Entrance-20210927200351.mp4
Entrance-20210927200400.mp4
Entrance-20210927200410.mp4
Entrance-20210927200421.mp4
Entrance-20210927200431.mp4
Entrance-20210927200441.mp4
Entrance-20210927200450.mp4
Entrance-20210927200500.mp4
Entrance-20210927200511.mp4
Entrance-20210927200521.mp4
Entrance-20210927200530.mp4
Entrance-20210927200540.mp4
Entrance-20210927200552.mp4
Entrance-20210927200601.mp4
Entrance-20210927200611.mp4
Entrance-20210927200620.mp4
Entrance-20210927200632.mp4
Entrance-20210927200641.mp4
Entrance-20210927200651.mp4
Entrance-20210927200701.mp4
Entrance-20210927200710.mp4
Entrance-20210927200722.mp4
Entrance-20210927200731.mp4
Entrance-20210927200741.mp4
Entrance-20210927200751.mp4
Entrance-20210927200802.mp4
Entrance-20210927200812.mp4
Entrance-20210927200821.mp4
Entrance-20210927200831.mp4
Entrance-20210927200840.mp4
Entrance-20210927200852.mp4
Entrance-20210927200902.mp4
Entrance-20210927200911.mp4
Entrance-20210927200921.mp4
Entrance-20210927200932.mp4
Entrance-20210927200942.mp4
Entrance-20210927200951.mp4
Entrance-20210927201001.mp4
Entrance-20210927201011.mp4
Entrance-20210927201022.mp4
Entrance-20210927201032.mp4
Entrance-20210927201041.mp4
Entrance-20210927201051.mp4
Entrance-20210927201102.mp4
Entrance-20210927201112.mp4
Entrance-20210927201122.mp4
Entrance-20210927201131.mp4
Entrance-20210927201143.mp4
Entrance-20210927201152.mp4
Entrance-20210927201202.mp4
Entrance-20210927201212.mp4
Entrance-20210927201221.mp4
Entrance-20210927201233.mp4
Entrance-20210927201242.mp4
Entrance-20210927201252.mp4
Entrance-20210927201301.mp4
Entrance-20210927201313.mp4
Entrance-20210927201323.mp4
Entrance-20210927201332.mp4
Entrance-20210927201342.mp4
Entrance-20210927201351.mp4
Entrance-20210927201403.mp4
Entrance-20210927201412.mp4
Entrance-20210927201422.mp4
Entrance-20210927201432.mp4
Entrance-20210927201443.mp4
Entrance-20210927201453.mp4
Entrance-20210927201502.mp4
Entrance-20210927201512.mp4
Entrance-20210927201522.mp4
Entrance-20210927201533.mp4
Entrance-20210927201543.mp4
Entrance-20210927201552.mp4
Entrance-20210927201602.mp4
Entrance-20210927201613.mp4
Entrance-20210927201623.mp4
Entrance-20210927201633.mp4
Entrance-20210927201642.mp4
Entrance-20210927201654.mp4
Entrance-20210927201703.mp4
Entrance-20210927201713.mp4
Entrance-20210927201722.mp4
Entrance-20210927201732.mp4
Entrance-20210927201744.mp4
Entrance-20210927201753.mp4
Entrance-20210927201803.mp4
Entrance-20210927201812.mp4
Entrance-20210927201824.mp4
Entrance-20210927201833.mp4
Entrance-20210927201843.mp4
Entrance-20210927201853.mp4
Entrance-20210927201902.mp4
Entrance-20210927201914.mp4
Entrance-20210927201923.mp4
Entrance-20210927201933.mp4
Entrance-20210927201943.mp4
Entrance-20210927201954.mp4
Entrance-20210927202004.mp4
Entrance-20210927202013.mp4
Entrance-20210927202023.mp4
Entrance-20210927202034.mp4
Entrance-20210927202044.mp4
Entrance-20210927202054.mp4
Entrance-20210927202103.mp4
Entrance-20210927202113.mp4
Entrance-20210927202124.mp4
Entrance-20210927202134.mp4
Entrance-20210927202143.mp4
Entrance-20210927202153.mp4
Entrance-20210927202205.mp4
Entrance-20210927202214.mp4
Entrance-20210927202224.mp4
Entrance-20210927202233.mp4
Entrance-20210927202243.mp4
Entrance-20210927202254.mp4
Entrance-20210927202304.mp4
Entrance-20210927202314.mp4
Entrance-20210927202323.mp4
Entrance-20210927202335.mp4
Entrance-20210927202344.mp4
Entrance-20210927202354.mp4
Entrance-20210927202404.mp4
Entrance-20210927202413.mp4
Entrance-20210927202425.mp4
Entrance-20210927202434.mp4
Entrance-20210927202444.mp4
Entrance-20210927202453.mp4
Entrance-20210927202505.mp4
Entrance-20210927202515.mp4
Entrance-20210927202524.mp4
Entrance-20210927202534.mp4
Entrance-20210927202545.mp4
Entrance-20210927202555.mp4
Entrance-20210927202604.mp4
Entrance-20210927202614.mp4
Laundry-20210927185508.mp4
Laundry-20210927185518.mp4
Laundry-20210927185529.mp4
Laundry-20210927185539.mp4
Laundry-20210927185548.mp4
Laundry-20210927185558.mp4
Laundry-20210927185607.mp4
Laundry-20210927185619.mp4
Laundry-20210927185629.mp4
Laundry-20210927185638.mp4
Laundry-20210927185648.mp4
Laundry-20210927185657.mp4
Laundry-20210927185709.mp4
Laundry-20210927185718.mp4
Laundry-20210927185728.mp4
Laundry-20210927185738.mp4
Laundry-20210927185749.mp4
Laundry-20210927185759.mp4
Laundry-20210927185808.mp4
Laundry-20210927185818.mp4
Laundry-20210927185828.mp4
Laundry-20210927185839.mp4
Laundry-20210927185849.mp4
Laundry-20210927185858.mp4
Laundry-20210927185908.mp4
Laundry-20210927185917.mp4
Laundry-20210927185929.mp4
Laundry-20210927185939.mp4
Laundry-20210927185948.mp4
Laundry-20210927185958.mp4
Laundry-20210927190007.mp4
Laundry-20210927190019.mp4
Laundry-20210927190028.mp4
Laundry-20210927190038.mp4
Laundry-20210927190048.mp4
Laundry-20210927190059.mp4
Laundry-20210927190109.mp4
Laundry-20210927190118.mp4
Laundry-20210927190128.mp4
Laundry-20210927190137.mp4
Laundry-20210927190149.mp4
Laundry-20210927190159.mp4
Laundry-20210927190208.mp4
Laundry-20210927190218.mp4
Laundry-20210927190227.mp4
Laundry-20210927190239.mp4
Laundry-20210927190249.mp4
Laundry-20210927190258.mp4
Laundry-20210927190308.mp4
Laundry-20210927190317.mp4
Laundry-20210927190329.mp4
Laundry-20210927190338.mp4
Laundry-20210927190348.mp4
Laundry-20210927190410.mp4
Laundry-20210927190420.mp4
Laundry-20210927190430.mp4
Laundry-20210927190439.mp4
Laundry-20210927190449.mp4
Laundry-20210927190500.mp4
Laundry-20210927190510.mp4
Laundry-20210927190519.mp4
Laundry-20210927190529.mp4
Laundry-20210927190541.mp4
Laundry-20210927190550.mp4
Laundry-20210927190600.mp4
Laundry-20210927190609.mp4
Laundry-20210927190621.mp4
Laundry-20210927190630.mp4
Laundry-20210927190640.mp4
Laundry-20210927190650.mp4
Laundry-20210927190659.mp4
Laundry-20210927190711.mp4
Laundry-20210927190720.mp4
Laundry-20210927190730.mp4
Laundry-20210927190740.mp4
Laundry-20210927190751.mp4
Laundry-20210927190801.mp4
Laundry-20210927190810.mp4
Laundry-20210927190820.mp4
Laundry-20210927190829.mp4
Laundry-20210927190841.mp4
Laundry-20210927190851.mp4
Laundry-20210927190900.mp4
Laundry-20210927190910.mp4
Laundry-20210927190921.mp4
Laundry-20210927190931.mp4
Laundry-20210927190940.mp4
Laundry-20210927190950.mp4
Laundry-20210927191000.mp4
Laundry-20210927191011.mp4
Laundry-20210927191021.mp4
Laundry-20210927191030.mp4
Laundry-20210927191040.mp4
Laundry-20210927191051.mp4
Laundry-20210927191101.mp4
Laundry-20210927191111.mp4
Laundry-20210927191120.mp4
Laundry-20210927191132.mp4
Laundry-20210927191141.mp4
Laundry-20210927191151.mp4
Laundry-20210927191201.mp4
Laundry-20210927191210.mp4
Laundry-20210927191222.mp4
Laundry-20210927191231.mp4
Laundry-20210927191241.mp4
Laundry-20210927191250.mp4
Laundry-20210927191302.mp4
Laundry-20210927191312.mp4
Laundry-20210927191321.mp4
Laundry-20210927191331.mp4
Laundry-20210927191340.mp4
Laundry-20210927191352.mp4
Laundry-20210927191401.mp4
Laundry-20210927191411.mp4
Laundry-20210927191421.mp4
Laundry-20210927191432.mp4
Laundry-20210927191442.mp4
Laundry-20210927191451.mp4
Laundry-20210927191501.mp4
Laundry-20210927191511.mp4
Laundry-20210927191522.mp4
Laundry-20210927191532.mp4
Laundry-20210927191541.mp4
Laundry-20210927191551.mp4
Laundry-20210927191602.mp4
Laundry-20210927191612.mp4
Laundry-20210927191622.mp4
Laundry-20210927191631.mp4
Laundry-20210927191641.mp4
Laundry-20210927191652.mp4
Laundry-20210927191702.mp4
Laundry-20210927191711.mp4
Laundry-20210927191721.mp4
Laundry-20210927191733.mp4
Laundry-20210927191742.mp4
Laundry-20210927191752.mp4
Laundry-20210927191801.mp4
Laundry-20210927191813.mp4
Laundry-20210927191822.mp4
Laundry-20210927191832.mp4
Laundry-20210927191842.mp4
Laundry-20210927191851.mp4
Laundry-20210927191903.mp4
Laundry-20210927191912.mp4
Laundry-20210927191922.mp4
Laundry-20210927191932.mp4
Laundry-20210927191943.mp4
Laundry-20210927191953.mp4
Laundry-20210927192002.mp4
Laundry-20210927192012.mp4
Laundry-20210927192021.mp4
Laundry-20210927192033.mp4
Laundry-20210927192043.mp4
Laundry-20210927192052.mp4
Laundry-20210927192102.mp4
Laundry-20210927192113.mp4
Laundry-20210927192123.mp4
Laundry-20210927192132.mp4
Laundry-20210927192142.mp4
Laundry-20210927192152.mp4
Laundry-20210927192203.mp4
Laundry-20210927192213.mp4
Laundry-20210927192222.mp4
Laundry-20210927192232.mp4
Laundry-20210927192243.mp4
Laundry-20210927192253.mp4
Laundry-20210927192303.mp4
Laundry-20210927192312.mp4
Laundry-20210927192324.mp4
Laundry-20210927192333.mp4
Laundry-20210927192343.mp4
Laundry-20210927192353.mp4
Laundry-20210927192402.mp4
Laundry-20210927192414.mp4
Laundry-20210927192423.mp4
Laundry-20210927192433.mp4
Laundry-20210927192442.mp4
Laundry-20210927192454.mp4
Laundry-20210927192504.mp4
Laundry-20210927192513.mp4
Laundry-20210927192523.mp4
Laundry-20210927192532.mp4
Laundry-20210927192544.mp4
Laundry-20210927192553.mp4
Laundry-20210927192603.mp4
Laundry-20210927192613.mp4
Laundry-20210927192624.mp4
Laundry-20210927192634.mp4
Laundry-20210927192643.mp4
Laundry-20210927192653.mp4
Laundry-20210927192704.mp4
Laundry-20210927192714.mp4
Laundry-20210927192724.mp4
Laundry-20210927192733.mp4
Laundry-20210927192743.mp4
Laundry-20210927192754.mp4
Laundry-20210927192804.mp4
Laundry-20210927192814.mp4
Laundry-20210927192823.mp4
Laundry-20210927192835.mp4
Laundry-20210927192844.mp4
Laundry-20210927192854.mp4
Laundry-20210927192903.mp4
Laundry-20210927192913.mp4
Laundry-20210927192925.mp4
Laundry-20210927192934.mp4
Laundry-20210927192944.mp4
Laundry-20210927192953.mp4
Laundry-20210927193005.mp4
Laundry-20210927193014.mp4
Laundry-20210927193024.mp4
Laundry-20210927193034.mp4
Laundry-20210927193043.mp4
Laundry-20210927193055.mp4
Laundry-20210927193104.mp4
Laundry-20210927193114.mp4
Laundry-20210927193124.mp4
Laundry-20210927193135.mp4
Laundry-20210927193145.mp4
Laundry-20210927193154.mp4
Laundry-20210927193204.mp4
Laundry-20210927193215.mp4
Laundry-20210927193225.mp4
Laundry-20210927193235.mp4
Laundry-20210927193244.mp4
Laundry-20210927193254.mp4
Laundry-20210927193305.mp4
Laundry-20210927193315.mp4
Laundry-20210927193324.mp4
Laundry-20210927193334.mp4
Laundry-20210927193346.mp4
Laundry-20210927193355.mp4
Laundry-20210927193405.mp4
Laundry-20210927193414.mp4
Laundry-20210927193424.mp4
Laundry-20210927193435.mp4
Laundry-20210927193445.mp4
Laundry-20210927193455.mp4
Laundry-20210927193504.mp4
Laundry-20210927193516.mp4
Laundry-20210927193525.mp4
Laundry-20210927193535.mp4
Laundry-20210927193545.mp4
Laundry-20210927193554.mp4
Laundry-20210927193606.mp4
Laundry-20210927193615.mp4
Laundry-20210927193625.mp4
Laundry-20210927193634.mp4
Laundry-20210927193646.mp4
Laundry-20210927193656.mp4
Laundry-20210927193705.mp4
Laundry-20210927193715.mp4
Laundry-20210927193726.mp4
Laundry-20210927193736.mp4
Laundry-20210927193745.mp4
Laundry-20210927193755.mp4
Laundry-20210927193805.mp4
Laundry-20210927193816.mp4
Laundry-20210927193826.mp4
Laundry-20210927193835.mp4
Laundry-20210927193845.mp4
Laundry-20210927193856.mp4
Laundry-20210927193906.mp4
Laundry-20210927193916.mp4
Laundry-20210927193925.mp4
Laundry-20210927193935.mp4
Laundry-20210927193946.mp4
Laundry-20210927193956.mp4
Laundry-20210927194006.mp4
Laundry-20210927194015.mp4
Laundry-20210927194027.mp4
Laundry-20210927194036.mp4
Laundry-20210927194046.mp4
Laundry-20210927194055.mp4
Laundry-20210927194105.mp4
Laundry-20210927194117.mp4
Laundry-20210927194126.mp4
Laundry-20210927194136.mp4
Laundry-20210927194145.mp4
Laundry-20210927194157.mp4
Laundry-20210927194206.mp4
Laundry-20210927194216.mp4
Laundry-20210927194226.mp4
Laundry-20210927194237.mp4
Laundry-20210927194247.mp4
Laundry-20210927194256.mp4
Laundry-20210927194306.mp4
Laundry-20210927194316.mp4
Laundry-20210927194327.mp4
Laundry-20210927194337.mp4
Laundry-20210927194346.mp4
Laundry-20210927194356.mp4
Laundry-20210927194407.mp4
Laundry-20210927194417.mp4
Laundry-20210927194427.mp4
Laundry-20210927194436.mp4
Laundry-20210927194446.mp4
Laundry-20210927194457.mp4
Laundry-20210927194507.mp4
Laundry-20210927194516.mp4
Laundry-20210927194526.mp4
Laundry-20210927194538.mp4
Laundry-20210927194547.mp4
Laundry-20210927194557.mp4
Laundry-20210927194606.mp4
Laundry-20210927194616.mp4
Laundry-20210927194627.mp4
Laundry-20210927194637.mp4
Laundry-20210927194647.mp4
Laundry-20210927194656.mp4
Laundry-20210927194708.mp4
Laundry-20210927194717.mp4
Laundry-20210927194727.mp4
Laundry-20210927194737.mp4
Laundry-20210927194748.mp4
Laundry-20210927194758.mp4
Laundry-20210927194807.mp4
Laundry-20210927194817.mp4
Laundry-20210927194826.mp4
Laundry-20210927194838.mp4
Laundry-20210927194848.mp4
Laundry-20210927194857.mp4
Laundry-20210927194907.mp4
Laundry-20210927194918.mp4
Laundry-20210927194928.mp4
Laundry-20210927194937.mp4
Laundry-20210927194947.mp4
Laundry-20210927194957.mp4
Laundry-20210927195008.mp4
Laundry-20210927195018.mp4
Laundry-20210927195027.mp4
Laundry-20210927195037.mp4
Laundry-20210927195048.mp4
Laundry-20210927195058.mp4
Laundry-20210927195108.mp4
Laundry-20210927195117.mp4
Laundry-20210927195127.mp4
Laundry-20210927195138.mp4
Laundry-20210927195148.mp4
Laundry-20210927195158.mp4
Laundry-20210927195207.mp4
Laundry-20210927195219.mp4
Laundry-20210927195228.mp4
Laundry-20210927195238.mp4
Laundry-20210927195247.mp4
Laundry-20210927195259.mp4
Laundry-20210927195309.mp4
Laundry-20210927195318.mp4
Laundry-20210927195328.mp4
Laundry-20210927195337.mp4
Laundry-20210927195349.mp4
Laundry-20210927195358.mp4
Laundry-20210927195408.mp4
Laundry-20210927195418.mp4
Laundry-20210927195429.mp4
Laundry-20210927195439.mp4
Laundry-20210927195448.mp4
Laundry-20210927195458.mp4
Laundry-20210927195508.mp4
Laundry-20210927195519.mp4
Laundry-20210927195529.mp4
Laundry-20210927195538.mp4
Laundry-20210927195548.mp4
Laundry-20210927195559.mp4
Laundry-20210927195609.mp4
Laundry-20210927195619.mp4
Laundry-20210927195628.mp4
Laundry-20210927195640.mp4
Laundry-20210927195649.mp4
Laundry-20210927195659.mp4
Laundry-20210927195708.mp4
Laundry-20210927195718.mp4
Laundry-20210927195730.mp4
Laundry-20210927195739.mp4
Laundry-20210927195749.mp4
Laundry-20210927195758.mp4
Laundry-20210927195810.mp4
Laundry-20210927195819.mp4
Laundry-20210927195829.mp4
Laundry-20210927195839.mp4
Laundry-20210927195848.mp4
Laundry-20210927195900.mp4
Laundry-20210927195909.mp4
Laundry-20210927195919.mp4
Laundry-20210927195929.mp4
Laundry-20210927195940.mp4
Laundry-20210927195950.mp4
Laundry-20210927195959.mp4
Laundry-20210927200009.mp4
Laundry-20210927200018.mp4
Laundry-20210927200030.mp4
Laundry-20210927200040.mp4
Laundry-20210927200049.mp4
Laundry-20210927200059.mp4
Laundry-20210927200110.mp4
Laundry-20210927200120.mp4
Laundry-20210927200129.mp4
Laundry-20210927200139.mp4
Laundry-20210927200149.mp4
Laundry-20210927200200.mp4
Laundry-20210927200210.mp4
Laundry-20210927200219.mp4
Laundry-20210927200229.mp4
Laundry-20210927200240.mp4
Laundry-20210927200250.mp4
Laundry-20210927200300.mp4
Laundry-20210927200309.mp4
Laundry-20210927200321.mp4
Laundry-20210927200330.mp4
Laundry-20210927200340.mp4
Laundry-20210927200350.mp4
Laundry-20210927200359.mp4
Laundry-20210927200411.mp4
Laundry-20210927200420.mp4
Laundry-20210927200430.mp4
Laundry-20210927200439.mp4
Laundry-20210927200451.mp4
Laundry-20210927200501.mp4
Laundry-20210927200510.mp4
Laundry-20210927200520.mp4
Laundry-20210927200529.mp4
Laundry-20210927200541.mp4
Laundry-20210927200550.mp4
Laundry-20210927200600.mp4
Laundry-20210927200610.mp4
Laundry-20210927200621.mp4
Laundry-20210927200631.mp4
Laundry-20210927200640.mp4
Laundry-20210927200650.mp4
Laundry-20210927200701.mp4
Laundry-20210927200711.mp4
Laundry-20210927200721.mp4
Laundry-20210927200730.mp4
Laundry-20210927200740.mp4
Laundry-20210927200751.mp4
Laundry-20210927200801.mp4
Laundry-20210927200811.mp4
Laundry-20210927200820.mp4
Laundry-20210927200832.mp4
Laundry-20210927200841.mp4
Laundry-20210927200851.mp4
Laundry-20210927200900.mp4
Laundry-20210927200910.mp4
Laundry-20210927200922.mp4
Laundry-20210927200931.mp4
Laundry-20210927200941.mp4
Laundry-20210927200950.mp4
Laundry-20210927201002.mp4
Laundry-20210927201011.mp4
Laundry-20210927201021.mp4
Laundry-20210927201031.mp4
Laundry-20210927201040.mp4
Laundry-20210927201052.mp4
Laundry-20210927201101.mp4
Laundry-20210927201111.mp4
Laundry-20210927201121.mp4
Laundry-20210927201132.mp4
Laundry-20210927201142.mp4
Laundry-20210927201151.mp4
Laundry-20210927201201.mp4
Laundry-20210927201212.mp4
Laundry-20210927201222.mp4
Laundry-20210927201232.mp4
Laundry-20210927201241.mp4
Laundry-20210927201251.mp4
Laundry-20210927201302.mp4
Laundry-20210927201312.mp4
Laundry-20210927201321.mp4
Laundry-20210927201331.mp4
Laundry-20210927201343.mp4
Laundry-20210927201352.mp4
Laundry-20210927201402.mp4
Laundry-20210927201411.mp4
Laundry-20210927201421.mp4
Laundry-20210927201432.mp4
Laundry-20210927201442.mp4
Laundry-20210927201452.mp4
Laundry-20210927201501.mp4
Laundry-20210927201513.mp4
Laundry-20210927201522.mp4
Laundry-20210927201532.mp4
Laundry-20210927201542.mp4
Laundry-20210927201553.mp4
Laundry-20210927201603.mp4
Laundry-20210927201612.mp4
Laundry-20210927201622.mp4
Laundry-20210927201631.mp4
Laundry-20210927201643.mp4
Laundry-20210927201653.mp4
Laundry-20210927201702.mp4
Laundry-20210927201712.mp4
Laundry-20210927201723.mp4
Laundry-20210927201733.mp4
Laundry-20210927201743.mp4
Laundry-20210927201752.mp4
Laundry-20210927201802.mp4
Laundry-20210927201813.mp4
Laundry-20210927201823.mp4
Laundry-20210927201832.mp4
Laundry-20210927201842.mp4
Laundry-20210927201853.mp4
Laundry-20210927201903.mp4
Laundry-20210927201913.mp4
Laundry-20210927201922.mp4
Laundry-20210927201932.mp4
Laundry-20210927201943.mp4
Laundry-20210927201953.mp4
Laundry-20210927202003.mp4
Laundry-20210927202012.mp4
Laundry-20210927202024.mp4
Laundry-20210927202033.mp4
Laundry-20210927202043.mp4
Laundry-20210927202052.mp4
Laundry-20210927202104.mp4
Laundry-20210927202114.mp4
Laundry-20210927202123.mp4
Laundry-20210927202133.mp4
Laundry-20210927202142.mp4
Laundry-20210927202154.mp4
Laundry-20210927202203.mp4
Laundry-20210927202213.mp4
Laundry-20210927202223.mp4
Laundry-20210927202234.mp4
Laundry-20210927202244.mp4
Laundry-20210927202253.mp4
Laundry-20210927202303.mp4
Laundry-20210927202313.mp4
Laundry-20210927202324.mp4
Laundry-20210927202334.mp4
Laundry-20210927202343.mp4
Laundry-20210927202353.mp4
Laundry-20210927202404.mp4
Laundry-20210927202414.mp4
Laundry-20210927202424.mp4
Laundry-20210927202433.mp4
Laundry-20210927202445.mp4
Laundry-20210927202454.mp4
Laundry-20210927202504.mp4
Laundry-20210927202514.mp4
Laundry-20210927202523.mp4
Laundry-20210927202535.mp4
Laundry-20210927202544.mp4
Laundry-20210927202554.mp4
Laundry-20210927202603.mp4
Laundry-20210927202615.mp4
Rumpus-20210927185508.mp4
Rumpus-20210927185518.mp4
Rumpus-20210927185527.mp4
Rumpus-20210927185539.mp4
Rumpus-20210927185548.mp4
Rumpus-20210927185558.mp4
Rumpus-20210927185608.mp4
Rumpus-20210927185617.mp4
Rumpus-20210927185629.mp4
Rumpus-20210927185638.mp4
Rumpus-20210927185648.mp4
Rumpus-20210927185657.mp4
Rumpus-20210927185707.mp4
Rumpus-20210927185719.mp4
Rumpus-20210927185728.mp4
Rumpus-20210927185738.mp4
Rumpus-20210927185747.mp4
Rumpus-20210927185759.mp4
Rumpus-20210927185808.mp4
Rumpus-20210927185818.mp4
Rumpus-20210927185828.mp4
Rumpus-20210927185837.mp4
Rumpus-20210927185849.mp4
Rumpus-20210927185859.mp4
Rumpus-20210927185907.mp4
Rumpus-20210927185919.mp4
Rumpus-20210927185928.mp4
Rumpus-20210927185938.mp4
Rumpus-20210927185948.mp4
Rumpus-20210927185957.mp4
Rumpus-20210927190009.mp4
Rumpus-20210927190018.mp4
Rumpus-20210927190028.mp4
Rumpus-20210927190037.mp4
Rumpus-20210927190049.mp4
Rumpus-20210927190059.mp4
Rumpus-20210927190108.mp4
Rumpus-20210927190118.mp4
Rumpus-20210927190127.mp4
Rumpus-20210927190139.mp4
Rumpus-20210927190148.mp4
Rumpus-20210927190158.mp4
Rumpus-20210927190208.mp4
Rumpus-20210927190217.mp4
Rumpus-20210927190229.mp4
Rumpus-20210927190238.mp4
Rumpus-20210927190248.mp4
Rumpus-20210927190257.mp4
Rumpus-20210927190307.mp4
Rumpus-20210927190319.mp4
Rumpus-20210927190328.mp4
Rumpus-20210927190338.mp4
Rumpus-20210927190347.mp4
Rumpus-20210927190410.mp4
Rumpus-20210927190419.mp4
Rumpus-20210927190429.mp4
Rumpus-20210927190440.mp4
Rumpus-20210927190450.mp4
Rumpus-20210927190500.mp4
Rumpus-20210927190509.mp4
Rumpus-20210927190521.mp4
Rumpus-20210927190530.mp4
Rumpus-20210927190540.mp4
Rumpus-20210927190550.mp4
Rumpus-20210927190559.mp4
Rumpus-20210927190611.mp4
Rumpus-20210927190620.mp4
Rumpus-20210927190630.mp4
Rumpus-20210927190639.mp4
Rumpus-20210927190651.mp4
Rumpus-20210927190701.mp4
Rumpus-20210927190710.mp4
Rumpus-20210927190720.mp4
Rumpus-20210927190729.mp4
Rumpus-20210927190741.mp4
Rumpus-20210927190750.mp4
Rumpus-20210927190800.mp4
Rumpus-20210927190810.mp4
Rumpus-20210927190821.mp4
Rumpus-20210927190831.mp4
Rumpus-20210927190840.mp4
Rumpus-20210927190850.mp4
Rumpus-20210927190900.mp4
Rumpus-20210927190911.mp4
Rumpus-20210927190921.mp4
Rumpus-20210927190930.mp4
Rumpus-20210927190940.mp4
Rumpus-20210927190951.mp4
Rumpus-20210927191001.mp4
Rumpus-20210927191011.mp4
Rumpus-20210927191020.mp4
Rumpus-20210927191030.mp4
Rumpus-20210927191041.mp4
Rumpus-20210927191051.mp4
Rumpus-20210927191100.mp4
Rumpus-20210927191110.mp4
Rumpus-20210927191122.mp4
Rumpus-20210927191131.mp4
Rumpus-20210927191141.mp4
Rumpus-20210927191150.mp4
Rumpus-20210927191202.mp4
Rumpus-20210927191211.mp4
Rumpus-20210927191221.mp4
Rumpus-20210927191231.mp4
Rumpus-20210927191240.mp4
Rumpus-20210927191252.mp4
Rumpus-20210927191301.mp4
Rumpus-20210927191311.mp4
Rumpus-20210927191321.mp4
Rumpus-20210927191332.mp4
Rumpus-20210927191342.mp4
Rumpus-20210927191351.mp4
Rumpus-20210927191401.mp4
Rumpus-20210927191410.mp4
Rumpus-20210927191422.mp4
Rumpus-20210927191432.mp4
Rumpus-20210927191441.mp4
Rumpus-20210927191451.mp4
Rumpus-20210927191502.mp4
Rumpus-20210927191512.mp4
Rumpus-20210927191521.mp4
Rumpus-20210927191531.mp4
Rumpus-20210927191541.mp4
Rumpus-20210927191552.mp4
Rumpus-20210927191602.mp4
Rumpus-20210927191611.mp4
Rumpus-20210927191621.mp4
Rumpus-20210927191632.mp4
Rumpus-20210927191642.mp4
Rumpus-20210927191652.mp4
Rumpus-20210927191701.mp4
Rumpus-20210927191711.mp4
Rumpus-20210927191722.mp4
Rumpus-20210927191732.mp4
Rumpus-20210927191742.mp4
Rumpus-20210927191751.mp4
Rumpus-20210927191803.mp4
Rumpus-20210927191812.mp4
Rumpus-20210927191822.mp4
Rumpus-20210927191831.mp4
Rumpus-20210927191843.mp4
Rumpus-20210927191853.mp4
Rumpus-20210927191902.mp4
Rumpus-20210927191912.mp4
Rumpus-20210927191921.mp4
Rumpus-20210927191933.mp4
Rumpus-20210927191942.mp4
Rumpus-20210927191952.mp4
Rumpus-20210927192002.mp4
Rumpus-20210927192013.mp4
Rumpus-20210927192023.mp4
Rumpus-20210927192032.mp4
Rumpus-20210927192042.mp4
Rumpus-20210927192052.mp4
Rumpus-20210927192103.mp4
Rumpus-20210927192113.mp4
Rumpus-20210927192122.mp4
Rumpus-20210927192132.mp4
Rumpus-20210927192143.mp4
Rumpus-20210927192153.mp4
Rumpus-20210927192203.mp4
Rumpus-20210927192212.mp4
Rumpus-20210927192224.mp4
Rumpus-20210927192233.mp4
Rumpus-20210927192243.mp4
Rumpus-20210927192252.mp4
Rumpus-20210927192302.mp4
Rumpus-20210927192314.mp4
Rumpus-20210927192323.mp4
Rumpus-20210927192333.mp4
Rumpus-20210927192342.mp4
Rumpus-20210927192354.mp4
Rumpus-20210927192403.mp4
Rumpus-20210927192413.mp4
Rumpus-20210927192423.mp4
Rumpus-20210927192432.mp4
Rumpus-20210927192444.mp4
Rumpus-20210927192453.mp4
Rumpus-20210927192503.mp4
Rumpus-20210927192513.mp4
Rumpus-20210927192524.mp4
Rumpus-20210927192534.mp4
Rumpus-20210927192543.mp4
Rumpus-20210927192553.mp4
Rumpus-20210927192602.mp4
Rumpus-20210927192614.mp4
Rumpus-20210927192624.mp4
Rumpus-20210927192633.mp4
Rumpus-20210927192643.mp4
Rumpus-20210927192654.mp4
Rumpus-20210927192704.mp4
Rumpus-20210927192713.mp4
Rumpus-20210927192723.mp4
Rumpus-20210927192735.mp4
Rumpus-20210927192744.mp4
Rumpus-20210927192754.mp4
Rumpus-20210927192803.mp4
Rumpus-20210927192813.mp4
Rumpus-20210927192824.mp4
Rumpus-20210927192834.mp4
Rumpus-20210927192844.mp4
Rumpus-20210927192853.mp4
Rumpus-20210927192905.mp4
Rumpus-20210927192914.mp4
Rumpus-20210927192924.mp4
Rumpus-20210927192934.mp4
Rumpus-20210927192943.mp4
Rumpus-20210927192955.mp4
Rumpus-20210927193004.mp4
Rumpus-20210927193014.mp4
Rumpus-20210927193023.mp4
Rumpus-20210927193035.mp4
Rumpus-20210927193045.mp4
Rumpus-20210927193054.mp4
Rumpus-20210927193104.mp4
Rumpus-20210927193113.mp4
Rumpus-20210927193125.mp4
Rumpus-20210927193134.mp4
Rumpus-20210927193144.mp4
Rumpus-20210927193154.mp4
Rumpus-20210927193205.mp4
Rumpus-20210927193215.mp4
Rumpus-20210927193224.mp4
Rumpus-20210927193234.mp4
Rumpus-20210927193245.mp4
Rumpus-20210927193255.mp4
Rumpus-20210927193305.mp4
Rumpus-20210927193314.mp4
Rumpus-20210927193324.mp4
Rumpus-20210927193335.mp4
Rumpus-20210927193345.mp4
Rumpus-20210927193355.mp4
Rumpus-20210927193404.mp4
Rumpus-20210927193416.mp4
Rumpus-20210927193425.mp4
Rumpus-20210927193435.mp4
Rumpus-20210927193444.mp4
Rumpus-20210927193454.mp4
Rumpus-20210927193506.mp4
Rumpus-20210927193515.mp4
Rumpus-20210927193525.mp4
Rumpus-20210927193534.mp4
Rumpus-20210927193546.mp4
Rumpus-20210927193555.mp4
Rumpus-20210927193605.mp4
Rumpus-20210927193615.mp4
Rumpus-20210927193626.mp4
Rumpus-20210927193636.mp4
Rumpus-20210927193645.mp4
Rumpus-20210927193655.mp4
Rumpus-20210927193705.mp4
Rumpus-20210927193716.mp4
Rumpus-20210927193726.mp4
Rumpus-20210927193735.mp4
Rumpus-20210927193745.mp4
Rumpus-20210927193756.mp4
Rumpus-20210927193806.mp4
Rumpus-20210927193816.mp4
Rumpus-20210927193825.mp4
Rumpus-20210927193835.mp4
Rumpus-20210927193846.mp4
Rumpus-20210927193856.mp4
Rumpus-20210927193905.mp4
Rumpus-20210927193915.mp4
Rumpus-20210927193927.mp4
Rumpus-20210927193936.mp4
Rumpus-20210927193946.mp4
Rumpus-20210927193955.mp4
Rumpus-20210927194005.mp4
Rumpus-20210927194016.mp4
Rumpus-20210927194026.mp4
Rumpus-20210927194036.mp4
Rumpus-20210927194045.mp4
Rumpus-20210927194057.mp4
Rumpus-20210927194106.mp4
Rumpus-20210927194116.mp4
Rumpus-20210927194126.mp4
Rumpus-20210927194135.mp4
Rumpus-20210927194147.mp4
Rumpus-20210927194156.mp4
Rumpus-20210927194206.mp4
Rumpus-20210927194215.mp4
Rumpus-20210927194227.mp4
Rumpus-20210927194237.mp4
Rumpus-20210927194246.mp4
Rumpus-20210927194256.mp4
Rumpus-20210927194307.mp4
Rumpus-20210927194317.mp4
Rumpus-20210927194326.mp4
Rumpus-20210927194336.mp4
Rumpus-20210927194346.mp4
Rumpus-20210927194357.mp4
Rumpus-20210927194407.mp4
Rumpus-20210927194416.mp4
Rumpus-20210927194426.mp4
Rumpus-20210927194437.mp4
Rumpus-20210927194447.mp4
Rumpus-20210927194457.mp4
Rumpus-20210927194506.mp4
Rumpus-20210927194516.mp4
Rumpus-20210927194527.mp4
Rumpus-20210927194537.mp4
Rumpus-20210927194547.mp4
Rumpus-20210927194556.mp4
Rumpus-20210927194608.mp4
Rumpus-20210927194617.mp4
Rumpus-20210927194627.mp4
Rumpus-20210927194636.mp4
Rumpus-20210927194648.mp4
Rumpus-20210927194658.mp4
Rumpus-20210927194707.mp4
Rumpus-20210927194717.mp4
Rumpus-20210927194726.mp4
Rumpus-20210927194738.mp4
Rumpus-20210927194747.mp4
Rumpus-20210927194757.mp4
Rumpus-20210927194807.mp4
Rumpus-20210927194818.mp4
Rumpus-20210927194828.mp4
Rumpus-20210927194837.mp4
Rumpus-20210927194847.mp4
Rumpus-20210927194857.mp4
Rumpus-20210927194908.mp4
Rumpus-20210927194918.mp4
Rumpus-20210927194927.mp4
Rumpus-20210927194937.mp4
Rumpus-20210927194948.mp4
Rumpus-20210927194958.mp4
Rumpus-20210927195008.mp4
Rumpus-20210927195017.mp4
Rumpus-20210927195027.mp4
Rumpus-20210927195038.mp4
Rumpus-20210927195048.mp4
Rumpus-20210927195057.mp4
Rumpus-20210927195107.mp4
Rumpus-20210927195119.mp4
Rumpus-20210927195128.mp4
Rumpus-20210927195138.mp4
Rumpus-20210927195147.mp4
Rumpus-20210927195157.mp4
Rumpus-20210927195208.mp4
Rumpus-20210927195218.mp4
Rumpus-20210927195228.mp4
Rumpus-20210927195237.mp4
Rumpus-20210927195249.mp4
Rumpus-20210927195258.mp4
Rumpus-20210927195308.mp4
Rumpus-20210927195318.mp4
Rumpus-20210927195329.mp4
Rumpus-20210927195339.mp4
Rumpus-20210927195348.mp4
Rumpus-20210927195358.mp4
Rumpus-20210927195407.mp4
Rumpus-20210927195419.mp4
Rumpus-20210927195429.mp4
Rumpus-20210927195438.mp4
Rumpus-20210927195448.mp4
Rumpus-20210927195459.mp4
Rumpus-20210927195509.mp4
Rumpus-20210927195518.mp4
Rumpus-20210927195528.mp4
Rumpus-20210927195538.mp4
Rumpus-20210927195549.mp4
Rumpus-20210927195559.mp4
Rumpus-20210927195608.mp4
Rumpus-20210927195618.mp4
Rumpus-20210927195629.mp4
Rumpus-20210927195639.mp4
Rumpus-20210927195649.mp4
Rumpus-20210927195658.mp4
Rumpus-20210927195710.mp4
Rumpus-20210927195719.mp4
Rumpus-20210927195729.mp4
Rumpus-20210927195739.mp4
Rumpus-20210927195748.mp4
Rumpus-20210927195800.mp4
Rumpus-20210927195809.mp4
Rumpus-20210927195819.mp4
Rumpus-20210927195828.mp4
Rumpus-20210927195840.mp4
Rumpus-20210927195850.mp4
Rumpus-20210927195859.mp4
Rumpus-20210927195909.mp4
Rumpus-20210927195918.mp4
Rumpus-20210927195930.mp4
Rumpus-20210927195939.mp4
Rumpus-20210927195949.mp4
Rumpus-20210927195959.mp4
Rumpus-20210927200010.mp4
Rumpus-20210927200020.mp4
Rumpus-20210927200029.mp4
Rumpus-20210927200039.mp4
Rumpus-20210927200049.mp4
Rumpus-20210927200100.mp4
Rumpus-20210927200110.mp4
Rumpus-20210927200119.mp4
Rumpus-20210927200129.mp4
Rumpus-20210927200140.mp4
Rumpus-20210927200150.mp4
Rumpus-20210927200200.mp4
Rumpus-20210927200209.mp4
Rumpus-20210927200219.mp4
Rumpus-20210927200230.mp4
Rumpus-20210927200240.mp4
Rumpus-20210927200249.mp4
Rumpus-20210927200259.mp4
Rumpus-20210927200311.mp4
Rumpus-20210927200320.mp4
Rumpus-20210927200330.mp4
Rumpus-20210927200339.mp4
Rumpus-20210927200351.mp4
Rumpus-20210927200400.mp4
Rumpus-20210927200410.mp4
Rumpus-20210927200420.mp4
Rumpus-20210927200429.mp4
Rumpus-20210927200441.mp4
Rumpus-20210927200450.mp4
Rumpus-20210927200500.mp4
Rumpus-20210927200510.mp4
Rumpus-20210927200521.mp4
Rumpus-20210927200531.mp4
Rumpus-20210927200540.mp4
Rumpus-20210927200550.mp4
Rumpus-20210927200559.mp4
Rumpus-20210927200611.mp4
Rumpus-20210927200621.mp4
Rumpus-20210927200630.mp4
Rumpus-20210927200640.mp4
Rumpus-20210927200651.mp4
Rumpus-20210927200701.mp4
Rumpus-20210927200710.mp4
Rumpus-20210927200720.mp4
Rumpus-20210927200732.mp4
Rumpus-20210927200741.mp4
Rumpus-20210927200751.mp4
Rumpus-20210927200800.mp4
Rumpus-20210927200810.mp4
Rumpus-20210927200821.mp4
Rumpus-20210927200831.mp4
Rumpus-20210927200841.mp4
Rumpus-20210927200850.mp4
Rumpus-20210927200902.mp4
Rumpus-20210927200911.mp4
Rumpus-20210927200921.mp4
Rumpus-20210927200931.mp4
Rumpus-20210927200940.mp4
Rumpus-20210927200952.mp4
Rumpus-20210927201001.mp4
Rumpus-20210927201011.mp4
Rumpus-20210927201020.mp4
Rumpus-20210927201032.mp4
Rumpus-20210927201042.mp4
Rumpus-20210927201051.mp4
Rumpus-20210927201101.mp4
Rumpus-20210927201110.mp4
Rumpus-20210927201122.mp4
Rumpus-20210927201131.mp4
Rumpus-20210927201141.mp4
Rumpus-20210927201151.mp4
Rumpus-20210927201202.mp4
Rumpus-20210927201212.mp4
Rumpus-20210927201221.mp4
Rumpus-20210927201231.mp4
Rumpus-20210927201242.mp4
Rumpus-20210927201252.mp4
Rumpus-20210927201302.mp4
Rumpus-20210927201311.mp4
Rumpus-20210927201321.mp4
Rumpus-20210927201332.mp4
Rumpus-20210927201342.mp4
Rumpus-20210927201352.mp4
Rumpus-20210927201401.mp4
Rumpus-20210927201413.mp4
Rumpus-20210927201422.mp4
Rumpus-20210927201432.mp4
Rumpus-20210927201441.mp4
Rumpus-20210927201451.mp4
Rumpus-20210927201503.mp4
Rumpus-20210927201512.mp4
Rumpus-20210927201522.mp4
Rumpus-20210927201531.mp4
Rumpus-20210927201543.mp4
Rumpus-20210927201552.mp4
Rumpus-20210927201602.mp4
Rumpus-20210927201612.mp4
Rumpus-20210927201621.mp4
Rumpus-20210927201633.mp4
Rumpus-20210927201642.mp4
Rumpus-20210927201652.mp4
Rumpus-20210927201702.mp4
Rumpus-20210927201713.mp4
Rumpus-20210927201723.mp4
Rumpus-20210927201732.mp4
Rumpus-20210927201742.mp4
Rumpus-20210927201753.mp4
Rumpus-20210927201803.mp4
Rumpus-20210927201813.mp4
Rumpus-20210927201822.mp4
Rumpus-20210927201832.mp4
Rumpus-20210927201843.mp4
Rumpus-20210927201853.mp4
Rumpus-20210927201902.mp4
Rumpus-20210927201912.mp4
Rumpus-20210927201924.mp4
Rumpus-20210927201933.mp4
Rumpus-20210927201943.mp4
Rumpus-20210927201952.mp4
Rumpus-20210927202002.mp4
Rumpus-20210927202013.mp4
Rumpus-20210927202023.mp4
Rumpus-20210927202033.mp4
Rumpus-20210927202042.mp4
Rumpus-20210927202054.mp4
Rumpus-20210927202103.mp4
Rumpus-20210927202113.mp4
Rumpus-20210927202123.mp4
Rumpus-20210927202134.mp4
Rumpus-20210927202144.mp4
Rumpus-20210927202153.mp4
Rumpus-20210927202203.mp4
Rumpus-20210927202212.mp4
Rumpus-20210927202224.mp4
Rumpus-20210927202234.mp4
Rumpus-20210927202243.mp4
Rumpus-20210927202253.mp4
Rumpus-20210927202304.mp4
Rumpus-20210927202314.mp4
Rumpus-20210927202323.mp4
Rumpus-20210927202333.mp4
Rumpus-20210927202343.mp4
Rumpus-20210927202354.mp4
Rumpus-20210927202404.mp4
Rumpus-20210927202413.mp4
Rumpus-20210927202423.mp4
Rumpus-20210927202434.mp4
Rumpus-20210927202444.mp4
Rumpus-20210927202454.mp4
Rumpus-20210927202503.mp4
Rumpus-20210927202515.mp4
Rumpus-20210927202524.mp4
Rumpus-20210927202534.mp4
Rumpus-20210927202544.mp4
Rumpus-20210927202553.mp4
Rumpus-20210927202605.mp4
Rumpus-20210927202614.mp4

docker-compose.yml

services:
  frigate:
    container_name: frigate-0.8
    image: blakeblackshear/frigate:0.8.4-amd64
    hostname: frigate
    restart: always
    shm_size: '300m'
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/bus/usb:/dev/bus/usb
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /usr/docker/frigate-0.8/config.yml:/config/config.yml:ro
      - /usr/docker/frigate-0.8/labelmap.txt:/labelmap.txt:ro
      - /usr/docker/frigate-0.8/database:/media/frigate/database
      - /mnt/nas/frigate:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: '1gb'
    network_mode: bridge
    ports:
      - 5000:5000
    environment:
      FRIGATE_MQTT_PASSWORD: ""
      FRIGATE_USER: ""
      FRIGATE_PASSWORD: ""
      LIBVA_DRIVER_NAME: "radeonsi"

config.yml

database:
  path: /media/frigate/database/frigate.db

detectors:
  coral:
    type: edgetpu
    device: usb

mqtt:
  host: 192.168.1.5
  port: 1883
  user: mqtt
  password: "{FRIGATE_MQTT_PASSWORD}"

ffmpeg:
  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128
  input_args: -avoid_negative_ts make_zero -flags low_delay -fflags nobuffer -fflags +genpts+discardcorrupt -rw_timeout 5000000 -strict experimental -use_wallclock_as_timestamps 1 -f live_flv
  output_args:
    clips: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy #-an

detect:
  max_disappeared: 35

motion:
  contour_area: 75
  frame_height: 180

cameras:
  Driveway:
    ffmpeg:
      inputs:
        - path: http://192.168.1.71/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.71/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - clips
    width: 896
    height: 672
    fps: 5
    clips:
      enabled: true
      pre_capture: 2
      post_capture: 3
      retain:
        default: 14
    objects:
      track:
        - person
        - vehicle
        - dog
    motion:
      mask:
        - 265,650,265,625,630,625,630,650
    rtmp:
      enabled: false
    snapshots:
      enabled: true
      timestamp: false
      crop: false
      bounding_box: true
      retain:
        default: 14

  Entrance:
    ffmpeg:
      inputs:
        - path: http://192.168.1.72/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.72/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - clips
    width: 896
    height: 672
    fps: 5
    motion:
      mask:
        - 265,650,265,625,630,625,630,650
    clips:
      enabled: true
      pre_capture: 2
      post_capture: 3
      retain:
        default: 14
    objects:
      track:
        - person
        - dog
    rtmp:
      enabled: false
    snapshots:
      enabled: true
      timestamp: false
      crop: false
      bounding_box: true
      retain:
        default: 14

  Rumpus:
    ffmpeg:
      inputs:
        - path: http://192.168.1.73/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.73/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - clips
    width: 896
    height: 672
    fps: 5
    motion:
      mask:
        - 265,650,265,625,630,625,630,650
    clips:
      enabled: true
      pre_capture: 2
      post_capture: 3
      retain:
        default: 14
    objects:
      track:
        - person
        - dog
    rtmp:
      enabled: false
    snapshots:
      enabled: true
      timestamp: false
      crop: false
      bounding_box: true
      retain:
        default: 14

  Deck:
    ffmpeg:
      inputs:
        - path: http://192.168.1.74/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.74/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - clips
    width: 896
    height: 672
    fps: 5
    motion:
      mask:
        - 265,650,265,625,630,625,630,650
        - 0,585,29,592,60,617,74,672,0,672
    clips:
      enabled: true
      pre_capture: 2
      post_capture: 3
      retain:
        default: 14
    objects:
      track:
        - person
    rtmp:
      enabled: false
    snapshots:
      enabled: true
      timestamp: false
      crop: false
      bounding_box: true
      retain:
        default: 14

  Laundry:
    ffmpeg:
      inputs:
        - path: http://192.168.1.75/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.75/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - clips
    width: 896
    height: 672
    fps: 5
    motion:
      mask:
        - 265,650,265,625,630,625,630,650
    clips:
      enabled: true
      pre_capture: 2
      post_capture: 3
      retain:
        default: 14
    objects:
      track:
        - person
    rtmp:
      enabled: false
    snapshots:
      enabled: true
      timestamp: false
      crop: false
      bounding_box: true
      retain:
        default: 14

  Balcony:
    ffmpeg:
      inputs:
        - path: http://192.168.1.76/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.76/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - clips
    width: 896
    height: 672
    fps: 5
    motion:
      mask:
        - 265,650,265,625,630,625,630,650
    clips:
      enabled: true
      pre_capture: 2
      post_capture: 3
      retain:
        default: 14
    objects:
      track:
        - person
        - vehicle
    rtmp:
      enabled: false
    snapshots:
      enabled: true
      timestamp: false
      crop: false
      bounding_box: true
      retain:
        default: 14

@blakeblackshear
Copy link
Owner

I'm not sure what could be causing this. Frigate clears files from the cache once ffmpeg stops writing to them. 0.8.4 behaves a little different because it will wait until there are no more active objects being tracked to clear them. 0.9.0 moves them regardless. Perhaps Frigate sees that a process still has a handle on those files or something, so they are skipped over.

@blakeblackshear
Copy link
Owner

This is how Frigate checks to see if ffmpeg is still in the process of writing to a file. It's the only reason I can think of that these files wouldn't be cleared in the cache already.

@jasonpstokes
Copy link
Contributor Author

jasonpstokes commented Oct 20, 2021

After upgrading to 0.9.2 last night I've had this issue twice already. I'd previously run 0.9.1 for two weeks without (I think) seeing it - so have rolled back to that.

First error line recorded by Frigate is:
[2021-10-21 00:34:17] frigate.record ERROR : Unable to convert /tmp/cache/Laundry-20211021003339.ts to /media/frigate/recordings/2021-10/21/00/Laundry/33.39.mp4

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2021-10-20 23:09:47] frigate.app                    INFO    : Starting Frigate (0.9.2-25bb515)
Starting migrations
[2021-10-20 23:09:47] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2021-10-20 23:09:47] peewee_migrate                 INFO    : There is nothing to migrate
[2021-10-20 23:09:47] frigate.mqtt                   INFO    : MQTT connected
[2021-10-20 23:09:48] detector.coral                 INFO    : Starting detection process: 217
[2021-10-20 23:09:48] frigate.app                    INFO    : Output process started: 219
[2021-10-20 23:09:48] ws4py                          INFO    : Using epoll
[2021-10-20 23:09:48] frigate.app                    INFO    : Camera processor started for Driveway: 224
[2021-10-20 23:09:48] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2021-10-20 23:09:48] frigate.app                    INFO    : Camera processor started for Entrance: 227
[2021-10-20 23:09:50] frigate.edgetpu                INFO    : TPU found
[2021-10-20 23:09:48] frigate.app                    INFO    : Camera processor started for Rumpus: 229
[2021-10-20 23:09:48] frigate.app                    INFO    : Camera processor started for Deck: 230
[2021-10-20 23:09:48] frigate.app                    INFO    : Camera processor started for Laundry: 232
[2021-10-20 23:09:48] frigate.app                    INFO    : Camera processor started for Balcony: 234
[2021-10-20 23:09:48] frigate.app                    INFO    : Capture process started for Driveway: 235
[2021-10-20 23:09:48] frigate.app                    INFO    : Capture process started for Entrance: 238
[2021-10-20 23:09:48] frigate.app                    INFO    : Capture process started for Rumpus: 241
[2021-10-20 23:09:48] frigate.app                    INFO    : Capture process started for Deck: 245
[2021-10-20 23:09:48] frigate.app                    INFO    : Capture process started for Laundry: 248
[2021-10-20 23:09:48] frigate.app                    INFO    : Capture process started for Balcony: 254
[2021-10-20 23:09:48] ws4py                          INFO    : Using epoll
[2021-10-20 23:09:57] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:39216]
[2021-10-20 23:10:00] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51566]
[2021-10-20 23:10:15] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51566]
[2021-10-20 23:10:18] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51690]
[2021-10-20 23:10:24] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51690]
[2021-10-20 23:10:24] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51726]
[2021-10-20 23:10:31] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:51726]
[2021-10-20 23:10:57] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:39216]
[2021-10-20 23:15:21] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52788]
[2021-10-20 23:15:26] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52788]
[2021-10-20 23:15:27] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52894]
[2021-10-20 23:15:33] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52894]
[2021-10-20 23:15:33] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52922]
[2021-10-20 23:15:38] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:8082 | Remote => 127.0.0.1:52922]
[2021-10-21 00:34:17] frigate.record                 ERROR   : Unable to convert /tmp/cache/Laundry-20211021003339.ts to /media/frigate/recordings/2021-10/21/00/Laundry/33.39.mp4
[2021-10-21 00:34:17] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Laundry-20211021003339.ts':
  Duration: 00:00:11.51, start: 1.400000, bitrate: 2787 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 90k tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 90k tbr, 90k tbn, 180k tbc
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 90k tbr, 90k tbn, 180k tbc
[2021-10-21 06:24:49] ffmpeg.Entrance.record         ERROR   : [segment @ 0x557ff643e440] Non-monotonous DTS in output stream 0:0; previous: 2316267720, current: 2316267720; changing to 2316267721. This may result in incorrect timestamps in the output file.

=========== [removed ~90 lines of this] =============

[2021-10-21 06:24:49] ffmpeg.Entrance.record         ERROR   : [segment @ 0x557ff643e440] Non-monotonous DTS in output stream 0:0; previous: 2348319150, current: 2348319150; changing to 2348319151. This may result in incorrect timestamps in the output file.
[2021-10-21 06:24:49] ffmpeg.Entrance.record         ERROR   : av_interleaved_write_frame(): No space left on device
[2021-10-21 06:24:49] ffmpeg.Entrance.record         ERROR   : [segment @ 0x557ff643e440] Failure occurred when ending segment '/tmp/cache/Entrance-20211021062439.ts'
[2021-10-21 06:24:49] ffmpeg.Rumpus.record           ERROR   : [segment @ 0x5649cdf2cd00] Non-monotonous DTS in output stream 0:0; previous: 2328838920, current: 2328838920; changing to 2328838921. This may result in incorrect timestamps in the output file.
[2021-10-21 06:24:49] ffmpeg.Entrance.record         ERROR   : Error writing trailer of /tmp/cache/Entrance-%Y%m%d%H%M%S.ts: No space left on device
[2021-10-21 06:24:49] ffmpeg.Rumpus.record           ERROR   : [segment @ 0x5649cdf2cd00] Non-monotonous DTS in output stream 0:0; previous: 2328981840, current: 2328981840; changing to 2328981841. This may result in incorrect timestamps in the output file.
[2021-10-21 06:24:49] watchdog.Entrance              INFO    : Terminating the existing ffmpeg process...
[2021-10-21 06:24:49] ffmpeg.Rumpus.record           ERROR   : [segment @ 0x5649cdf2cd00] Non-monotonous DTS in output stream 0:0; previous: 2329198740, current: 2329198740; changing to 2329198741. This may result in incorrect timestamps in the output file.
[2021-10-21 06:24:49] ffmpeg.Rumpus.record           ERROR   : [segment @ 0x5649cdf2cd00] Non-monotonous DTS in output stream 0:0; previous: 2329414470, current: 2329414470; changing to 2329414471. This may result in incorrect timestamps in the output file.
[2021-10-21 06:24:49] watchdog.Entrance              INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-21 06:24:49] ffmpeg.Rumpus.record           ERROR   : [segment @ 0x5649cdf2cd00] Non-monotonous DTS in output stream 0:0; previous: 2329702650, current: 2329702650; changing to 2329702651. This may result in incorrect timestamps in the output file.
...
[2021-10-21 06:24:49] ffmpeg.Rumpus.record           ERROR   : av_interleaved_write_frame(): No space left on device
[2021-10-21 06:24:49] ffmpeg.Rumpus.record           ERROR   : [segment @ 0x5649cdf2cd00] Failure occurred when ending segment '/tmp/cache/Rumpus-20211021062439.ts'
[2021-10-21 06:24:49] ffmpeg.Rumpus.record           ERROR   : Error writing trailer of /tmp/cache/Rumpus-%Y%m%d%H%M%S.ts: No space left on device
[2021-10-21 06:24:49] watchdog.Rumpus                INFO    : Terminating the existing ffmpeg process...
[2021-10-21 06:24:49] watchdog.Rumpus                INFO    : Waiting for ffmpeg to exit gracefully...

root@frigate:/opt/frigate# df -h
Filesystem                     Size  Used Avail Use% Mounted on
overlay                        233G   15G  206G   7% /
tmpfs                           64M     0   64M   0% /dev
shm                            300M   11M  290M   4% /dev/shm
/dev/sda2                      233G   15G  206G   7% /labelmap.txt
192.168.30.8:/storage/frigate  1.8T  223G  1.6T  13% /media/frigate
tmpfs                          1.0G  1.0G     0 100% /tmp/cache
tmpfs                          7.7G     0  7.7G   0% /proc/asound
tmpfs                          7.7G     0  7.7G   0% /proc/acpi
tmpfs                          7.7G     0  7.7G   0% /sys/firmware

After restart

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2021-10-21 08:57:25] frigate.app                    INFO    : Starting Frigate (0.9.2-25bb515)
Starting migrations
[2021-10-21 08:57:25] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2021-10-21 08:57:25] peewee_migrate                 INFO    : There is nothing to migrate
[2021-10-21 08:57:25] frigate.mqtt                   INFO    : MQTT connected
[2021-10-21 08:57:25] detector.coral                 INFO    : Starting detection process: 216
[2021-10-21 08:57:25] frigate.app                    INFO    : Output process started: 218
[2021-10-21 08:57:25] frigate.app                    INFO    : Camera processor started for Driveway: 223
[2021-10-21 08:57:25] frigate.app                    INFO    : Camera processor started for Entrance: 226
[2021-10-21 08:57:25] ws4py                          INFO    : Using epoll
[2021-10-21 08:57:25] frigate.app                    INFO    : Camera processor started for Rumpus: 228
[2021-10-21 08:57:25] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2021-10-21 08:57:25] frigate.app                    INFO    : Camera processor started for Deck: 229
[2021-10-21 08:57:28] frigate.edgetpu                INFO    : TPU found
[2021-10-21 08:57:25] frigate.app                    INFO    : Camera processor started for Laundry: 231
[2021-10-21 08:57:25] frigate.app                    INFO    : Camera processor started for Balcony: 232
[2021-10-21 08:57:25] frigate.app                    INFO    : Capture process started for Driveway: 234
[2021-10-21 08:57:25] frigate.app                    INFO    : Capture process started for Entrance: 236
[2021-10-21 08:57:25] frigate.app                    INFO    : Capture process started for Rumpus: 241
[2021-10-21 08:57:25] frigate.app                    INFO    : Capture process started for Deck: 243
[2021-10-21 08:57:25] frigate.app                    INFO    : Capture process started for Laundry: 246
[2021-10-21 08:57:25] frigate.app                    INFO    : Capture process started for Balcony: 248
[2021-10-21 08:57:25] ws4py                          INFO    : Using epoll
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 15652170, current: 15652170; changing to 15652171. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 15832170, current: 15832170; changing to 15832171. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Driveway.record         ERROR   : [segment @ 0x559c81bf3ac0] Non-monotonous DTS in output stream 0:0; previous: 3596400, current: 3596400; changing to 3596401. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 16011360, current: 16011360; changing to 16011361. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 16191720, current: 16191720; changing to 16191721. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 16371000, current: 16371000; changing to 16371001. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 16551630, current: 16551630; changing to 16551631. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 16732260, current: 16732260; changing to 16732261. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 16911270, current: 16911270; changing to 16911271. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 17271360, current: 17271360; changing to 17271361. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 17450460, current: 17450460; changing to 17450461. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 17631540, current: 17631540; changing to 17631541. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 17811270, current: 17811270; changing to 17811271. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 18351000, current: 18351000; changing to 18351001. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 18531180, current: 18531180; changing to 18531181. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 18891450, current: 18891450; changing to 18891451. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 19250460, current: 19250460; changing to 19250461. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 19610730, current: 19610730; changing to 19610731. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 19790280, current: 19790280; changing to 19790281. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 19969830, current: 19969830; changing to 19969831. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 20331180, current: 20331180; changing to 20331181. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 20689290, current: 20689290; changing to 20689291. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 21228570, current: 21228570; changing to 21228571. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 21408930, current: 21408930; changing to 21408931. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 22309650, current: 22309650; changing to 22309651. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 22489650, current: 22489650; changing to 22489651. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 23029650, current: 23029650; changing to 23029651. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 23211090, current: 23211090; changing to 23211091. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 23389560, current: 23389560; changing to 23389561. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 23570190, current: 23570190; changing to 23570191. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 24648750, current: 24648750; changing to 24648751. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 24829200, current: 24829200; changing to 24829201. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 25009560, current: 25009560; changing to 25009561. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 25187940, current: 25187940; changing to 25187941. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 25549380, current: 25549380; changing to 25549381. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 25907850, current: 25907850; changing to 25907851. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 26628300, current: 26628300; changing to 26628301. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 27348120, current: 27348120; changing to 27348121. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 28248840, current: 28248840; changing to 28248841. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 28427400, current: 28427400; changing to 28427401. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 28787040, current: 28787040; changing to 28787041. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 28967490, current: 28967490; changing to 28967491. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 29328480, current: 29328480; changing to 29328481. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 29506950, current: 29506950; changing to 29506951. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 29686770, current: 29686770; changing to 29686771. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 30046950, current: 30046950; changing to 30046951. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 30408120, current: 30408120; changing to 30408121. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 30947130, current: 30947130; changing to 30947131. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 31127220, current: 31127220; changing to 31127221. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 31307400, current: 31307400; changing to 31307401. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 31846680, current: 31846680; changing to 31846681. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 32026590, current: 32026590; changing to 32026591. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 32206950, current: 32206950; changing to 32206951. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 32385960, current: 32385960; changing to 32385961. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 32926410, current: 32926410; changing to 32926411. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 33106050, current: 33106050; changing to 33106051. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 33646410, current: 33646410; changing to 33646411. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 33826320, current: 33826320; changing to 33826321. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 34006050, current: 34006050; changing to 34006051. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 34186950, current: 34186950; changing to 34186951. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 34544880, current: 34544880; changing to 34544881. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 34905510, current: 34905510; changing to 34905511. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 35446590, current: 35446590; changing to 35446591. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 35625690, current: 35625690; changing to 35625691. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 35806860, current: 35806860; changing to 35806861. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 35985510, current: 35985510; changing to 35985511. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 37965780, current: 37965780; changing to 37965781. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 38144970, current: 38144970; changing to 38144971. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 38504700, current: 38504700; changing to 38504701. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 38684700, current: 38684700; changing to 38684701. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 39224340, current: 39224340; changing to 39224341. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 39404250, current: 39404250; changing to 39404251. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 39586050, current: 39586050; changing to 39586051. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 40125240, current: 40125240; changing to 40125241. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 40305150, current: 40305150; changing to 40305151. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 40484160, current: 40484160; changing to 40484161. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 40842810, current: 40842810; changing to 40842811. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 41203350, current: 41203350; changing to 41203351. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 41383980, current: 41383980; changing to 41383981. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Laundry.record          ERROR   : [segment @ 0x55c74fe78a80] Non-monotonous DTS in output stream 0:0; previous: 21953880, current: 21953880; changing to 21953881. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Laundry.record          ERROR   : [segment @ 0x55c74fe78a80] Non-monotonous DTS in output stream 0:0; previous: 22133700, current: 22133700; changing to 22133701. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Laundry.record          ERROR   : [segment @ 0x55c74fe78a80] Non-monotonous DTS in output stream 0:0; previous: 22313250, current: 22313250; changing to 22313251. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Laundry.record          ERROR   : [segment @ 0x55c74fe78a80] Non-monotonous DTS in output stream 0:0; previous: 23031810, current: 23031810; changing to 23031811. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Laundry.record          ERROR   : [segment @ 0x55c74fe78a80] Non-monotonous DTS in output stream 0:0; previous: 23212530, current: 23212530; changing to 23212531. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Laundry.record          ERROR   : [segment @ 0x55c74fe78a80] Non-monotonous DTS in output stream 0:0; previous: 23392890, current: 23392890; changing to 23392891. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 41562900, current: 41562900; changing to 41562901. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 41742720, current: 41742720; changing to 41742721. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 41923890, current: 41923890; changing to 41923891. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 42104250, current: 42104250; changing to 42104251. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 42282360, current: 42282360; changing to 42282361. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 42642630, current: 42642630; changing to 42642631. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 42823620, current: 42823620; changing to 42823621. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 43003170, current: 43003170; changing to 43003171. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 43182630, current: 43182630; changing to 43182631. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 44083170, current: 44083170; changing to 44083171. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 44803080, current: 44803080; changing to 44803081. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 44981820, current: 44981820; changing to 44981821. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 45343080, current: 45343080; changing to 45343081. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 45522900, current: 45522900; changing to 45522901. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 45703710, current: 45703710; changing to 45703711. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 45882990, current: 45882990; changing to 45882991. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 46241280, current: 46241280; changing to 46241281. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 46422180, current: 46422180; changing to 46422181. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Non-monotonous DTS in output stream 0:0; previous: 46602450, current: 46602450; changing to 46602451. This may result in incorrect timestamps in the output file.
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : av_interleaved_write_frame(): No space left on device
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : [segment @ 0x55be355a6f80] Failure occurred when ending segment '/tmp/cache/Entrance-20211021090618.ts'
[2021-10-21 09:06:25] ffmpeg.Entrance.record         ERROR   : Error writing trailer of /tmp/cache/Entrance-%Y%m%d%H%M%S.ts: No space left on device
[2021-10-21 09:06:25] watchdog.Entrance              INFO    : Terminating the existing ffmpeg process...
[2021-10-21 09:06:25] watchdog.Entrance              INFO    : Waiting for ffmpeg to exit gracefully...

@blakeblackshear
Copy link
Owner

It's odd that you have multiple streams listed by ffmpeg like this. For some reason it isn't able to convert the .ts file to .mp4 for you. Any chance you can send me one of the .ts files in your /tmp/cache folder? Also, can you post the ffprobe output of the stream you use for record?

    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 90k tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 90k tbr, 90k tbn, 180k tbc
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 90k tbr, 90k tbn, 180k tbc

@blakeblackshear
Copy link
Owner

Also, can you get the output of ls -lah /tmp/cache?

@blakeblackshear
Copy link
Owner

I'm not sure what the difference is, but someone sent me a RLC-511 for testing, so I might be able to reproduce this.

@rhatguy
Copy link

rhatguy commented Oct 21, 2021

Just updated to 0.92 myself, and experiencing issues with my /tmp/cache filling up. 0.91 was running stable for over 2 weeks and the only thing that changed in my config was the version of frigate. Nothing special in my /tmp/cache other than files slowly piling up. Interestingly I do see files getting removed but over the course of probably an hour or so, it starts to fill up. I also run reolink cams. /tmp/cache is 1GB in total. output of ffprobe and ls -lah below.

root@86cdc0b35448:/tmp/cache# ffprobe underdeck_cam-20211020200049.ts
ffprobe version 4.3.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-rkmpp --enable-libdrm --enable-v4l2_m2m --enable-neon --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from 'underdeck_cam-20211020200049.ts':
  Duration: 00:00:09.91, start: 1.400000, bitrate: 8317 kb/s
  Program 1
    Metadata:
      service_name    : Stream
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuvj420p(pc, progressive), 2560x1440, 90k tbr, 90k tbn, 180k tbc
root@86cdc0b35448:/tmp/cache# ls -lah /tmp/cache
total 673M
drwxr-xr-x 2 root root 1.9K Oct 20 19:59 .
drwxrwxrwt 1 root root 4.0K Oct 20 12:23 ..
-rw-r--r-- 1 root root 8.3M Oct 20 19:56 back_porch_cam-20211020195630.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:56 back_porch_cam-20211020195640.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 back_porch_cam-20211020195650.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 back_porch_cam-20211020195700.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 back_porch_cam-20211020195710.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 back_porch_cam-20211020195720.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 back_porch_cam-20211020195730.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 back_porch_cam-20211020195740.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 back_porch_cam-20211020195750.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 back_porch_cam-20211020195800.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 back_porch_cam-20211020195810.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 back_porch_cam-20211020195820.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 back_porch_cam-20211020195830.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 back_porch_cam-20211020195840.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:59 back_porch_cam-20211020195850.ts
-rw-r--r-- 1 root root 4.8M Oct 20 19:59 back_porch_cam-20211020195900.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:56 backyard_cam-20211020195638.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:56 backyard_cam-20211020195648.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 backyard_cam-20211020195658.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 backyard_cam-20211020195708.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 backyard_cam-20211020195718.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 backyard_cam-20211020195728.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 backyard_cam-20211020195738.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 backyard_cam-20211020195748.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 backyard_cam-20211020195758.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 backyard_cam-20211020195808.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 backyard_cam-20211020195818.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 backyard_cam-20211020195828.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 backyard_cam-20211020195838.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 backyard_cam-20211020195848.ts
-rw-r--r-- 1 root root 6.0M Oct 20 19:59 backyard_cam-20211020195858.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:56 driveway_cam-20211020195638.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:56 driveway_cam-20211020195648.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:57 driveway_cam-20211020195658.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:57 driveway_cam-20211020195708.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:57 driveway_cam-20211020195718.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:57 driveway_cam-20211020195728.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:57 driveway_cam-20211020195738.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:57 driveway_cam-20211020195748.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:58 driveway_cam-20211020195758.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:58 driveway_cam-20211020195808.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:58 driveway_cam-20211020195818.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:58 driveway_cam-20211020195828.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:58 driveway_cam-20211020195838.ts
-rw-r--r-- 1 root root 4.2M Oct 20 19:58 driveway_cam-20211020195848.ts
-rw-r--r-- 1 root root 2.8M Oct 20 19:59 driveway_cam-20211020195858.ts
-rw-r--r-- 1 root root 7.5M Oct 20 19:56 front_door_cam-20211020195638.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:57 front_door_cam-20211020195650.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:57 front_door_cam-20211020195700.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:57 front_door_cam-20211020195710.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:57 front_door_cam-20211020195720.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:57 front_door_cam-20211020195730.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:57 front_door_cam-20211020195740.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:58 front_door_cam-20211020195750.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:58 front_door_cam-20211020195800.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:58 front_door_cam-20211020195810.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:58 front_door_cam-20211020195820.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:58 front_door_cam-20211020195830.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:58 front_door_cam-20211020195840.ts
-rw-r--r-- 1 root root 6.2M Oct 20 19:59 front_door_cam-20211020195850.ts
-rw-r--r-- 1 root root 3.5M Oct 20 19:59 front_door_cam-20211020195900.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:56 garage_cam-20211020195630.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:56 garage_cam-20211020195640.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 garage_cam-20211020195650.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 garage_cam-20211020195700.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 garage_cam-20211020195710.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 garage_cam-20211020195720.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 garage_cam-20211020195729.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:57 garage_cam-20211020195740.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 garage_cam-20211020195750.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 garage_cam-20211020195800.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 garage_cam-20211020195810.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 garage_cam-20211020195820.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 garage_cam-20211020195830.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:58 garage_cam-20211020195840.ts
-rw-r--r-- 1 root root 8.3M Oct 20 19:59 garage_cam-20211020195850.ts
-rw-r--r-- 1 root root 4.8M Oct 20 19:59 garage_cam-20211020195900.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:56 underdeck_cam-20211020195639.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:56 underdeck_cam-20211020195649.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:57 underdeck_cam-20211020195659.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:57 underdeck_cam-20211020195709.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:57 underdeck_cam-20211020195719.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:57 underdeck_cam-20211020195729.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:57 underdeck_cam-20211020195739.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:57 underdeck_cam-20211020195749.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:58 underdeck_cam-20211020195759.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:58 underdeck_cam-20211020195809.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:58 underdeck_cam-20211020195819.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:58 underdeck_cam-20211020195829.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:58 underdeck_cam-20211020195839.ts
-rw-r--r-- 1 root root 9.9M Oct 20 19:58 underdeck_cam-20211020195849.ts
-rw-r--r-- 1 root root 5.8M Oct 20 19:59 underdeck_cam-20211020195859.ts

@blakeblackshear
Copy link
Owner

@rhatguy what about your frigate logs?

@blakeblackshear blakeblackshear pinned this issue Oct 21, 2021
@blakeblackshear
Copy link
Owner

blakeblackshear commented Oct 21, 2021

Currently trying to reproduce this with my reolink RLC-511 and this config:

  reolink:
    ffmpeg:
      input_args:
        - -avoid_negative_ts
        - make_zero
        - -fflags
        - nobuffer+genpts+discardcorrupt
        - -flags
        - low_delay
        - -strict
        - experimental
        - -analyzeduration
        - 1000M
        - -probesize
        - 1000M
        - -rw_timeout
        - "5000000"
      inputs:
        - path: http://192.168.2.187/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user=admin&password=
          roles:
            - detect
        - path: http://192.168.2.187/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=admin&password=
          roles:
            - record
    detect:
      width: 896
      height: 672
      fps: 5
    record:
      enabled: true
    objects:
      track:
        - person
    rtmp:
      enabled: false
    snapshots:
      enabled: false

@jasonpstokes
Copy link
Contributor Author

jasonpstokes commented Oct 21, 2021

Sorry Blake, been tied up at work.

Changed my stack to 0.9.2 and before I'd collected all this, the tmpfs was full again!

File from tmp/cache here (for a couple hours)

docker-compose.yml
services:
  frigate:
    container_name: frigate
    image: blakeblackshear/frigate:0.9.2-amd64
    hostname: frigate
    restart: always
    shm_size: '300m'
    devices:
      - /dev/dri/renderD128:/dev/dri/renderD128
      - /dev/bus/usb:/dev/bus/usb
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /usr/docker/frigate/config.yml:/config/config.yml:ro
      - /usr/docker/frigate/labelmap.txt:/labelmap.txt:ro
      - /usr/docker/frigate/database:/media/frigate/database
      - /mnt/nas/frigate:/media/frigate
      - type: tmpfs
        target: /tmp/cache
        tmpfs:
          size: '1gb'
    network_mode: bridge
    ports:
      - 0.0.0.0:5000:5000
    environment:
      FRIGATE_MQTT_PASSWORD: ""
      FRIGATE_USER: ""
      FRIGATE_PASSWORD: ""
      LIBVA_DRIVER_NAME: "radeonsi"
partial config.yml
detect:
  width: 896
  height: 672
  fps: 5

ffmpeg:
  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
  input_args: -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rw_timeout 5000000 -use_wallclock_as_timestamps 1 -f live_flv

record:
  retain_days: 0
  enabled: true
  events:
    pre_capture: 3
    retain:
      default: 15

cameras:
  Driveway:
    ffmpeg:
      inputs:
        - path: rtmp://192.168.1.71/bcs/channel0_ext.bcs?channel=0&stream=2&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: rtmp://192.168.1.71/bcs/channel0_main.bcs?channel=0&stream=0&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
        - vehicle
        - dog
      filters:
        person:
          mask: 
            - 510,120,540,120,540,160,510,160
    rtmp:
      enabled: false
ls -lah /tmp/cache
root@frigate:/tmp/cache# ls -lah /tmp/cache
total 1015M
drwxr-xr-x 2 root root 3.4K Oct 21 14:04 .
drwxrwxrwt 1 root root 4.0K Oct 21 13:34 ..
-rw-r--r-- 1 root root 7.9M Oct 21 13:53 Balcony-20211021135333.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:53 Balcony-20211021135343.ts
-rw-r--r-- 1 root root 1.0M Oct 21 13:53 Balcony-20211021135353.ts
-rw-r--r-- 1 root root 6.3M Oct 21 13:54 Balcony-20211021135404.ts
-rw-r--r-- 1 root root 7.8M Oct 21 13:54 Balcony-20211021135413.ts
-rw-r--r-- 1 root root 1.5M Oct 21 13:59 Balcony-20211021135924.ts
-rw-r--r-- 1 root root 2.3M Oct 21 13:59 Balcony-20211021135934.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:59 Balcony-20211021135944.ts
-rw-r--r-- 1 root root 7.0M Oct 21 14:00 Balcony-20211021135954.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:00 Balcony-20211021140014.ts
-rw-r--r-- 1 root root 3.5M Oct 21 14:00 Balcony-20211021140022.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:00 Balcony-20211021140034.ts
-rw-r--r-- 1 root root 2.3M Oct 21 14:00 Balcony-20211021140044.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:01 Balcony-20211021140054.ts
-rw-r--r-- 1 root root 7.8M Oct 21 14:01 Balcony-20211021140102.ts
-rw-r--r-- 1 root root 6.7M Oct 21 14:01 Balcony-20211021140112.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Balcony-20211021140124.ts
-rw-r--r-- 1 root root 1.3M Oct 21 14:01 Balcony-20211021140134.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Balcony-20211021140144.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Balcony-20211021140154.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Balcony-20211021140204.ts
-rw-r--r-- 1 root root 2.8M Oct 21 14:02 Balcony-20211021140214.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Balcony-20211021140224.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Balcony-20211021140234.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:02 Balcony-20211021140244.ts
-rw-r--r-- 1 root root 4.5M Oct 21 14:02 Balcony-20211021140252.ts
-rw-r--r-- 1 root root 7.8M Oct 21 14:03 Balcony-20211021140304.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:03 Balcony-20211021140326.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Balcony-20211021140334.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Balcony-20211021140344.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:04 Balcony-20211021140354.ts
-rw-r--r-- 1 root root 3.8M Oct 21 14:04 Balcony-20211021140404.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:04 Balcony-20211021140416.ts
-rw-r--r-- 1 root root 5.7M Oct 21 14:04 Balcony-20211021140426.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:53 Deck-20211021135335.ts
-rw-r--r-- 1 root root 6.8M Oct 21 13:53 Deck-20211021135345.ts
-rw-r--r-- 1 root root 6.3M Oct 21 13:54 Deck-20211021135405.ts
-rw-r--r-- 1 root root 9.4M Oct 21 13:54 Deck-20211021135413.ts
-rw-r--r-- 1 root root 1.0M Oct 21 13:59 Deck-20211021135925.ts
-rw-r--r-- 1 root root 1.3M Oct 21 13:59 Deck-20211021135935.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:59 Deck-20211021135945.ts
-rw-r--r-- 1 root root 6.0M Oct 21 14:00 Deck-20211021135955.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:00 Deck-20211021140015.ts
-rw-r--r-- 1 root root 1.2M Oct 21 14:00 Deck-20211021140025.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:00 Deck-20211021140035.ts
-rw-r--r-- 1 root root 1.3M Oct 21 14:00 Deck-20211021140045.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Deck-20211021140055.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Deck-20211021140105.ts
-rw-r--r-- 1 root root 4.3M Oct 21 14:01 Deck-20211021140115.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Deck-20211021140125.ts
-rw-r--r-- 1 root root 644K Oct 21 14:01 Deck-20211021140135.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Deck-20211021140145.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Deck-20211021140155.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Deck-20211021140205.ts
-rw-r--r-- 1 root root 2.5M Oct 21 14:02 Deck-20211021140215.ts
-rw-r--r-- 1 root root 7.5M Oct 21 14:02 Deck-20211021140225.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Deck-20211021140245.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Deck-20211021140255.ts
-rw-r--r-- 1 root root 5.7M Oct 21 14:03 Deck-20211021140305.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Deck-20211021140325.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Deck-20211021140335.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Deck-20211021140345.ts
-rw-r--r-- 1 root root 5.0M Oct 21 14:04 Deck-20211021140355.ts
-rw-r--r-- 1 root root 2.8M Oct 21 14:04 Deck-20211021140405.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:04 Deck-20211021140415.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:04 Deck-20211021140425.ts
-rw-r--r-- 1 root root 512K Oct 21 14:04 Deck-20211021140433.ts
-rw-r--r-- 1 root root 6.3M Oct 21 13:53 Driveway-20211021135327.ts
-rw-r--r-- 1 root root 9.4M Oct 21 13:53 Driveway-20211021135336.ts
-rw-r--r-- 1 root root 4.8M Oct 21 13:53 Driveway-20211021135348.ts
-rw-r--r-- 1 root root 6.3M Oct 21 13:54 Driveway-20211021135407.ts
-rw-r--r-- 1 root root 6.3M Oct 21 13:59 Driveway-20211021135927.ts
-rw-r--r-- 1 root root 1.0M Oct 21 13:59 Driveway-20211021135936.ts
-rw-r--r-- 1 root root 6.3M Oct 21 13:59 Driveway-20211021135947.ts
-rw-r--r-- 1 root root 9.4M Oct 21 14:00 Driveway-20211021135956.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:00 Driveway-20211021140008.ts
-rw-r--r-- 1 root root 9.4M Oct 21 14:00 Driveway-20211021140016.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:00 Driveway-20211021140028.ts
-rw-r--r-- 1 root root 8.5M Oct 21 14:00 Driveway-20211021140036.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:01 Driveway-20211021140055.ts
-rw-r--r-- 1 root root 9.4M Oct 21 14:01 Driveway-20211021140104.ts
-rw-r--r-- 1 root root 3.8M Oct 21 14:01 Driveway-20211021140116.ts
-rw-r--r-- 1 root root 5.5M Oct 21 14:01 Driveway-20211021140127.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:01 Driveway-20211021140147.ts
-rw-r--r-- 1 root root 9.4M Oct 21 14:02 Driveway-20211021140156.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:02 Driveway-20211021140208.ts
-rw-r--r-- 1 root root 1.5M Oct 21 14:02 Driveway-20211021140216.ts
-rw-r--r-- 1 root root 5.5M Oct 21 14:02 Driveway-20211021140227.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:02 Driveway-20211021140247.ts
-rw-r--r-- 1 root root 2.0M Oct 21 14:02 Driveway-20211021140256.ts
-rw-r--r-- 1 root root 5.3M Oct 21 14:03 Driveway-20211021140307.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:03 Driveway-20211021140327.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:03 Driveway-20211021140336.ts
-rw-r--r-- 1 root root 9.4M Oct 21 14:03 Driveway-20211021140344.ts
-rw-r--r-- 1 root root 4.3M Oct 21 14:04 Driveway-20211021140356.ts
-rw-r--r-- 1 root root 1.0M Oct 21 14:04 Driveway-20211021140407.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:04 Driveway-20211021140415.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:04 Driveway-20211021140424.ts
-rw-r--r-- 1 root root 1.5M Oct 21 14:04 Driveway-20211021140432.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:53 Entrance-20211021135335.ts
-rw-r--r-- 1 root root 7.0M Oct 21 13:53 Entrance-20211021135345.ts
-rw-r--r-- 1 root root 6.3M Oct 21 13:54 Entrance-20211021135404.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:54 Entrance-20211021135413.ts
-rw-r--r-- 1 root root 6.3M Oct 21 13:59 Entrance-20211021135924.ts
-rw-r--r-- 1 root root 3.6M Oct 21 13:59 Entrance-20211021135933.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:59 Entrance-20211021135944.ts
-rw-r--r-- 1 root root 6.8M Oct 21 14:00 Entrance-20211021135955.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:00 Entrance-20211021140014.ts
-rw-r--r-- 1 root root 1.3M Oct 21 14:00 Entrance-20211021140025.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:00 Entrance-20211021140034.ts
-rw-r--r-- 1 root root 2.0M Oct 21 14:00 Entrance-20211021140045.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Entrance-20211021140055.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Entrance-20211021140105.ts
-rw-r--r-- 1 root root 4.5M Oct 21 14:01 Entrance-20211021140115.ts
-rw-r--r-- 1 root root 7.8M Oct 21 14:01 Entrance-20211021140124.ts
-rw-r--r-- 1 root root 1.0M Oct 21 14:01 Entrance-20211021140135.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Entrance-20211021140144.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Entrance-20211021140155.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Entrance-20211021140205.ts
-rw-r--r-- 1 root root 2.5M Oct 21 14:02 Entrance-20211021140215.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Entrance-20211021140224.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Entrance-20211021140235.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Entrance-20211021140245.ts
-rw-r--r-- 1 root root 2.8M Oct 21 14:02 Entrance-20211021140255.ts
-rw-r--r-- 1 root root 7.5M Oct 21 14:03 Entrance-20211021140304.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:03 Entrance-20211021140324.ts
-rw-r--r-- 1 root root 9.4M Oct 21 14:03 Entrance-20211021140333.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Entrance-20211021140345.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:04 Entrance-20211021140355.ts
-rw-r--r-- 1 root root 5.0M Oct 21 14:04 Entrance-20211021140403.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:04 Entrance-20211021140414.ts
-rw-r--r-- 1 root root 7.0M Oct 21 14:04 Entrance-20211021140425.ts
-rw-r--r-- 1 root root 9.4M Oct 21 13:53 Laundry-20211021135333.ts
-rw-r--r-- 1 root root 7.3M Oct 21 13:53 Laundry-20211021135344.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:54 Laundry-20211021135404.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:54 Laundry-20211021135414.ts
-rw-r--r-- 1 root root 1.8M Oct 21 13:59 Laundry-20211021135924.ts
-rw-r--r-- 1 root root 2.3M Oct 21 13:59 Laundry-20211021135934.ts
-rw-r--r-- 1 root root 7.9M Oct 21 13:59 Laundry-20211021135944.ts
-rw-r--r-- 1 root root 7.8M Oct 21 14:00 Laundry-20211021135954.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:00 Laundry-20211021140004.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:00 Laundry-20211021140014.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:00 Laundry-20211021140024.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:00 Laundry-20211021140034.ts
-rw-r--r-- 1 root root 2.5M Oct 21 14:00 Laundry-20211021140044.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Laundry-20211021140054.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Laundry-20211021140104.ts
-rw-r--r-- 1 root root 5.0M Oct 21 14:01 Laundry-20211021140114.ts
-rw-r--r-- 1 root root 6.3M Oct 21 14:01 Laundry-20211021140124.ts
-rw-r--r-- 1 root root 2.8M Oct 21 14:01 Laundry-20211021140132.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:01 Laundry-20211021140144.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Laundry-20211021140154.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Laundry-20211021140204.ts
-rw-r--r-- 1 root root 2.8M Oct 21 14:02 Laundry-20211021140214.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Laundry-20211021140224.ts
-rw-r--r-- 1 root root 768K Oct 21 14:02 Laundry-20211021140234.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:02 Laundry-20211021140244.ts
-rw-r--r-- 1 root root 3.1M Oct 21 14:02 Laundry-20211021140254.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Laundry-20211021140304.ts
-rw-r--r-- 1 root root 508K Oct 21 14:03 Laundry-20211021140314.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Laundry-20211021140324.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Laundry-20211021140334.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:03 Laundry-20211021140344.ts
-rw-r--r-- 1 root root 5.5M Oct 21 14:04 Laundry-20211021140354.ts
-rw-r--r-- 1 root root 3.8M Oct 21 14:04 Laundry-20211021140404.ts
-rw-r--r-- 1 root root 7.9M Oct 21 14:04 Laundry-20211021140414.ts
-rw-r--r-- 1 root root 7.3M Oct 21 14:04 Laundry-20211021140424.ts
ffprobe
root@frigate:/tmp/cache# ffprobe Entrance-20211021135005.ts
ffprobe version 4.3.1 Copyright (c) 2007-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from 'Entrance-20211021135005.ts':
  Duration: 00:00:05.80, start: 1.400000, bitrate: 6719 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 90k tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s

@blakeblackshear
Copy link
Owner

@jasonpstokes any particular reason you aren't running the recommended settings for reolink? I just ran those with my camera overnight without any issues.

@blakeblackshear
Copy link
Owner

That .ts file converted without an issue. Can you look for one that has a failure message like below, but isn't due to running out of disk space?

Unable to convert /tmp/cache/Laundry-20211021003339.ts to /media/frigate/recordings/2021-10/21/00/Laundry/33.39.mp4

@rhatguy
Copy link

rhatguy commented Oct 21, 2021

Just grabbed the frigate logs from my system. Doesn't look terribly interesting. I can drop back to 0.91 and test if this happens there, but the only thing that changed on my setup between 0.91 and 0.92 was the frigate version. No other config changes.

[2021-10-21 07:52:17] frigate.app INFO : Capture process started for driveway_cam: 239
[2021-10-21 07:52:17] frigate.app INFO : Capture process started for backyard_cam: 242
[2021-10-21 07:52:17] frigate.app INFO : Capture process started for back_porch_cam: 247
[2021-10-21 07:52:17] frigate.app INFO : Capture process started for underdeck_cam: 254
[2021-10-21 07:52:17] ws4py INFO : Using epoll
[2021-10-21 07:52:20] frigate.edgetpu INFO : TPU found
[2021-10-21 08:18:37] ffmpeg.driveway_cam.record ERROR : av_interleaved_write_frame(): No space left on device
[2021-10-21 08:18:37] ffmpeg.driveway_cam.record ERROR : [segment @ 0x556f9a5ff0] Failure occurred when ending segment '/tmp/cache/driveway_cam-20211021081829.ts'
[2021-10-21 08:18:37] ffmpeg.driveway_cam.record ERROR : Error writing trailer of /tmp/cache/driveway_cam-%Y%m%d%H%M%S.ts: No space left on device
[2021-10-21 08:18:37] watchdog.driveway_cam INFO : Terminating the existing ffmpeg process...
[2021-10-21 08:18:37] watchdog.driveway_cam INFO : Waiting for ffmpeg to exit gracefully...

@rhatguy
Copy link

rhatguy commented Oct 21, 2021

Dropped my system back to 0.9.1 and the issue doesn't appear to be happening anymore. After watching it for a while, my /tmp/cache stays <~10% utilization. Not sure if it has anything to do with it, but my reolinks are running through an rtsp-simple-server on another machine as thats the only way I've been able to keep them stable with frigate.

root@raspberrypi:/# cat /root/docker-compose.yml
version: "3.6"
services:
  frigate:
    container_name: frigate
    restart: unless-stopped
    privileged: true
    image: blakeblackshear/frigate:0.9.1-aarch64
    shm_size: '512mb'
    devices:
      - /dev/bus/usb:/dev/bus/usb
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /config/config.yml:/config/config.yml:ro
      - /storage:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "1935:1935" # RTMP feeds
    environment:
      FRIGATE_RTSP_PASSWORD: "password"
    healthcheck:
      test: ["CMD", "wget" , "-q", "-O-", "http://localhost:5000"]
      interval: 30s
      timeout: 10s
      retries: 5
      start_period: 3m
root@raspberrypi:/# cat /config/config.yml
mqtt:
  host: <mqtthost>
  user: <user>
  password: <pass>
ffmpeg:
  hwaccel_args: -c:v h264_v4l2m2m
  output_args:
    detect: -f rawvideo -pix_fmt yuv420p
    record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an
detectors:
  coral:
    type: edgetpu
    device: usb
detect:
  width: 640
  height: 480
  fps: 5
record:
  enabled: True
  retain_days: 0
  events:
    pre_capture: 10
    post_capture: 5
    objects:
      - person
    retain:
      default: 30
snapshots:
  enabled: True
  timestamp: False
  bounding_box: True
  crop: False
  retain:
    default: 30

cameras:
#front_door_cam
  front_door_cam:
    ffmpeg:
      inputs:
        - path: rtsp://10.1.1.1:8554/front_door_cam_sub
          roles:
            - detect
            - rtmp
        - path: rtsp://10.1.1.1:8554/front_door_cam_main
          roles:
            - record
    detect:
      width: 640
      height: 480
      fps: 4
    motion:
      mask: 137,193,303,143,431,114,640,132,640,0,0,0,0,260
    objects:
      mask: 137,193,303,143,431,114,640,132,640,0,0,0,0,260
      track:
        - person

@jasonpstokes
Copy link
Contributor Author

jasonpstokes commented Oct 21, 2021

@jasonpstokes any particular reason you aren't running the recommended settings for reolink? I just ran those with my camera overnight without any issues.

Valid question! As I'm running a different (upgraded?) camera model (520A) to most here with beta firmware, as part of this conversation here I've been trying different streams and FFmpeg options. So the above config is what I was running in 0.9.1, when you released 0.9.2. :-)

Switching back to the Reolink recommended and 0.9.2 fills tmpfs within a few minutes - every time. None of the files listed in the errors are present in tmpfs. Spent a few hours last night and again this morning trying different things, hoping it was something I'm doing, but it's the same config as 0.9.1! Sometimes, once tmpfs is full, Frigate will lock Docker up and I have to hard power off my Debian server to recover!!

logs
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[2021-10-22 08:10:13] frigate.app                    INFO    : Starting Frigate (0.9.2-25bb515)
Starting migrations
[2021-10-22 08:10:13] peewee_migrate                 INFO    : Starting migrations
There is nothing to migrate
[2021-10-22 08:10:13] peewee_migrate                 INFO    : There is nothing to migrate
[2021-10-22 08:10:13] frigate.mqtt                   INFO    : MQTT connected
[2021-10-22 08:10:13] detector.coral                 INFO    : Starting detection process: 218
[2021-10-22 08:10:13] frigate.app                    INFO    : Output process started: 220
[2021-10-22 08:10:13] frigate.app                    INFO    : Camera processor started for Driveway: 224
[2021-10-22 08:10:13] frigate.app                    INFO    : Camera processor started for Entrance: 226
[2021-10-22 08:10:13] ws4py                          INFO    : Using epoll
[2021-10-22 08:10:13] frigate.app                    INFO    : Camera processor started for Rumpus: 229
[2021-10-22 08:10:13] frigate.app                    INFO    : Camera processor started for Deck: 231
[2021-10-22 08:10:13] frigate.app                    INFO    : Camera processor started for Laundry: 232
[2021-10-22 08:10:13] frigate.app                    INFO    : Camera processor started for Balcony: 234
[2021-10-22 08:10:13] frigate.app                    INFO    : Capture process started for Driveway: 236
[2021-10-22 08:10:13] frigate.app                    INFO    : Capture process started for Entrance: 239
[2021-10-22 08:10:13] frigate.app                    INFO    : Capture process started for Rumpus: 242
[2021-10-22 08:10:13] frigate.app                    INFO    : Capture process started for Deck: 246
[2021-10-22 08:10:13] frigate.app                    INFO    : Capture process started for Laundry: 250
[2021-10-22 08:10:13] frigate.app                    INFO    : Capture process started for Balcony: 253
[2021-10-22 08:10:13] ws4py                          INFO    : Using epoll
[2021-10-22 08:10:13] frigate.edgetpu                INFO    : Attempting to load TPU as usb
[2021-10-22 08:10:15] frigate.edgetpu                INFO    : TPU found
[2021-10-22 08:10:34] frigate.record                 ERROR   : Unable to convert /tmp/cache/Balcony-20211022081016.ts to /media/frigate/recordings/2021-10/22/08/Balcony/10.16.mp4
[2021-10-22 08:10:34] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Balcony-20211022081016.ts':
  Duration: 00:00:10.62, start: 2.816000, bitrate: 6179 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:34] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:51498]
[2021-10-22 08:10:34] frigate.record                 ERROR   : Unable to convert /tmp/cache/Laundry-20211022081016.ts to /media/frigate/recordings/2021-10/22/08/Laundry/10.16.mp4
[2021-10-22 08:10:34] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Laundry-20211022081016.ts':
  Duration: 00:00:10.62, start: 2.827000, bitrate: 6178 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:35] frigate.record                 ERROR   : Unable to convert /tmp/cache/Entrance-20211022081016.ts to /media/frigate/recordings/2021-10/22/08/Entrance/10.16.mp4
[2021-10-22 08:10:35] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Entrance-20211022081016.ts':
  Duration: 00:00:10.62, start: 2.794000, bitrate: 6172 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:36] frigate.record                 ERROR   : Unable to convert /tmp/cache/Deck-20211022081015.ts to /media/frigate/recordings/2021-10/22/08/Deck/10.15.mp4
[2021-10-22 08:10:36] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Deck-20211022081015.ts':
  Duration: 00:00:10.69, start: 2.759000, bitrate: 6135 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:42] frigate.record                 ERROR   : Unable to convert /tmp/cache/Rumpus-20211022081027.ts to /media/frigate/recordings/2021-10/22/08/Rumpus/10.27.mp4
[2021-10-22 08:10:42] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Rumpus-20211022081027.ts':
  Duration: 00:00:10.02, start: 1.400000, bitrate: 6545 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:43] frigate.record                 ERROR   : Unable to convert /tmp/cache/Balcony-20211022081027.ts to /media/frigate/recordings/2021-10/22/08/Balcony/10.27.mp4
[2021-10-22 08:10:43] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Balcony-20211022081027.ts':
  Duration: 00:00:10.03, start: 1.400000, bitrate: 6543 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:44] frigate.record                 ERROR   : Unable to convert /tmp/cache/Driveway-20211022081027.ts to /media/frigate/recordings/2021-10/22/08/Driveway/10.27.mp4
[2021-10-22 08:10:44] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Driveway-20211022081027.ts':
  Duration: 00:00:12.01, start: 1.400000, bitrate: 6538 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:45] frigate.record                 ERROR   : Unable to convert /tmp/cache/Laundry-20211022081026.ts to /media/frigate/recordings/2021-10/22/08/Laundry/10.26.mp4
[2021-10-22 08:10:45] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Laundry-20211022081026.ts':
  Duration: 00:00:10.04, start: 1.400000, bitrate: 6532 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:46] frigate.record                 ERROR   : Unable to convert /tmp/cache/Entrance-20211022081026.ts to /media/frigate/recordings/2021-10/22/08/Entrance/10.26.mp4
[2021-10-22 08:10:46] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Entrance-20211022081026.ts':
  Duration: 00:00:10.01, start: 1.400000, bitrate: 6552 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:46] frigate.record                 ERROR   : Unable to convert /tmp/cache/Deck-20211022081026.ts to /media/frigate/recordings/2021-10/22/08/Deck/10.26.mp4
[2021-10-22 08:10:46] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Deck-20211022081026.ts':
  Duration: 00:00:10.03, start: 1.400000, bitrate: 6542 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:52] frigate.record                 ERROR   : Unable to convert /tmp/cache/Driveway-20211022081039.ts to /media/frigate/recordings/2021-10/22/08/Driveway/10.39.mp4
[2021-10-22 08:10:52] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Driveway-20211022081039.ts':
  Duration: 00:00:08.02, start: 1.400000, bitrate: 6526 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:53] frigate.record                 ERROR   : Unable to convert /tmp/cache/Rumpus-20211022081037.ts to /media/frigate/recordings/2021-10/22/08/Rumpus/10.37.mp4
[2021-10-22 08:10:53] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Rumpus-20211022081037.ts':
  Duration: 00:00:10.01, start: 1.400000, bitrate: 6556 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:54] frigate.record                 ERROR   : Unable to convert /tmp/cache/Balcony-20211022081037.ts to /media/frigate/recordings/2021-10/22/08/Balcony/10.37.mp4
[2021-10-22 08:10:54] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Balcony-20211022081037.ts':
  Duration: 00:00:10.01, start: 1.400000, bitrate: 6548 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:55] frigate.record                 ERROR   : Unable to convert /tmp/cache/Laundry-20211022081036.ts to /media/frigate/recordings/2021-10/22/08/Laundry/10.36.mp4
[2021-10-22 08:10:55] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Laundry-20211022081036.ts':
  Duration: 00:00:10.03, start: 1.400000, bitrate: 6539 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:56] frigate.record                 ERROR   : Unable to convert /tmp/cache/Entrance-20211022081036.ts to /media/frigate/recordings/2021-10/22/08/Entrance/10.36.mp4
[2021-10-22 08:10:56] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Entrance-20211022081036.ts':
  Duration: 00:00:10.06, start: 1.400000, bitrate: 6529 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:10:57] frigate.record                 ERROR   : Unable to convert /tmp/cache/Deck-20211022081036.ts to /media/frigate/recordings/2021-10/22/08/Deck/10.36.mp4
[2021-10-22 08:10:57] frigate.record                 ERROR   : ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libfreetype --enable-libvidstab --enable-libmfx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpegts, from '/tmp/cache/Deck-20211022081036.ts':
  Duration: 00:00:10.02, start: 1.400000, bitrate: 6544 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100]: Video: h264 ([27][0][0][0] / 0x001B), yuv420p(progressive), 2560x1920, 30 fps, 30 tbr, 90k tbn, 180k tbc
    Stream #0:1[0x101]: Audio: aac ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp, 64 kb/s
[2021-10-22 08:12:23] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:51498]
[2021-10-22 08:14:33] ffmpeg.Driveway.record         ERROR   : [segment @ 0x5597d044e680] Failure occurred when ending segment '/tmp/cache/Driveway-20211022081418.ts'
[2021-10-22 08:14:33] ffmpeg.Driveway.record         ERROR   : av_interleaved_write_frame(): No space left on device
[2021-10-22 08:14:33] ffmpeg.Driveway.record         ERROR   : Error writing trailer of /tmp/cache/Driveway-%Y%m%d%H%M%S.ts: Invalid argument
[2021-10-22 08:14:33] watchdog.Driveway              INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:33] watchdog.Driveway              INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-22 08:14:33] ffmpeg.Rumpus.record           ERROR   : av_interleaved_write_frame(): No space left on device
[2021-10-22 08:14:33] ffmpeg.Entrance.record         ERROR   : [segment @ 0x563e22fe0b40] Failure occurred when ending segment '/tmp/cache/Entrance-20211022081416.ts'
[2021-10-22 08:14:33] ffmpeg.Entrance.record         ERROR   : av_interleaved_write_frame(): No space left on device
[2021-10-22 08:14:33] ffmpeg.Entrance.record         ERROR   : Error writing trailer of /tmp/cache/Entrance-%Y%m%d%H%M%S.ts: Invalid argument
[2021-10-22 08:14:33] watchdog.Entrance              INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:33] watchdog.Entrance              INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-22 08:14:33] ffmpeg.Rumpus.record           ERROR   : [segment @ 0x56115f228000] Failure occurred when ending segment '/tmp/cache/Rumpus-20211022081417.ts'
[2021-10-22 08:14:33] ffmpeg.Rumpus.record           ERROR   : Error writing trailer of /tmp/cache/Rumpus-%Y%m%d%H%M%S.ts: No space left on device
[2021-10-22 08:14:33] watchdog.Rumpus                INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:33] watchdog.Rumpus                INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-22 08:14:33] ffmpeg.Balcony.record          ERROR   : av_interleaved_write_frame(): No space left on device
[2021-10-22 08:14:33] ffmpeg.Balcony.record          ERROR   : [segment @ 0x556250668980] Failure occurred when ending segment '/tmp/cache/Balcony-20211022081417.ts'
[2021-10-22 08:14:33] ffmpeg.Balcony.record          ERROR   : Error writing trailer of /tmp/cache/Balcony-%Y%m%d%H%M%S.ts: No space left on device
[2021-10-22 08:14:33] watchdog.Balcony               INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:33] watchdog.Balcony               INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-22 08:14:33] ffmpeg.Deck.record             ERROR   : av_interleaved_write_frame(): No space left on device
[2021-10-22 08:14:33] ffmpeg.Deck.record             ERROR   : [segment @ 0x5607625cb5c0] Failure occurred when ending segment '/tmp/cache/Deck-20211022081425.ts'
[2021-10-22 08:14:33] ffmpeg.Deck.record             ERROR   : Error writing trailer of /tmp/cache/Deck-%Y%m%d%H%M%S.ts: No space left on device
[2021-10-22 08:14:33] watchdog.Deck                  INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:33] watchdog.Deck                  INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-22 08:14:33] ffmpeg.Laundry.record          ERROR   : [segment @ 0x56326c639b80] Failure occurred when ending segment '/tmp/cache/Laundry-20211022081416.ts'
[2021-10-22 08:14:33] ffmpeg.Laundry.record          ERROR   : av_interleaved_write_frame(): No space left on device
[2021-10-22 08:14:33] ffmpeg.Laundry.record          ERROR   : Error writing trailer of /tmp/cache/Laundry-%Y%m%d%H%M%S.ts: Invalid argument
[2021-10-22 08:14:33] watchdog.Laundry               INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:33] watchdog.Laundry               INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-22 08:14:43] watchdog.Driveway              INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:43] watchdog.Driveway              INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-22 08:14:43] watchdog.Rumpus                INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:43] watchdog.Rumpus                INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-22 08:14:43] watchdog.Entrance              INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:43] watchdog.Entrance              INFO    : Waiting for ffmpeg to exit gracefully...
[2021-10-22 08:14:43] watchdog.Balcony               INFO    : Terminating the existing ffmpeg process...
[2021-10-22 08:14:43] watchdog.Balcony               INFO    : Waiting for ffmpeg to exit gracefully...
...
config.yml
# config-http-0.9.2

database:
  path: /media/frigate/database/frigate.db

detectors:
  coral:
    type: edgetpu
    device: usb

mqtt:
  host: 192.168.1.5
  port: 1883
  user: mqtt
  password: "{FRIGATE_MQTT_PASSWORD}"

birdseye:
  mode: continuous
  width: 1792
  height: 1344

detect:
  width: 896
  height: 672
  fps: 5

ffmpeg:
  hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
  input_args: -avoid_negative_ts make_zero -fflags nobuffer+genpts+discardcorrupt -flags low_delay -strict experimental -analyzeduration 1000M -probesize 1000M -rw_timeout 5000000

live:
  height: 672
  quality: 8

motion:
  mask:
    - 265,650,265,625,630,625,630,650

record:
  retain_days: 0
  enabled: true
  events:
    pre_capture: 3
    retain:
      default: 15

snapshots:
  enabled: true
  clean_copy: false
  bounding_box: true
  retain:
    default: 15

timestamp_style:
  format: "%d/%m/%Y %H:%M:%S"

cameras:
  Driveway:
    ffmpeg:
      inputs:
        - path: http://192.168.1.71/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.71/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
        - vehicle
        - dog
      filters:
        person:
          mask: 
            - 510,120,540,120,540,160,510,160
    rtmp:
      enabled: false

  Entrance:
    ffmpeg:
      inputs:
        - path: http://192.168.1.72/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.72/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
        - dog
      filters:
        person:
          mask: 82,462,107,558,182,546,181,444
    rtmp:
      enabled: false

  Rumpus:
    ffmpeg:
      inputs:
        - path: http://192.168.1.73/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.73/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
        - dog
    rtmp:
      enabled: false

  Deck:
    ffmpeg:
      inputs:
        - path: http://192.168.1.74/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.74/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
    rtmp:
      enabled: false

  Laundry:
    ffmpeg:
      inputs:
        - path: http://192.168.1.75/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.75/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
    rtmp:
      enabled: false

  Balcony:
    ffmpeg:
      inputs:
        - path: http://192.168.1.76/flv?port=1935&app=bcs&stream=channel0_ext.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - detect
        - path: http://192.168.1.76/flv?port=1935&app=bcs&stream=channel0_main.bcs&user={FRIGATE_USER}&password={FRIGATE_PASSWORD}
          roles:
            - record
    objects:
      track:
        - person
        - vehicle
    rtmp:
      enabled: false

I've also seen this issue in 0.9.2 (using the same config): 'No VA display found for device /dev/dri/renderD128.,'. But radeontop shows the GPU is being used (and Frigate is only process that would use it).

logs
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.,
[s6-init] ensuring user provided files have correct perms...exited 0.,
[fix-attrs.d] applying ownership & permissions fixes...,
[fix-attrs.d] done.,
[cont-init.d] executing container initialization scripts...,
[cont-init.d] done.,
[services.d] starting services,
[services.d] done.,
[2021-10-22 09:32:51] frigate.app                    INFO    : Starting Frigate (0.9.2-25bb515),
Starting migrations,
[2021-10-22 09:32:51] peewee_migrate                 INFO    : Starting migrations,
There is nothing to migrate,
[2021-10-22 09:32:51] peewee_migrate                 INFO    : There is nothing to migrate,
[2021-10-22 09:32:51] frigate.mqtt                   INFO    : MQTT connected,
[2021-10-22 09:32:51] detector.coral                 INFO    : Starting detection process: 216,
[2021-10-22 09:32:51] frigate.app                    INFO    : Output process started: 218,
[2021-10-22 09:32:51] frigate.edgetpu                INFO    : Attempting to load TPU as usb,
[2021-10-22 09:32:51] frigate.app                    INFO    : Camera processor started for Driveway: 221,
[2021-10-22 09:32:51] frigate.app                    INFO    : Camera processor started for Entrance: 224,
[2021-10-22 09:32:51] frigate.app                    INFO    : Camera processor started for Rumpus: 227,
[2021-10-22 09:32:51] frigate.app                    INFO    : Camera processor started for Deck: 229,
[2021-10-22 09:32:51] frigate.app                    INFO    : Camera processor started for Laundry: 230,
[2021-10-22 09:32:51] frigate.app                    INFO    : Camera processor started for Balcony: 232,
[2021-10-22 09:32:51] frigate.app                    INFO    : Capture process started for Driveway: 233,
[2021-10-22 09:32:51] frigate.app                    INFO    : Capture process started for Entrance: 235,
[2021-10-22 09:32:51] frigate.app                    INFO    : Capture process started for Rumpus: 238,
[2021-10-22 09:32:51] frigate.app                    INFO    : Capture process started for Deck: 243,
[2021-10-22 09:32:51] frigate.app                    INFO    : Capture process started for Laundry: 246,
[2021-10-22 09:32:51] frigate.app                    INFO    : Capture process started for Balcony: 255,
[2021-10-22 09:32:51] ws4py                          INFO    : Using epoll,
[2021-10-22 09:32:51] ws4py                          INFO    : Using epoll,
[2021-10-22 09:32:54] frigate.edgetpu                INFO    : TPU found,
[2021-10-22 09:33:05] frigate.video                  INFO    : Entrance: ffmpeg sent a broken frame. memoryview assignment: lvalue and rvalue have different structures,
[2021-10-22 09:33:05] frigate.video                  INFO    : Entrance: ffmpeg process is not running. exiting capture thread...,
[2021-10-22 09:33:11] watchdog.Entrance              ERROR   : FFMPEG process crashed unexpectedly for Entrance.,
[2021-10-22 09:33:11] watchdog.Entrance              ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.,
[2021-10-22 09:33:11] watchdog.Entrance              ERROR   : You may have invalid args defined for this camera.,
[2021-10-22 09:33:11] ffmpeg.Entrance.detect         ERROR   : [AVHWDeviceContext @ 0x5612f9962c80] No VA display found for device /dev/dri/renderD128.,
[2021-10-22 09:33:11] ffmpeg.Entrance.detect         ERROR   : Device creation failed: -22.,
[2021-10-22 09:33:11] ffmpeg.Entrance.detect         ERROR   : [h264 @ 0x5612f9758c80] No device available for decoder: device type vaapi needed for codec h264.,
[2021-10-22 09:33:11] ffmpeg.Entrance.detect         ERROR   : Device setup failed for decoder on input stream #0:0 : Invalid argument,
[2021-10-22 09:33:13] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:52972],
[2021-10-22 09:33:15] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:52972],
[2021-10-22 09:33:15] ws4py                          INFO    : Managing websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:53080],
[2021-10-22 09:33:21] ws4py                          INFO    : Terminating websocket [Local => 127.0.0.1:5002 | Remote => 127.0.0.1:53080]

PS as per my comment here, the detect resolution in the recommended settings are not correct/native for the Reolink path specified, plus I have issues if I use the quotation marks for the rw_timeout value as a global setting.

@blakeblackshear
Copy link
Owner

I have now been running with the same arguments you posted for 12 hours without any issues with my Reolink. It's strange that the files leftover are not mentioned in the logs. Is /tmp/cache empty on startup? I can't imagine why it wouldn't be, but worth checking. I always have at most 2 files per camera in my cache directory. In the event a cache file fails to convert, it should be deleted anyway.

@jasonpstokes
Copy link
Contributor Author

Yes /tmp/cache is empty on startup.
Under 0.9.1 I can see one or two files per camera, using around 70Mb total.

@blakeblackshear
Copy link
Owner

Not sure why it would be slow, but if the copy operation from /tmp/cache to /media/frigate/recordings is slower than the camera writes to that directory, it would explain it. There must be something about your video that makes ffmpeg process them slow. If that is indeed what is happening, you should be able to copy it out and run the same ffmpeg command to see more about what it's doing.

@jasonpstokes
Copy link
Contributor Author

_frigate_logs.txt

root@frigate:/tmp/cache# df -h
Filesystem                     Size  Used Avail Use% Mounted on
overlay                        233G   17G  204G   8% /
tmpfs                           64M     0   64M   0% /dev
shm                            300M  7.8M  293M   3% /dev/shm
/dev/sda2                      233G   17G  204G   8% /labelmap.txt
192.168.30.8:/storage/frigate  1.8T  237G  1.6T  14% /media/frigate
tmpfs                          1.0G  753M  272M  74% /tmp/cache
tmpfs                          7.7G     0  7.7G   0% /proc/asound
tmpfs                          7.7G     0  7.7G   0% /proc/acpi
tmpfs                          7.7G     0  7.7G   0% /sys/firmware
root@frigate:/tmp/cache# ls
Balcony-20211022130835.ts  Balcony-20211022131225.ts  Deck-20211022131204.ts      Driveway-20211022131137.ts  Entrance-20211022131115.ts  Laundry-20211022131055.ts  Rumpus-20211022131025.ts
Balcony-20211022130845.ts  Balcony-20211022131235.ts  Deck-20211022131214.ts      Driveway-20211022131145.ts  Entrance-20211022131125.ts  Laundry-20211022131105.ts  Rumpus-20211022131035.ts
Balcony-20211022130855.ts  Deck-20211022130834.ts     Deck-20211022131224.ts      Driveway-20211022131157.ts  Entrance-20211022131135.ts  Laundry-20211022131115.ts  Rumpus-20211022131045.ts
Balcony-20211022131035.ts  Deck-20211022130844.ts     Deck-20211022131234.ts      Driveway-20211022131205.ts  Entrance-20211022131145.ts  Laundry-20211022131125.ts  Rumpus-20211022131055.ts
Balcony-20211022131045.ts  Deck-20211022130854.ts     Driveway-20211022130825.ts  Driveway-20211022131217.ts  Entrance-20211022131155.ts  Laundry-20211022131135.ts  Rumpus-20211022131105.ts
Balcony-20211022131055.ts  Deck-20211022131034.ts     Driveway-20211022130837.ts  Driveway-20211022131225.ts  Entrance-20211022131205.ts  Laundry-20211022131145.ts  Rumpus-20211022131115.ts
Balcony-20211022131105.ts  Deck-20211022131044.ts     Driveway-20211022130845.ts  Driveway-20211022131237.ts  Entrance-20211022131215.ts  Laundry-20211022131155.ts  Rumpus-20211022131125.ts
Balcony-20211022131115.ts  Deck-20211022131054.ts     Driveway-20211022131025.ts  Entrance-20211022130835.ts  Entrance-20211022131225.ts  Laundry-20211022131205.ts  Rumpus-20211022131135.ts
Balcony-20211022131125.ts  Deck-20211022131104.ts     Driveway-20211022131037.ts  Entrance-20211022130845.ts  Entrance-20211022131235.ts  Laundry-20211022131215.ts  Rumpus-20211022131145.ts
Balcony-20211022131135.ts  Deck-20211022131114.ts     Driveway-20211022131045.ts  Entrance-20211022130855.ts  Laundry-20211022130835.ts   Laundry-20211022131225.ts  Rumpus-20211022131155.ts
Balcony-20211022131145.ts  Deck-20211022131124.ts     Driveway-20211022131057.ts  Entrance-20211022131035.ts  Laundry-20211022130845.ts   Laundry-20211022131235.ts  Rumpus-20211022131205.ts
Balcony-20211022131155.ts  Deck-20211022131134.ts     Driveway-20211022131105.ts  Entrance-20211022131045.ts  Laundry-20211022130855.ts   Rumpus-20211022130835.ts   Rumpus-20211022131215.ts
Balcony-20211022131205.ts  Deck-20211022131144.ts     Driveway-20211022131117.ts  Entrance-20211022131055.ts  Laundry-20211022131035.ts   Rumpus-20211022130845.ts   Rumpus-20211022131225.ts
Balcony-20211022131215.ts  Deck-20211022131154.ts     Driveway-20211022131125.ts  Entrance-20211022131105.ts  Laundry-20211022131045.ts   Rumpus-20211022130855.ts   Rumpus-20211022131235.ts

@blakeblackshear
Copy link
Owner

It looks like the ffmpeg command to copy/convert the .ts file to .mp4 is taking long enough that it can't keep up. It's a straight copy command so there shouldn't be any conversion happening, but it must be enough overhead that it makes a difference compared the previous copy/delete operation. I assume the .ts file you posted earlier would be the same as these?

@jasonpstokes
Copy link
Contributor Author

I'd assume so, but here's some more

https://drive.google.com/file/d/1oq-oT7Z0nmB8hU93bOXENRjUE5o0Xsfw/view?usp=sharing

(please let me know when you have so I can delete)

@blakeblackshear
Copy link
Owner

Got the zip.

See if this build fixes it: blakeblackshear/frigate:cache-debug-4bf61d6-amd64

Based on your compose, it looks like you are using a network drive at /media/frigate. Is that right?

@jasonpstokes
Copy link
Contributor Author

Thanks, running now.

Yes, an nfs4 mounted NAS in Debian 11.
mount -t nfs4 192.168.30.8:/storage /mnt/nas

@jasonpstokes
Copy link
Contributor Author

It's made a big difference, the files were (very) slowly increasing in number, up to about six or seven for each camera - and then suddenly a heap disappeared!

_frigate_logs (2).txt

@blakeblackshear
Copy link
Owner

Let's see if it keeps up over time. I have an idea for a way I can tweak things if this works.

@jasonpstokes
Copy link
Contributor Author

_frigate_logs (3).txt

It crashed :-(

root@frigate:/opt/frigate# df -h
Filesystem                     Size  Used Avail Use% Mounted on
overlay                        233G   18G  203G   8% /
tmpfs                           64M     0   64M   0% /dev
shm                            300M   19M  282M   7% /dev/shm
/dev/sda2                      233G   18G  203G   8% /labelmap.txt
192.168.30.8:/storage/frigate  1.8T  237G  1.6T  14% /media/frigate
tmpfs                          1.0G  716M  309M  70% /tmp/cache
tmpfs                          7.7G     0  7.7G   0% /proc/asound
tmpfs                          7.7G     0  7.7G   0% /proc/acpi
tmpfs                          7.7G     0  7.7G   0% /sys/firmware
root@frigate:/opt/frigate# ls -lh /tmp/cache
total 716M
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Balcony-20211022134513.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Balcony-20211022134523.ts
-rw-r--r-- 1 root root 7.8M Oct 22 13:45 Balcony-20211022134533.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Balcony-20211022134543.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Balcony-20211022134553.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Balcony-20211022134603.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Balcony-20211022134613.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Balcony-20211022134623.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Balcony-20211022134633.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Balcony-20211022134643.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Balcony-20211022134653.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Balcony-20211022134703.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Balcony-20211022134713.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Balcony-20211022134723.ts
-rw-r--r-- 1 root root 7.8M Oct 22 13:47 Balcony-20211022134733.ts
-rw-r--r-- 1 root root 7.0M Oct 22 13:47 Balcony-20211022134743.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Deck-20211022134514.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Deck-20211022134534.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Deck-20211022134544.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Deck-20211022134554.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Deck-20211022134604.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Deck-20211022134614.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Deck-20211022134624.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Deck-20211022134634.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Deck-20211022134644.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Deck-20211022134654.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Deck-20211022134704.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Deck-20211022134714.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Deck-20211022134724.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Deck-20211022134733.ts
-rw-r--r-- 1 root root 5.8M Oct 22 13:47 Deck-20211022134743.ts
-rw-r--r-- 1 root root 9.4M Oct 22 13:45 Driveway-20211022134504.ts
-rw-r--r-- 1 root root 6.3M Oct 22 13:45 Driveway-20211022134517.ts
-rw-r--r-- 1 root root 6.3M Oct 22 13:45 Driveway-20211022134536.ts
-rw-r--r-- 1 root root 9.4M Oct 22 13:45 Driveway-20211022134544.ts
-rw-r--r-- 1 root root 6.3M Oct 22 13:46 Driveway-20211022134556.ts
-rw-r--r-- 1 root root 9.4M Oct 22 13:46 Driveway-20211022134604.ts
-rw-r--r-- 1 root root 6.3M Oct 22 13:46 Driveway-20211022134616.ts
-rw-r--r-- 1 root root 9.4M Oct 22 13:46 Driveway-20211022134624.ts
-rw-r--r-- 1 root root 6.3M Oct 22 13:46 Driveway-20211022134636.ts
-rw-r--r-- 1 root root 9.4M Oct 22 13:46 Driveway-20211022134644.ts
-rw-r--r-- 1 root root 6.3M Oct 22 13:47 Driveway-20211022134656.ts
-rw-r--r-- 1 root root 9.4M Oct 22 13:47 Driveway-20211022134704.ts
-rw-r--r-- 1 root root 6.3M Oct 22 13:47 Driveway-20211022134716.ts
-rw-r--r-- 1 root root 9.4M Oct 22 13:47 Driveway-20211022134724.ts
-rw-r--r-- 1 root root 6.3M Oct 22 13:47 Driveway-20211022134736.ts
-rw-r--r-- 1 root root 5.0M Oct 22 13:47 Driveway-20211022134744.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Entrance-20211022134514.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Entrance-20211022134534.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Entrance-20211022134544.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Entrance-20211022134554.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Entrance-20211022134604.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Entrance-20211022134614.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Entrance-20211022134624.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Entrance-20211022134634.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Entrance-20211022134644.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Entrance-20211022134654.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Entrance-20211022134704.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Entrance-20211022134714.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Entrance-20211022134724.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Entrance-20211022134734.ts
-rw-r--r-- 1 root root 5.5M Oct 22 13:47 Entrance-20211022134744.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Laundry-20211022134512.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Laundry-20211022134522.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Laundry-20211022134532.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Laundry-20211022134542.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Laundry-20211022134552.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Laundry-20211022134602.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Laundry-20211022134612.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Laundry-20211022134622.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Laundry-20211022134632.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Laundry-20211022134642.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Laundry-20211022134652.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Laundry-20211022134702.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Laundry-20211022134712.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Laundry-20211022134722.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Laundry-20211022134732.ts
-rw-r--r-- 1 root root 7.0M Oct 22 13:47 Laundry-20211022134742.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Rumpus-20211022134513.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Rumpus-20211022134533.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:45 Rumpus-20211022134543.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Rumpus-20211022134553.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Rumpus-20211022134603.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Rumpus-20211022134613.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Rumpus-20211022134623.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Rumpus-20211022134633.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:46 Rumpus-20211022134643.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Rumpus-20211022134653.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Rumpus-20211022134703.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Rumpus-20211022134713.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Rumpus-20211022134723.ts
-rw-r--r-- 1 root root 7.9M Oct 22 13:47 Rumpus-20211022134733.ts
-rw-r--r-- 1 root root 6.3M Oct 22 13:47 Rumpus-20211022134743.ts

@d3structivepyr0
Copy link

I was also seeing this, or something similar in 0.9.1. The cache was filling up with files, many of size 0, and 0 space was left in cache. It was taking closer to a day or two between restarts before this would happen, so I hadn't tried really looking into it until tonight. I had changed the ffmpeg segment time option from 10 to 600 seconds when I updated from 0.8.4 to 0.9.1, and after seeing the cache fill up and crash the first time, I increased my cache size to 3gb.

I pulled the dev build to see if I could add anything beyond what Jason is already providing, and reset my cache back to 1.5gb. With the segment time still at 600, I saw the issue once. Lots of unmoved files and a full cache. I changed my segment time back to 10 seconds and it's running now. I normally don't comment, but since the faststart flag wasn't added until 0.9.2, I thought it might be important to mention that I was seeing this in 0.9.1.

@blakeblackshear
Copy link
Owner

In 0.9.1, all that happened was a simple copy operation from the cache.

@jasonpstokes
Copy link
Contributor Author

FYI that dev build has been running for two hours and it's looking good! Will report back tomorrow morning.

root@frigate:/opt/frigate# df -h
Filesystem                     Size  Used Avail Use% Mounted on
overlay                        233G   18G  203G   8% /
tmpfs                           64M     0   64M   0% /dev
shm                            300M  7.8M  293M   3% /dev/shm
/dev/sda2                      233G   18G  203G   8% /labelmap.txt
192.168.30.8:/storage/frigate  1.8T  241G  1.6T  14% /media/frigate
tmpfs                          1.0G   40M  985M   4% /tmp/cache
tmpfs                          7.7G     0  7.7G   0% /proc/asound
tmpfs                          7.7G     0  7.7G   0% /proc/acpi
tmpfs                          7.7G     0  7.7G   0% /sys/firmware

root@frigate:/opt/frigate# ls -lh /tmp/cache
total 29M
-rw-r--r-- 1 root root 3.2M Oct 22 22:37 Balcony-20211022223741.ts
-rw-r--r-- 1 root root 1.0M Oct 22 22:37 Balcony-20211022223750.ts
-rw-r--r-- 1 root root 3.2M Oct 22 22:37 Deck-20211022223740.ts
-rw-r--r-- 1 root root 1.3M Oct 22 22:37 Deck-20211022223750.ts
-rw-r--r-- 1 root root 6.3M Oct 22 22:37 Driveway-20211022223741.ts
-rw-r--r-- 1 root root 2.3M Oct 22 22:37 Driveway-20211022223750.ts
-rw-r--r-- 1 root root 3.2M Oct 22 22:37 Entrance-20211022223741.ts
-rw-r--r-- 1 root root 1.0M Oct 22 22:37 Entrance-20211022223751.ts
-rw-r--r-- 1 root root 3.2M Oct 22 22:37 Laundry-20211022223742.ts
-rw-r--r-- 1 root root 768K Oct 22 22:37 Laundry-20211022223751.ts
-rw-r--r-- 1 root root 3.2M Oct 22 22:37 Rumpus-20211022223742.ts
-rw-r--r-- 1 root root 512K Oct 22 22:37 Rumpus-20211022223752.ts

@blakeblackshear
Copy link
Owner

I have another dev build for testing blakeblackshear/frigate:cache-debug-0437066-amd64

It has the following changes:

  • Run the ffmpeg conversion to mp4 in the cache directly and add back faststart. Then copy the mp4 file out of the cache the same way it was done in 0.9.1
  • Tighten up the loop for cache cleanup. Previously, it would wait 5 seconds between checks, and now the wait will dynamically decrease as the time it takes to move the cache increases. This should help it keep up.

@rhatguy
Copy link

rhatguy commented Oct 22, 2021

Just reporting back that the issues I'm experiencing seem the same as the ones Jason is, but just wanted to add that I'm writing from /tmp/cache to a local 2TB SSD, so I don't think its a bandwidth problem moving the files out of /tmp/cache. I'm remote this weekend from somewhere with very little internet so I probably won't be able to help debug much:( Thanks all for sticking with this one!

@bdruth
Copy link

bdruth commented Oct 22, 2021

I'll throw my hat in, too - finally sat down to figure out why docker was taking up over 50GB after a few days, locking up my root partition even though all my volumes were elsewhere. Turns out it was this /tmp/cache bug in the frigate container. Just pulled the dev build - will keep an eye on it over the weekend.

@tonyhardcode
Copy link

Should we avoid 0.9.2 and wait for the next version?

@jasonpstokes
Copy link
Contributor Author

tmpfs filled up overnight. Pulling the latest dev build now.

Thanks @rhatguy for confirming it's not just my set up or NAS.
Is this just Reolink streams - or are people with other cameras seeing this too?

@jasonpstokes
Copy link
Contributor Author

The latest build didn't last long, sorry.

_frigate_logs.txt

root@frigate:/opt/frigate# df -h
Filesystem                     Size  Used Avail Use% Mounted on
overlay                        233G   19G  202G   9% /
tmpfs                           64M     0   64M   0% /dev
shm                            300M  7.8M  293M   3% /dev/shm
/dev/sda2                      233G   19G  202G   9% /labelmap.txt
192.168.30.8:/storage/frigate  1.8T  242G  1.6T  14% /media/frigate
tmpfs                          1.0G 1002M   23M  98% /tmp/cache
tmpfs                          7.7G     0  7.7G   0% /proc/asound
tmpfs                          7.7G     0  7.7G   0% /proc/acpi
tmpfs                          7.7G     0  7.7G   0% /sys/firmware

root@frigate:/opt/frigate# ls -lh /tmp/cache
total 1015M
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Balcony-20211023095459.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Balcony-20211023095509.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Balcony-20211023095519.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Balcony-20211023095529.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Balcony-20211023095539.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Balcony-20211023095549.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:56 Balcony-20211023095559.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Balcony-20211023095831.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Balcony-20211023095841.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Balcony-20211023095851.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Balcony-20211023095901.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Balcony-20211023095911.ts
-rw-r--r-- 1 root root 7.8M Oct 23 09:59 Balcony-20211023095921.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Balcony-20211023095931.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Balcony-20211023095941.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Balcony-20211023095951.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Balcony-20211023100001.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Balcony-20211023100011.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Balcony-20211023100021.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Balcony-20211023100031.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Balcony-20211023100041.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:01 Balcony-20211023100051.ts
-rw-r--r-- 1 root root 2.8M Oct 23 10:01 Balcony-20211023100101.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Deck-20211023095459.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Deck-20211023095508.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Deck-20211023095518.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Deck-20211023095528.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Deck-20211023095538.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Deck-20211023095548.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:56 Deck-20211023095558.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Deck-20211023095828.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Deck-20211023095838.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Deck-20211023095848.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Deck-20211023095858.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Deck-20211023095908.ts
-rw-r--r-- 1 root root 4.7M Oct 23 09:59 Deck-20211023095918.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Deck-20211023095930.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Deck-20211023095940.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Deck-20211023095950.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Deck-20211023100000.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Deck-20211023100010.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Deck-20211023100020.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Deck-20211023100030.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Deck-20211023100040.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:01 Deck-20211023100050.ts
-rw-r--r-- 1 root root 2.8M Oct 23 10:01 Deck-20211023100100.ts
-rw-r--r-- 1 root root 9.5M Oct 23 09:55 Driveway-20211023095459.ts
-rw-r--r-- 1 root root 6.3M Oct 23 09:55 Driveway-20211023095511.ts
-rw-r--r-- 1 root root 9.4M Oct 23 09:55 Driveway-20211023095519.ts
-rw-r--r-- 1 root root 6.3M Oct 23 09:55 Driveway-20211023095531.ts
-rw-r--r-- 1 root root 9.4M Oct 23 09:55 Driveway-20211023095539.ts
-rw-r--r-- 1 root root 6.3M Oct 23 09:55 Driveway-20211023095551.ts
-rw-r--r-- 1 root root 9.4M Oct 23 09:56 Driveway-20211023095559.ts
-rw-r--r-- 1 root root 6.3M Oct 23 09:58 Driveway-20211023095831.ts
-rw-r--r-- 1 root root 9.4M Oct 23 09:58 Driveway-20211023095839.ts
-rw-r--r-- 1 root root 6.3M Oct 23 09:58 Driveway-20211023095851.ts
-rw-r--r-- 1 root root 9.4M Oct 23 09:59 Driveway-20211023095859.ts
-rw-r--r-- 1 root root 6.2M Oct 23 09:59 Driveway-20211023095911.ts
-rw-r--r-- 1 root root 9.4M Oct 23 09:59 Driveway-20211023095919.ts
-rw-r--r-- 1 root root 6.3M Oct 23 09:59 Driveway-20211023095931.ts
-rw-r--r-- 1 root root 9.4M Oct 23 09:59 Driveway-20211023095939.ts
-rw-r--r-- 1 root root 6.3M Oct 23 09:59 Driveway-20211023095951.ts
-rw-r--r-- 1 root root 9.4M Oct 23 10:00 Driveway-20211023095959.ts
-rw-r--r-- 1 root root 6.3M Oct 23 10:00 Driveway-20211023100011.ts
-rw-r--r-- 1 root root 9.4M Oct 23 10:00 Driveway-20211023100019.ts
-rw-r--r-- 1 root root 6.3M Oct 23 10:00 Driveway-20211023100031.ts
-rw-r--r-- 1 root root 9.4M Oct 23 10:00 Driveway-20211023100039.ts
-rw-r--r-- 1 root root 6.3M Oct 23 10:00 Driveway-20211023100051.ts
-rw-r--r-- 1 root root 4.8M Oct 23 10:01 Driveway-20211023100059.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Entrance-20211023095458.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Entrance-20211023095508.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Entrance-20211023095518.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Entrance-20211023095528.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Entrance-20211023095538.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Entrance-20211023095548.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:56 Entrance-20211023095558.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Entrance-20211023095828.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Entrance-20211023095838.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Entrance-20211023095848.ts
-rw-r--r-- 1 root root 1.0M Oct 23 09:58 Entrance-20211023095858.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Entrance-20211023095909.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Entrance-20211023095920.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Entrance-20211023095930.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Entrance-20211023095940.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Entrance-20211023095950.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Entrance-20211023100000.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Entrance-20211023100010.ts
-rw-r--r-- 1 root root 2.7M Oct 23 10:00 Entrance-20211023100020.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Entrance-20211023100029.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Entrance-20211023100040.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:01 Entrance-20211023100050.ts
-rw-r--r-- 1 root root 2.8M Oct 23 10:01 Entrance-20211023100100.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Laundry-20211023095459.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Laundry-20211023095509.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Laundry-20211023095519.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Laundry-20211023095529.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Laundry-20211023095539.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Laundry-20211023095549.ts
-rw-r--r-- 1 root root 7.6M Oct 23 10:01 Laundry-20211023095559.mp4
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Laundry-20211023095829.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Laundry-20211023095839.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Laundry-20211023095849.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Laundry-20211023095859.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Laundry-20211023095909.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Laundry-20211023095919.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Laundry-20211023095929.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Laundry-20211023095939.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Laundry-20211023095949.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Laundry-20211023095959.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Laundry-20211023100009.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Laundry-20211023100019.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Laundry-20211023100029.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Laundry-20211023100039.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Laundry-20211023100049.ts
-rw-r--r-- 1 root root 4.3M Oct 23 10:01 Laundry-20211023100059.ts
-rw-r--r-- 1 root root 7.0M Oct 23 09:55 Rumpus-20211023095459.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Rumpus-20211023095520.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Rumpus-20211023095531.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:55 Rumpus-20211023095541.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:56 Rumpus-20211023095551.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Rumpus-20211023095831.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:58 Rumpus-20211023095841.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Rumpus-20211023095851.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Rumpus-20211023095901.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Rumpus-20211023095911.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Rumpus-20211023095921.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Rumpus-20211023095931.ts
-rw-r--r-- 1 root root 7.9M Oct 23 09:59 Rumpus-20211023095941.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Rumpus-20211023095951.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Rumpus-20211023100001.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Rumpus-20211023100011.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Rumpus-20211023100021.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Rumpus-20211023100031.ts
-rw-r--r-- 1 root root 7.9M Oct 23 10:00 Rumpus-20211023100041.ts
-rw-r--r-- 1 root root 2.7M Oct 23 10:00 Rumpus-20211023100051.ts
-rw-r--r-- 1 root root 2.5M Oct 23 10:01 Rumpus-20211023100100.ts

@blakeblackshear
Copy link
Owner

Give blakeblackshear/frigate:0.9.3-57645c6-amd64 a try. This reverts all the cache handling changes introduced in 0.9.2, but doesn't have all the verbose logging from before.

@jasonpstokes
Copy link
Contributor Author

jasonpstokes commented Oct 23, 2021

Thanks, deployed and tmpfs usage is tiny. Will let you know.

Jumped back to 0.8.4 this morning to try the same FFmpeg options and paths in that, and tmpfs filled up straight away. But I used to run that fine! So what changed between 0.8.4 (tmpfs issue) -> 0.9.1 (works) -> 0.9.2 (tmpfs issue), and/or could this be caused by the (now recommended) ffmpeg arguments for the Reolink cameras?

@blakeblackshear
Copy link
Owner

O no! Are the segment files larger? I wonder if this is a performance issue for copying files to your NAS.

@jasonpstokes
Copy link
Contributor Author

jasonpstokes commented Oct 23, 2021

But @rhatguy uses an SSD and is having the same issue?

In 0.9.1 and 0.8.4 (this morning) I've used segment_time 30. In 0.9.2 I'm using the default - i.e. no output_args specified in my config, with audio enabled by default now.

In case it was the audio track and/or segment size, I just ran 0.8.4 again using the default output_args, but it still filled up tmpfs within minutes.

@blakeblackshear
Copy link
Owner

I'm not convinced it's the same issue.

The clips role in 0.8.4 ran the same command as record on 0.9.1 to create segments in the cache. I don't really recommend changing the default segment time from 10s. I don't see a good reason to with 0.9.x, and increasing it has downsides that I don't think people realize.

@d3structivepyr0
Copy link

With blakeblackshear/frigate:cache-debug-0437066-amd64 , I filled the cache (set to 1GB) with the segment size set to 60 seconds. Since you recommend against changing it, i'll leave it at 10 seconds going forward. And since you have a newer build not sure how useful this log file will be, but i uploaded it just in case. I'll pull your latest build and see if it happens again. Also possibly worth mentioning, I do have one Reolink camera (RLC-420). I'm now using the latest recommended settings from the docs for Reolink cameras.

frigate.log

@d3structivepyr0
Copy link

d3structivepyr0 commented Oct 23, 2021

Latest build failed again. Log attached. Looks like it isn't removing files at all now "Expiring 0 recordings". It filled up really quickly. I was watching the cache from inside the container and saw the file count just getting bigger and bigger.

frigate2.log

@blakeblackshear
Copy link
Owner

Latest build is blakeblackshear/frigate:0.9.3-a67b8ab-amd64. @d3structivepyr0 your issue looks like you ran out of space in the recordings folder and that caused the thread that cleans up the cache to fail.

@d3structivepyr0
Copy link

Ahh, correct. That makes sense why it was suddenly failing much faster than before. Cleared some space on the recordings disk and pulled the new build. Seems to be cleaning the cache for now.

@jasonpstokes
Copy link
Contributor Author

Latest dev build is good, after running ~20 hours tmpfs has 1 or 2 files per cam / ~50mb used.

It's also now run for nearly 30 mins with audio enabled. 👍 (Using the rest of the default output_args, incl. segment time!)

@blakeblackshear blakeblackshear unpinned this issue Oct 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants