Framerate autodetection wildly inaccurate with certain content

[joeyparrish]

I received a video from a colleague (drone.mp4) that breaks our framerate detection. We decide that it's 600 fps instead of 51.98, which leads to a bad keyframe interval for ffmpeg, which leads to failed segmentation.

Output from ffprobe:

$ ffprobe drone.mp4 
ffprobe version 4.3.2-0+deb11u2 Copyright (c) 2007-2021 the FFmpeg developers
...

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'drone.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 0
    compatible_brands: qt  
    creation_time   : 2019-07-23T18:34:13.000000Z
  Duration: 00:00:27.24, start: 0.000000, bitrate: 17878 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, smpte170m/bt709/bt709), 1920x1080, 17796 kb/s, 51.98 fps, 600 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      creation_time   : 2019-07-23T18:34:13.000000Z
      handler_name    : Core Media Video
      encoder         : H.264
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 61 kb/s (default)
    Metadata:
      creation_time   : 2019-07-23T18:34:13.000000Z
      handler_name    : Core Media Audio

[joeyparrish]

As a workaround, the frame_rate field can be used on the video section of the input config for such content.

[mariocynicys]

$ ffprobe zdrone.mp4 -select_streams v:0 -show_entries stream=avg_frame_rate -of compact=p=0:nk=1 2> /dev/null
169920/3269

This was using avg_frame_rate instead of r_frame_rate for the probing. This sof discusses the difference which i didn't quiet understand.

>>> 169920/3269
51.979198531661055

Update: I understand the difference now. The r_frame_rate will be different to fit some kind timestamp calculation when the video has mixed frame rates. See https://ffmpeg.org/faq.html#AVStream_002er_005fframe_005frate-is-wrong_002c-it-is-much-larger-than-the-frame-rate_002e
I think we should use avg_frame_rate then.

[joeyparrish]

Thank you for figuring this out! I'd be happy to review a PR.