GoPro MP4 videos are only analyzed for the first 18 frames

[GabrielSandor]

I ran dvr-scan with the default settings on some of my GoPro MP4 video files exported to my PC, and for some reason the analysis stops after 18 frames, even if the videos are more than 10 minutes long at 30 FPS and they do have some motion in them.

Command:
dvr-scan -i 2.mp4 --verbosity debug

Output:
INFO: controller._init_dvr_scan(): DVR-Scan v1.5.1
DEBUG: controller._init_dvr_scan(): User config file not found.
DEBUG: controller._init_dvr_scan(): Parsing program options.
INFO: scanner.init(): Initializing scan context...
INFO: video_joiner._load_input_videos(): Opened video 2.mp4 (1280 x 720 at 29.970 FPS).
DEBUG: scanner.scan_motion(): Using detector MOG2 with params: kernel_size = 5
INFO: scanner.scan_motion(): Scanning input video for motion events...
Detected: 0 | Progress: 0%|▏ | 18/18720 [00:00<03:04, 101.13 frames/s]
INFO: scanner._post_scan_motion(): Processed 18 frames read in 0.2 secs (avg 99.4 FPS).
INFO: scanner._post_scan_motion(): No motion events detected in input.

I would expect all frames in the videos to be analyzed.

Computing Environment:

• OS: Windows 10
• dvr-scan version: 1.5.1, installed from https://dvr-scan.readthedocs.io/en/latest/download/

[GabrielSandor]

Video info from VLC Media Player, in case it helps:

Codec: H264 - MPEG-4 AVC (part 10) (avc1)
Video resolution: 1280x720
Buffer dimensions: 1280x720
Frame rate: 29.970030
Decoded format:
Orientation: Top left
Color primaries: ITU-R BT.709
Color transfer function: ITU-R BT.709
Color space: ITU-R BT.709 Range
Chroma location: Left

[Breakthrough]

It might be an issue with the container - can you try re-muxing the videos with ffmpeg or mkvmerge and trying again? In the long term I want to add other backends like PyAV for video input, but this is still in progress. In the meantime the only way video input is supported is using OpenCV which sometimes leads to issues like this.

You can re-mux the video with:

ffmpeg -i input.mp4 -c:v copy out.mp4

Failing that, you may need to re-encode them using ffmpeg before processing:

ffmpeg -i input.mp4 -c:v libx264 -preset fast -crf 21 out.mp4

This one is strange since typically I would expect to see a warning:

https://github.com/Breakthrough/DVR-Scan/blob/master/dvr_scan/scanner.py#L750

[GabrielSandor]

@Breakthrough i just tried your advice and indeed it works, thank you! After re-muxing the video with ffmpeg, dvr-scan is able to progress through all the frames in the new video file.

I will use this as a workaround for now. Would you like me to keep this issue open until you attempt a fix in dvr-scan?

[abdullahfarwees]

@GabrielSandor to keep this issue open until you attempt a fix in dvr-scan, it would be great if you attach the sample GOpro files here. Developers might take that as a reference while fixing the issue in future

[GabrielSandor]

Hello @abdullahfarwees. Due to privacy concerns, i'd rather not upload the particular GoPro files that cause the issue in dvr-scan.

[Breakthrough]

@GabrielSandor I don't think there's an easy workaround for this issue in DVR-Scan aside from using a different backend for video input as outlined above. That would be a fairly significant undertaking, so any additional detail you can provide would be very helpful.

If you are unable to share the video, would you be willing to run the same video through a related program I maintain that has added support for this? The project is PySceneDetect (www.scenedetect.com) and can be downloaded/installed in the same way as DVR-Scan.

Specifically, if you could run the following commands to include debug logs and provide them, that would let me know if using PyAV would work to fix the issue.

First, using OpenCV, the output should also stop after 18 frames as in your original example:

scenedetect -i video.mp4 --verbosity debug detect-content

Next using PyAV for video decoding:

scenedetect -i video.mp4 --verbosity debug --backend pyav detect-content

Thanks!

[GabrielSandor]

Sorry @Breakthrough, it's been a while since I recorded the video and I'm afraid I can no longer find it, and in the meantime my GoPro has broken. I plan to buy a new one at some point, and if the problem re-appears I'll get back to you.

[Breakthrough]

mmokrejs was able to provide some useful context in #160 recently. I added some thoughts there on how to possibly address this. It should be possible to see if ffmpeg outputs any errors to stdout/stderr when decoding a video, so ideally DVR-Scan could warn the user after the final list of events.