Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect start times with variable frame rate (VFR) videos #20

Open
josvanr opened this issue Dec 10, 2019 · 4 comments
Open

Incorrect start times with variable frame rate (VFR) videos #20

josvanr opened this issue Dec 10, 2019 · 4 comments

Comments

@josvanr
Copy link

josvanr commented Dec 10, 2019

Hello

I'm using dvr-scan to find motion start times in webcam movies recorded in guvcview. When I looked up the start times in the movie with mplayer, I noticed the actual events and start times don't match up. I did some testing using different movie formats and cam settings. Sometimes, in low light conditions, the camera's frame rate drops. In these movies, dvr-scan seems to get the start times wrong. I also managed to make a couple of movies with constant frame rate (lot of more or less constant light) here the start times are correct.

Is there some work around for this? In my application I can't be sure the light is always constant. Also the frame rate variability seems to happen irrespective of cam settings (absolute exposure, raw formats ...) What I also tried is re-code the movie with ffmpeg using -r 30. Same bad result...

Jos

@josvanr
Copy link
Author

josvanr commented Dec 11, 2019

meantime I managed to get the frame rate constant by unchecking 'exposure,auto priority' on my C920, so now I can manage. But the software problem remains.

@josvanr
Copy link
Author

josvanr commented Dec 11, 2019

Hello

I did some more extensive testing and am attatching a small 'problematic'
example movie. The zip contains the original movie my_video-x7.mkv
and the movies generated by drv-scan, containing the motion.

The original movie was recorded using guvcview. It has motion events at 10
seconds and at 30 seconds. The first 10 secs, I'm pointing the cam at a light
area and the frame rate is 30 fps. At 10 secs, I whiggle the cam and pan it
over to a darker area in the room and leave it there.. The frame rate then
drops to 15 fps. At 30 secs, I whiggle the cam again and pan it back to the
light area and leave it. The frame rate then raises to 30 fps again.

The first event is detected correctly by dvr-scan at 9.4 secs. The generated movie
also shows the event correctly (my_video-x7.DSME_0001.avi). However,
the second event is detected at 22.3 secs instead of 30. The movie
my_video-x7.DSME_0002.avi does show the event, but it seems to
show a kind of time lapse version of the actual event. (seems to run faster).
(Note: at the very start of the original movie, the camera was focusing, this
was detected at the start, in my_video-x7.DSME_0000.avi)

This is the command I used to do the detection, plus output:

dvr-scan -t 0.001 -i my_video-x7.mkv
[DVR-Scan] Initializing scan context...
[DVR-Scan] Opened video my_video-x7.mkv (160 x 120 at 30.000 FPS).
[DVR-Scan] Scanning input video for motion events...
[DVR-Scan] Processed 1200 / 1200 frames read in 0.8 secs (avg 1582.0 FPS).
[DVR-Scan] Detected 3 motion events in input.
[DVR-Scan] Scan-only mode specified, list of motion events:

| Event # | Start Time | Duration | End Time |

| Event 1 | 00:00:01.2 | 00:00:02.2 | 00:00:03.4 |
| Event 2 | 00:00:09.4 | 00:00:09.0 | 00:00:18.3 |
| Event 3 | 00:00:22.3 | 00:00:06.7 | 00:00:29.1 |

[DVR-Scan] Motion events written to disk.

So yes I hope this can be resolved somehow . Or is this the way
it should work?

Jos

testmov.zip

@Breakthrough
Copy link
Owner

Technically this is working as expected, but this raises a good point of how to deal with variable framerate videos, which are more common on DVRs now a days. I may need to change the approach that DVR-Scan takes to instead rely on scenedetect, which uses ffmpeg as the tool to perform external splitting (or maybe provide an option to do it that way).

Thoughts? I was trying to come up with a "batteries included" approach, but I don't think OpenCV provides a way of outputting VFR videos, so I might not be able to address this without introducing an external dependency on ffmpeg.

@Breakthrough Breakthrough changed the title Incorrect start times Incorrect start times with variable frame rate (VFR) videos Jul 29, 2020
@Breakthrough Breakthrough added bug and removed bug labels Jul 29, 2020
@Breakthrough
Copy link
Owner

After some thought, the right way to go is definitely through better integration with PySceneDetect and FFMPEG to resolve this, as PySceneDetect also has the same issue (Breakthrough/PySceneDetect#168).

Even without VFR support, integration with PySceneDetect should prevent the issue you're seeing with the output video being sped up. Once that integration is complete (planned for v2.0), and the linked issue in PySceneDetect is resolved, DVR-Scan should no longer have any issues with VFR videos.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants