cv2.VideoCapture() dropping 2 frames per second

[3dsf]

System information (version)

• OpenCV => 4.5.1-dirty
• Operating System / Platform => Linux 64 Bit
• Compiler => CMake

openCV.config.txt from system install, rather than enviroment

Detailed description

cv2.VideoCapture() is missing 2 frames per second

5 second video @ 23.98 fps

• ffmpeg -i c.mp4 ffmpeg.%03d.png
  • 120 images out
• cv2.VideoCapture()
  • 110 images out
• cap.get(cv2.CAP_PROP_FRAME_COUNT)
  • 135

Image 110 from VideoCapture appears to be the same as image 120 from ffmpeg

Tested with {1..5} seconds, 2 frames missing per second

Crashes with a None frame error as no more frames arrive.

Steps to reproduce

audio, fps, width, height = getVideoMetaData(in_file)
print(audio, fps, width, height)

cap = cv2.VideoCapture(in_file)
frame_num = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter('out.mp4',fourcc,fps,(1280,720))

i = 0
while(cap.isOpened()):
    ret, frame = cap.read()

    out_frame = processFrame(frame) 
    i = i + 1
    print(i,"/",frame_num)

    out.write(out_frame)
    cv2.imwrite('output/' + str(i) + ".png", out_frame)
    if cv2.waitKey(25) & 0xFF == ord('q'):
        break

cap.release()
out.release()

Same results with

• conda opencv 4.51
• conda opencv 4.3
• system install opencv 4.51 (CV model gutted for testing)

Video :

• demo 5 sec vid

Issue submission checklist

• I report the issue, it's not a question
• I checked the problem with documentation, FAQ, open issues,
  forum.opencv.org, Stack Overflow, etc and have not found solution
• I updated to latest OpenCV version and the issue is still there
• There is reproducer code and related data files: videos, images, onnx, etc

[makingglitches]

I'm having an issue of the same with a webm video. It reports a frame rate of an even 30. VLC reports a frame rate of 23.976024 which I'm pretty sure from avmux is a standard frame rate which commonly appears. Standard H264 mp4 encoding.

This is version 4.5.2, fresh build.

capture the video

cap = cv2.VideoCapture('../../input/01.webm')

get the video frames' width and height for proper saving of videos

frame_width = int(cap.get(3))
frame_height = int(cap.get(4))

frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
frame_rate = cap.get(cv2.CAP_PROP_FPS) // reports 30 fps

[3dsf]

I ended up using ffmpeg-pythons's tensorflow implementation.

If interested, you can take a look at https://github.com/vijishmadhavan/SkinDeep/blob/master/videoProcessor.ipynb to see the crude parse of the ffmpeg banner for fps and such.

[makingglitches]

@3dsf how did you get it working ? on ubuntu ? I'm having difficulty building it because of that damn bazel crap of googles.

[3dsf]

hey @makingglitches,

I cut out the model I was running and put in a .py file so you can easily add in your own magic.

This requires python -m pip install ffmpeg-python numpy and I presume you need a system install of ffmpeg

wget https://gist.githubusercontent.com/3dsf/7e17d8f69b078464d63f61ffd9a2fa78/raw/0c827697f04a9b5a850bd27818dc76c8a7937f93/processVideo.py

add your stuff at processFrame()

I seem to remember a bug for no sound videos, but I was only doing this for audio matching; if it is important to you, I'll let you fix it (Should be super minor).

My understanding is that openCV uses ffmpeg-python, so I don't feel bad sharing this stopgap here. Based on your original comment and my original thoughts, this may have routes in how fps is calculated.

[makingglitches]

@3dsf alot of variables in that script need predefined as they are out of scope when used. Kind of funny how this just wraps ffmpeg like one of my projects does it looks like :P interpreting its highly parseable output.

[makingglitches]

Appears frames reported between webm and mp4 are differing between lib and ffprobe.

01.webm (an input file i'm using) in the lib reports 6544 frames
ffprobe reports smaller framecount, interestingly it does NOT REPORT AN FPS VALUE which I would think might relate to why the input fps is incorrect as well in the lib.

ffprobe -count_frames -v error -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 01.webm
6194

1.mp4 (another input file I'm using) in the lib reports 7538 frames, the same as ffprobe.

I don't know what that suggests since I would think ffmpeg would be getting used all around for detection of these things and I would tend towards testing its output.

Sorry for the earlier mistake, still waking up.

[3dsf]

@makingglitches
sorry, I'm confused, did the code work for you? or did it error when you changed the inputVid variable from input.mp4 to a .webm file?

[makingglitches]

@3dsf that code file you provided didn't run without being tweaked heavily

[3dsf]

@makingglitches
sorry to ask for the clarification, was that the original colab code I presented or was it the
wget https://gist.githubusercontent.com/3dsf/7e17d8f69b078464d63f61ffd9a2fa78/raw/0c827697f04a9b5a850bd27818dc76c8a7937f93/processVideo.py
that required tweaking?

[makingglitches]

@3dsf yes it did not run as is
Same as whenever this happened before and someone decided to roll everything back

Christ provide me with some notion that I'll be well off and this won't all be scrapped and I'll fix the damn code which will require a time investment for sure lol