Proposal: VideoCapture to drop frames for low latency still capture

[crackwitz]

Use case: someone wants to capture still images. However the camera API, trying not to drop any frames, buffers several frames. This results in a pipeline of "stale" images when frames are read at less than full configured frame rate.

Those who don't know of this behavior, are often puzzled and don't know how to solve it.

The usual solution is to spawn a thread to read from VideoCapture without delay, and keep the most recent frame around for whatever requests it. This is often implemented from scratch by whoever needs this functionality.

Idea: introduce a property/flag to cv::VideoCapture that would enable the described solution. A VideoCapture::read() should not return the same frame twice, i.e. it should block if the most recent frame was already read. This is usually implemented using std::condition_variable.

This might interact with #12077.

Some example code for the principle (and direct access to v4l) can be found here:

• http://triffid-hunter.no-ip.info/Camera.cpp
• http://triffid-hunter.no-ip.info/CameraTest.cpp

[alalek]

I believe this make sense for cameras only. Or live video streams (from cameras).
Saved video streams from storage should not be affected (OpenCV doesn't add delays between frames - it is processed like images sequence).

Currently this behavior is VideoIO backend implementation specific. Some backends has builtin low-latency support via frame drops (see #11768 for MSMF).

Please note:

• there is .retrieve() method. This call waits for the next frame.
• .read() has stream parameter (it is used for stereo cameras, IR channel, RGBD depth channel, etc). Need to specify somehow which channels to "keep".
• .read() method decodes (converts to BGR format if necessary) frames.

[ghost]

Using CAP_PROP_BUFFERSIZE can also be useful. If the buffers are full, next frames are dumped

[yyf1986]

Using CAP_PROP_BUFFERSIZE can also be useful. If the buffers are full, next frames are dumped

how to use CAP_PROP_BUFFERSIZE?

[eltonfernando]

cap=cv2.cv2.VideoCapture(0)
size=10
cap.set(cv2.CAP_PROP_BUFFERSIZE,size)

[fisakhan]

Can I set backend to DC1394 [Firewire] v 2.x backend in python-opencv 4.1.2 on Ubuntu 18.04?