My experiments with 1080p | OpenCV on RB3 Pt. 4 | Qualcomm RB3 Robotic Arm Project
2019-07-15 01:00:00 +0000
Qualcomm RB3 Robotic Arm Project
64-bit, 96Boards, aarch64, ARM, ARMv8, Consumer Edition, DB410c, dragonboard410c, Linaro, Linux, fedora, arm64, aarch64, rock960, FPGA, raspberry pi, arduino, shild, hat
Vision Pt. 4
Ever wondered why none of the opencv demos and instructions run at 1080p? Why everyone prefers 640x480 in this day and age?
Truth be told
The truth is... its REALLY REALLY HARD. It requires massive amounts of compute, usually in the form of server grade CPU with massive core counts and compute accelerator cards like GPUs, FPGAs, Xeon Phi etc. To top that, don't forget the massive amount of bandwidth required to carry 1080p video stream from the camera to the processor.
And unless its really required, 1080p is just a waste of processing power for most opencv enthusiast usecases, for people who don't have access to specialized hardware.
Ok, but still how do I stream 1080p
...with great difficulty.
Using OpenCV directly For reasons I understand and still get annoyed with, opencv defaults to 640x480 on most webcams.
So the code:
vs = cv2.VideoCapture(0), will stream 640x480 frames regardless of your webcam's capabilities.
To stream 1080p you can use the following block:
vs = cv2.VideoCapture(0) codec = cv2.VideoWriter_fourcc( 'M', 'J', 'P', 'G' ) vs.set(6, codec) vs.set(5, 30) vs.set(3, 1920) vs.set(4, 1080)
vs = cv2.VideoCapture(0): opens up stream with
codec = cv2.VideoWriter_fourcc( 'M', 'J', 'P', 'G' )&
vs.set(6, codec): Sets codec to MJPG as most webcams only provide 30fps 1920x1080 stream using this codec. And OpenCV Like to default to UYUV.
vs.set(5, 30): Sets FPS to 30.
vs.set(4, 1080): Sets Resolution to 1920x1080.
This Should give you a 1080p 30fps stream, but if you start doing any processing on this, things will slow down really fast. And this is same for other methods we'll discuss now.
Although imutils doesn't support manual resolution input ATM. There is an open pull request on the repo to add said functionality.
And the implimentation would look somethin like this:
vs = WebcamVideoStream(src=0, resolution=(1920, 1080)).start()
Gstreamer is by far the most flexible and efficient way to input a video stream into opencv.
To stream 1080p over gstreamer you can use this one liner:
vs = cv2.VideoCapture("v4l2src device=/dev/video0 ! image/jpeg,framerate=30/1,width=1920, height=1080,type=video ! jpegdec ! videoconvert ! video/x-raw ! appsink", cv2.CAP_GSTREAMER)
This uses gstreamer to input a 1920x1080p30 MJPG video stream from a webcam, decode it and re-encode it to a format that opencv appreciates.
You can also use the following code to test using a test stream and not using a webcam:
vs = cv2.VideoCapture("videotestsrc ! video/x-raw,width=1920,heigh=1080 ! videoconvert ! video/x-raw ! appsink", cv2.CAP_GSTREAMER)
Its less of a headache to stick with 640x480, although a high end 1080p webcam might help is other departments such as fast autofocus, auto white balance abd color correction etc.