does not work with opencv4

[karu2003]

cap.c: In function ‘int v4l2_display_pix_format(int)’:
cap.c:197:1: warning: no return statement in function returning non-void [-Wreturn-type]
197 | }
| ^
cap.c: In function ‘int v4l2_retrieve_frame(int, int, int, int)’:
cap.c:409:5: error: ‘IplImage’ was not declared in this scope
409 | IplImage frame_ipl;
| ^~~~~~~~
cap.c:409:15: error: ‘frame_ipl’ was not declared in this scope
409 | IplImage frame_ipl;
| ^~~~~~~~~
cap.c:410:5: error: ‘CvMat’ was not declared in this scope
410 | CvMat frame_bgr;
| ^~~~~
cap.c:462:27: error: invalid conversion from ‘void’ to ‘unsigned char’ [-fpermissive]
462 | frame_yuv = calloc(ALIGN_16B(width) * height * 3, sizeof(unsigned char));
| ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| void*
cap.c:464:9: error: ‘frame_bgr’ was not declared in this scope; did you mean ‘frame_yuv’?
464 | frame_bgr = cvMat(height, width, CV_8UC3, (void *) frame_yuv);
| ^~~~~~~~~
| frame_yuv
cap.c:464:21: error: ‘cvMat’ was not declared in this scope
464 | frame_bgr = cvMat(height, width, CV_8UC3, (void *) frame_yuv);
| ^~~~~
cap.c:466:9: error: ‘cvSaveImage’ was not declared in this scope
466 | cvSaveImage(frame_name, &frame_bgr, 0);
| ^~~~~~~~~~~
cap.c: In function ‘int v4l2_close_camera(int, int)’:
cap.c:491:1: warning: control reaches end of non-void function [-Wreturn-type]
491 | }
| ^

[avafinger]

Try with this headers:

#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc_c.h>
#include <opencv2/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>
#include <opencv2/opencv.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/highgui.hpp>

Edit:

You need namespace:

using namespace std;
using namespace cv;

cv::cvMat

[karu2003]

I build "cap". Does not work, does not save jpeg.
Reports this.
pi@bpim2z:~/src/cap-v4l2$ ./cap 1270 720 4 1 -999 -1 -1
---- cap parameters -----
width: 1270
height: 720
v4l2 buffers: 4
exposure: -999
hflip: -1
vflip: -1
Mode: V4L2_MODE_VIDEO
Driver: "sun6i-csi"
Card: "sun6i-csi-capture"
Bus: "platform:1cb0000.camera"
Version: 1.0
Capabilities: a4200001
Input: 0
v4l2: unable to set stream parm.
v4l2: failed to init camera.

My init setup.
media-ctl --device /dev/media0 --set-v4l2 '"ov5640 1-003c":0[fmt:YUYV8_2X8/1280x720]'
v4l2-ctl --set-fmt-video=width=1280,height=720

it works
fswebcam --displayfps 1 -S 30 -d /dev/video0 -r 1280x720 -p YUV420P - > cam_1280x720_yuv420p.jpg

[karu2003]

pi@bpim2z:~/src/cap-v4l2$ media-ctl --device /dev/media0 -p
Media controller API version 6.2.2

Media device information

driver sun6i-csi
model Allwinner A31 CSI Device
serial
bus info platform:1cb0000.camera
hw revision 0x0
driver version 6.2.2

Device topology

• entity 1: sun6i-csi-bridge (2 pads, 2 links)
  type V4L2 subdev subtype Unknown flags 0
  device node name /dev/v4l-subdev0
  pad0: Sink
  [fmt:YUYV8_2X8/1280x720 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
  <- "ov5640 1-003c":0 [ENABLED]
  pad1: Source
  [fmt:YUYV8_2X8/1280x720 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
  -> "sun6i-csi-capture":0 [ENABLED,IMMUTABLE]

• entity 4: sun6i-csi-capture (1 pad, 1 link)
  type Node subtype V4L flags 0
  device node name /dev/video0
  pad0: Sink
  <- "sun6i-csi-bridge":1 [ENABLED,IMMUTABLE]

• entity 10: ov5640 1-003c (1 pad, 1 link)
  type V4L2 subdev subtype Sensor flags 0
  device node name /dev/v4l-subdev1
  pad0: Source
  [fmt:YUYV8_2X8/1280x720@1/30 colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range
  crop.bounds:(0,0)/2624x1964
  crop:(0,250)/2624x1456]
  -> "sun6i-csi-bridge":0 [ENABLED]

[avafinger]

Try to lower FPS (try 30 or 15) here:

https://github.com/avafinger/cap-v4l2/blob/master/cap.c#L296

    parms.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    parms.parm.capture.capturemode =0;
    parms.parm.capture.timeperframe.numerator = 1;
    parms.parm.capture.timeperframe.denominator = 30; // or 15
    if (-1 == xioctl(fd, VIDIOC_S_PARM, &parms)) {
        CAP_ERROR_RET("unable to set stream parm.");
    }

[karu2003]

the same!
parms.parm.capture.timeperframe.denominator = 7; // or 15
---- cap parameters -----
width: 1270
height: 720
v4l2 buffers: 4
exposure: -999
hflip: -1
vflip: -1
Mode: V4L2_MODE_VIDEO
Driver: "sun6i-csi"
Card: "sun6i-csi-capture"
Bus: "platform:1cb0000.camera"
Version: 1.0
Capabilities: a4200001
Input: 0
v4l2: unable to set stream parm.
v4l2: failed to init camera.

[avafinger]

Did you try?
parms.parm.capture.timeperframe.denominator = 30;

Then comment the whole code:

#if 0
parms.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
parms.parm.capture.capturemode =0;
parms.parm.capture.timeperframe.numerator = 1;
parms.parm.capture.timeperframe.denominator = 30; // or 15
if (-1 == xioctl(fd, VIDIOC_S_PARM, &parms)) {
CAP_ERROR_RET("unable to set stream parm.");
}
#endif

[karu2003]

:(
parms.parm.capture.timeperframe.denominator = 30;

[avafinger]

If you want to grab a JPG image, the sensor (OV5640) is able to deliver it without any conversion (without OpenCV). PR applied.

[PanderMusubi]

See also #7

Unfortunately, I get this error:

./cap 1920 1080 8 1 -999 -1 -1
---- cap parameters -----
width: 1920
height: 1080
v4l2 buffers: 8
exposure: -999
hflip: -1
vflip: -1
Mode: V4L2_MODE_VIDEO
v4l2: doesn't support video capturing.
v4l2: failed to init camera.

The system I use is Armbian_23.02.2_Pine64_jammy_current_5.15.93.img with to /boot/armbianEnv.txt the following line added:

camera_type=ov5640

Here is additional info https://pastebin.com/QbRAmxka and https://pastebin.com/PVeY1ef3

Just to be sure, I am using the correct kernel module for this camera? https://pine64.com/product/pine64-5-megapixel-camera-module/?v=0446c16e2e66 Has been a while that I had this working. The s5k4ec has nothing to do with this camera, correct?

[karu2003]

See also #7

Unfortunately, I get this error:

./cap 1920 1080 8 1 -999 -1 -1 ---- cap parameters ----- width: 1920 height: 1080 v4l2 buffers: 8 exposure: -999 hflip: -1 vflip: -1 Mode: V4L2_MODE_VIDEO v4l2: doesn't support video capturing. v4l2: failed to init camera.

The system I use is Armbian_23.02.2_Pine64_jammy_current_5.15.93.img with to /boot/armbianEnv.txt the following line added:

camera_type=ov5640

Here is additional info https://pastebin.com/QbRAmxka and https://pastebin.com/PVeY1ef3

Just to be sure, I am using the correct kernel module for this camera? https://pine64.com/product/pine64-5-megapixel-camera-module/?v=0446c16e2e66 Has been a while that I had this working. The s5k4ec has nothing to do with this camera, correct?

where is your camera ov5640?
Check out lsmod.

and add /etc/modules

[PanderMusubi]

Okay, loaded it manually now. This looks good?

# modprobe ov5640
# lsmod|grep ov5640
ov5640                 32768  0
v4l2_fwnode            24576  1 ov5640
v4l2_async             24576  2 v4l2_fwnode,ov5640
videodev              204800  6 sunxi_cedrus,v4l2_async,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem
mc                     49152  6 sunxi_cedrus,videodev,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem
# lsmod|grep v4l2
v4l2_fwnode            24576  1 ov5640
v4l2_async             24576  2 v4l2_fwnode,ov5640
v4l2_mem2mem           36864  1 sunxi_cedrus
videobuf2_v4l2         24576  2 sunxi_cedrus,v4l2_mem2mem
videobuf2_common       49152  5 sunxi_cedrus,videobuf2_dma_contig,videobuf2_v4l2,v4l2_mem2mem,videobuf2_memops
videodev              204800  6 sunxi_cedrus,v4l2_async,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem
mc                     49152  6 sunxi_cedrus,videodev,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem

Also added it to /etc/modules and booting reports

/var/log/syslog:Mar 29 10:38:47 pine20 systemd-modules-load[366]: Inserted module 'ov5640'

Then still

~/cap-v4l2# ./cap 1920 1080 8 1 -999 -1 -1
---- cap parameters -----
width: 1920
height: 1080
v4l2 buffers: 8
exposure: -999
hflip: -1
vflip: -1
Mode: V4L2_MODE_VIDEO
v4l2: doesn't support video capturing.
v4l2: failed to init camera.

To be sure, this is the correct driver for this camera? It used to work with different version of Ubuntu in the past. What are other ways to see what might be the problem? Is the the DTB file in Armbian supporting this?

[karu2003]

I dont See CSI . I must have 2 Video Dev.

[PanderMusubi]

I don't have a CSI module

# find /lib/modules -name '*csi*'|grep -v scsi|grep -v nfcsim

has no results.

Also, all sun4 modules only are:

# find /lib/modules -name 'sun4*'
/lib/modules/5.15.93-sunxi64/kernel/drivers/iio/adc/sun4i-gpadc-iio.ko
/lib/modules/5.15.93-sunxi64/kernel/sound/soc/sunxi/sun4i-spdif.ko
/lib/modules/5.15.93-sunxi64/kernel/sound/soc/sunxi/sun4i-i2s.ko

I have been looking into the build config. I think this file is used https://github.com/armbian/build/blob/main/config/kernel/linux-sunxi64-current.config#L4479 and has ov5640 as a module but nothing on CSI. Should it have a line like https://github.com/armbian/build/blob/main/config/kernel/linux-sunxi-current.config#L4274 with =m ? Would that provide me with the required module?

(Additionally, I found https://github.com/armbian/build/blob/main/patch/kernel/archive/sunxi-6.1/patches.armbian/arm-dts-overlay-Add-Overlays-for-sunxi.patch#L1205 but I don't think this interferes.)

[karu2003]

the camera without csi will not work. is there csi in your driver tree? I also use Armbian. I added my overlays. i use Banana Pi M2 Zero.

[PanderMusubi]

Thanks. I'm building Armbian install image with CSI as module. Do you use SUN4I_CSI or SUN6I_CSI?

Where/what are the overlays you are using?

[avafinger]

CONFIG_VIDEO_SUN6I_CSI=m

and also check out how to use it:

https://github.com/avafinger/bananapi-zero-ubuntu-base-minimal#working-with-cameras

[PanderMusubi]

I have the kernel modules not in my build and I can load all of them. How can I check if the device tree overlay https://github.com/avafinger/OV5640_camera/blob/master/sun50i-a64-pine64-plus.dtb is already supported by the Armbian build?

[PanderMusubi]

Ah, just found dtdiff and dtc -I dtb -O dts file.dtb and will dive into it.

[karu2003]

I have overlay for the Camera OV5640. Only 1 command armbian-add overlay ......

[PanderMusubi]

I did the following:

1. build armbian edge, installed and booted
2. wget https://github.com/avafinger/OV5640_camera/raw/master/sun50i-a64-pine64-plus.dtb (I have noticed also a s5k4ec exists, doubled checked that I was using the correct one)
3. dtc -I dtb -O dts sun50i-a64-pine64-plus.dtb > sun50i-a64-pine64-plus.dts
4. armbian-add-overlay sun50i-a64-pine64-plus.dts which installs it in /boot/overlay-user/sun50i-a64-pine64-plus.dtbo and adds the line user_overlays=sun50i-a64-pine64-plus to /boot/armbianEnv.txt (in that file is also overlay_prefix=sun50i-a64)
5. added camera_type=ov5640 to /boot/armbianEnv.txt
6. added ov5640 to /etc/modules
7. rebooted and checked lsmod|grep ov56
   ov5640 40960 0
   v4l2_fwnode 24576 1 ov5640
   v4l2_async 24576 2 v4l2_fwnode,ov5640
   videodev 204800 5 sunxi_cedrus,v4l2_async,ov5640,videobuf2_v4l2,v4l2_mem2mem
   mc 53248 7 sunxi_cedrus,v4l2_async,videodev,ov5640,videobuf2_v4l2,videobuf2_common,v4l2_mem2mem
8. wget https://github.com/PanderMusubi/cap-v4l2/archive/refs/heads/master.zip and unzipped it
9. ran ./install_deps.sh and ./build_script_A64.sh
10. installed v4l-utils and ran v4l2-ctl --list-devices
    cedrus (platform:cedrus):
    /dev/video0
    /dev/media0
11. so missing the csi device I think, also running ./cap 1270 720 4 1 -999 -1 -1 gives an error
    ---- cap parameters -----
    width: 1270
    height: 720
    v4l2 buffers: 4
    exposure: -999
    hflip: -1
    vflip: -1
    Mode: V4L2_MODE_VIDEO
    v4l2: doesn't support video capturing.
    v4l2: failed to init camera.
12. also build fswebcam and ran ./fswebcam --displayfps 1 -S 30 -d /dev/video0 -r 1280x720 -p YUV420P - > cam_1280x720_yuv420p.jpg
    --- Opening /dev/video0...
    Trying source module v4l2...
    /dev/video0 opened.
    No input was specified, using the first.
    Unable to query input 0.
    VIDIOC_ENUMINPUT: Inappropriate ioctl for device
13. running media-ctl -p only reports on cedrus
14. running v4l2-ctl --list-devices give
    cedrus (platform:cedrus):
    /dev/video0
    /dev/media0

Any ideas how to fix this or make the next step?

[avafinger]

The error reported here:
https://github.com/avafinger/cap-v4l2/blob/master/cap.c#L285

That means your camera only supports MPLANE. There is no support for MPLANE in cap-v4l2

Change that part of the code to check if true:

 if (!(caps.capabilities & V4L2_CAP_VIDEO_CAPTURE)) {
     if ((cap.capabilities & V4L2_CAP_VIDEO_CAPTURE_MPLANE))  {
         CAP_ERROR_RET("doesn't not support mplane at the momment.");
     } else {
        CAP_ERROR_RET("doesn't support video capturing.");
    }
 }

I think the same is also true for fswebcam.
Maybe installing v4l2loopback would help.

Maybe i will revisit fswebcam and add mplane support if you don't find any suitable version on the net, i can't promise it soon.