Random horizontally flipped image

[hellochenwang]

I'm using v4l2 to do video capture with gc2035 on orange pi pc.
randomly, about 1 out of 20 times, it gives horizontally flipped image. the code was never changed.

I tried to use v4l2-ctl to change the setting, but got the following error:

root@pi:~# v4l2-ctl -l -d /dev/video0
error 25 getting ctrl White Balance, Automatic
error 25 getting ctrl Exposure
error 25 getting ctrl Horizontal Flip
error 25 getting ctrl Vertical Flip
error 25 getting ctrl Color Effects

I tried several gc2035 cameras on different orange pis, the random flipping happened to all of them.

Could this be a driver issue?

[avafinger]

Unfortunately v4l2-ctl does not work, so it can't help in this case.

This is the first report of such a problem.
I usually use the code for streaming not for capturing still image that i suppose you are doing?
For streaming i discard first frame.

I think we need more info:

• OS used
• How you are grabbing images
• load avarage at the moment it happens the issue

Have you check if sensor is correctly connected all the time? Do you have a second sensor to see if you have same problem?

[hellochenwang]

• OS: Armbian Linux gun 3.4.112-sun8i #14 SMP PREEMPT Wed Sep 14 20:29:31 CEST 2016 armv7l GNU/Linux

• The code I use is from https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/capture.c.html

• Load: root@pi:~# uptime
  09:57:53 up 9:58, 1 user, load average: 1.48, 1.77, 1.54

I tried 3 gc2035 sensors on 3 orange pi pcs with the same setup, it happened to all of them.

Here are two screenshots showing the issue:

• This what I get most of the time:
  

• Sometimes I get this:
  

gc2035.ko is set to use hres=0, image resolution is set to 640x480, and I get around 16fps.

I noticed that it's not only h-flipping, the angle of the camera also changed a little bit.
But physically the camera position, marker position was not moved at all.

I imagine the sensor captured a bigger image say 1600x1200, and then got confused somehow and cropped 640x480 in a shifted area and sent a wrong sequence of the pixels...

[hellochenwang]

In my code, I tried the following setting, driver doesn't seems to respond to it:

• input.status = 0;
• input.status = V4L2_IN_ST_HFLIP;

or maybe it does, but the random h-flip happened even before this point.

int v4l2_set_input(v4l2device *dev) {
        /*
         * define video input
         * vfe_v4l2 the driver is forced to input = -1
         * set as the input = 0, works fine.
         */
        struct v4l2_input input;
        int count = 0;
        CLEAR(input);
        input.index = count;
        input.status = 0;//V4L2_IN_ST_HFLIP;
        while(!xioctl(dev->fd, VIDIOC_ENUMINPUT, &input)) {
                input.index = ++count;
        }
        count -= 1;

        assert(count > -1);

        if(xioctl(dev->fd, VIDIOC_S_INPUT, &count) == -1) {
                printf("Error selecting input %d", count);
                return 1;
        }
        return 0;
}

What could be the cause?

[JulesThuillier]

Hello guys !
I might have a solution for you, I have been stucked with the same problem for months on my orange pi, not with the GC2035 but with the OV5640 camera.
The problem is with the IOCTL driver for the camera commands, and especially the file "linux/videodev2.h", that does not work.

I had replaced it (actually not replaced, just included another version in my program) with this version : https://gist.github.com/JulesThuillier/bc7d1a852a7dd070af2072d946e20eed

And all controls worked !

I hope that will fix your problem ;)
(And it could be an update for Armbian too..)

[avafinger]

@JulesThuillier

Please, upload to gist the file /usr/src/linux-headers-3.4.112-sun8i/include/linux/compiler.h to see what is wrong or if it alignment or else.

[avafinger]

@hellochenwang

Try to gab the images with https://github.com/avafinger/cap-v4l2 just to check if at some point you get the error.

Compile using the A64 script:
https://github.com/avafinger/cap-v4l2/blob/master/build_script_A64.sh

You will need Linux headers source files (/usr/src/linux-headers-XXXXXXX) when building for H3, and install the kernel headers.

[hellochenwang]

the videodev2.h header in the gist seems the same as the one I have. I only have one linux header, so I suppose <linux/compiler.h> and "/usr/src/linux-headers-3.4.112-sun8i/include/linux/compiler.h" would be the same?

root@pi:~# diff videodev2.h /usr/src/linux-headers-3.4.112-sun8i/include/linux/videodev2.h 
64,65c64
< //#include <linux/compiler.h>
< #include "/usr/src/linux-headers-3.4.112-sun8i/include/linux/compiler.h"
---
> #include <linux/compiler.h>

[hellochenwang]

I tried cap-v4l2, but I got the following error:

root@pi:~/cap-v4l2# ./cap 640 480 4 1 0 0 0
---- cap parameters -----
width: 640
height: 480
v4l2 buffers: 4
exposure: 0
hflip: 0
vflip: 0
Mode: V4L2_MODE_VIDEO
Driver: "sunxi-vfe"
Card: "sunxi-vfe"
Bus: "sunxi_vfe sunxi_vfe.0"
Version: 1.0
Capabilities: 05000001
Input: 0
v4l2: unable to set stream parm.
v4l2: failed to init camera.

the odd thing is the code i have is basically doing the same thing, it was able to pass VIDIOC_S_PARM.

the ioctl sequence of the code i have:

open

struct v4l2_capability caps;
xioctl(dev->fd, VIDIOC_QUERYCAP, &caps)

struct v4l2_fmtdesc fmtdesc;
xioctl(dev->fd, VIDIOC_ENUM_FMT, &fmtdesc)

struct v4l2_input input
xioctl(dev->fd, VIDIOC_ENUMINPUT, &input)  //hflip here

int count
xioctl(dev->fd, VIDIOC_S_INPUT, &count)

struct v4l2_format fmt;
xioctl(dev->fd, VIDIOC_S_FMT, &fmt)

struct v4l2_streamparm setfps;
xioctl(dev->fd, VIDIOC_S_PARM, &setfps)

struct v4l2_requestbuffers req;
xioctl(dev->fd, VIDIOC_REQBUFS, &req)

struct v4l2_buffer buf;
xioctl(dev->fd, VIDIOC_QUERYBUF, &buf)

struct v4l2_buffer buf;
xioctl(dev->fd, VIDIOC_QBUF, &buf)

enum v4l2_buf_type type;
xioctl(dev->fd, VIDIOC_STREAMON, &type)

struct v4l2_buffer buf;
xioctl(dev->fd, VIDIOC_DQBUF, &buf)

xioctl(dev->fd, VIDIOC_QBUF, &buf)

the ioctl sequence of cap-v4l2:

open

struct v4l2_capability caps;
xioctl(fd, VIDIOC_QUERYCAP, &caps)

struct v4l2_input input;
xioctl(fd, VIDIOC_ENUMINPUT, &input)

input.index
xioctl(fd, VIDIOC_S_INPUT, &input.index)

struct v4l2_streamparm parms			//unable to set stream parm
xioctl(fd, VIDIOC_S_PARM, &parms)

struct v4l2_fmtdesc fmt
ioctl(fd, VIDIOC_ENUM_FMT, &fmt)

struct v4l2_frmsizeenum fsize;
xioctl(fd, VIDIOC_ENUM_FRAMESIZES, &fsize)

struct v4l2_format fmt;
xioctl(fd, VIDIOC_TRY_FMT, &fmt)

xioctl(fd, VIDIOC_S_FMT, &fmt)

struct v4l2_control control;		//v4l2_set_exposure
xioctl(fd, VIDIOC_G_CTRL, &control)
xioctl(fd, VIDIOC_S_CTRL, &control);

struct v4l2_control control;		//v4l2_set_hflip
xioctl(fd, VIDIOC_G_CTRL, &control);
xioctl(fd, VIDIOC_S_CTRL, &control);


struct v4l2_control control;		//v4l2_set_vflip
xioctl(fd, VIDIOC_G_CTRL, &control);
xioctl(fd, VIDIOC_S_CTRL, &control);

struct v4l2_requestbuffers req;
xioctl(fd, VIDIOC_REQBUFS, &req)

struct v4l2_buffer buf;
xioctl(fd, VIDIOC_QUERYBUF, &buf)

xioctl(fd, VIDIOC_QBUF, &buf)

type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
xioctl(fd, VIDIOC_STREAMON, &type)

xioctl(fd, VIDIOC_DQBUF, &buf)
xioctl(fd, VIDIOC_QBUF, &buf)

enum v4l2_buf_type type;
xioctl(fd, VIDIOC_STREAMOFF, &type)

[avafinger]

try first with: ./cap 640 480 4 1 -999 -1 -1

see the results.

[hellochenwang]

root@pi:~/cap-v4l2# ./cap 640 480 4 1 -999 -1 -1
---- cap parameters -----
width: 640
height: 480
v4l2 buffers: 4
exposure: -999
hflip: -1
vflip: -1
Mode: V4L2_MODE_VIDEO
Driver: "sunxi-vfe"
Card: "sunxi-vfe"
Bus: "sunxi_vfe sunxi_vfe.0"
Version: 1.0
Capabilities: 05000001
Input: 0
v4l2: unable to set stream parm.
v4l2: failed to init camera.
root@pi:~/cap-v4l2# 

the code stops at xioctl(fd, VIDIOC_S_PARM, &parms)

[hellochenwang]

dmesg has no difference between the two situations.

dmesg |grep 'VFE\|CSI\|GC2035'

FLIPPED:

[  825.748969] [VFE]vfe_open
[  825.748993] [VFE]..........................vfe clk open!.......................
[  825.749031] [VFE]vfe_open ok
[  825.751537] [VFE_ERR]input index(1) > dev->dev_qty(1)-1 invalid!
[  825.751571] [VFE]Set vfe core clk = 108000000, after Set vfe core clk = 100000000 
[  825.775687] [VFE]mclk on
[  825.799765] [CSI][GC2035]enable oe!
[  825.800121] [CSI][GC2035]V4L2_IDENT_SENSOR=2035
[  826.313868] [VFE]buffer_setup, buffer count=4, size=460800
[  826.426448] [VFE]capture video mode!
[  826.486079] [VFE]capture video first frame done!
[  826.545731] [VFE_WARN] Nobody is waiting on this video buffer,buf = 0xeeafec80


NORMAL:
[123235.064076] [VFE]vfe_open
[123235.064099] [VFE]..........................vfe clk open!.......................
[123235.064137] [VFE]vfe_open ok
[123235.066255] [VFE_ERR]input index(1) > dev->dev_qty(1)-1 invalid!
[123235.066287] [VFE]Set vfe core clk = 108000000, after Set vfe core clk = 100000000 
[123235.090050] [VFE]mclk on
[123235.114130] [CSI][GC2035]enable oe!
[123235.114484] [CSI][GC2035]V4L2_IDENT_SENSOR=2035
[123235.633380] [VFE]buffer_setup, buffer count=4, size=460800
[123235.714623] [VFE]capture video mode!
[123235.800431] [VFE]capture video first frame done!
[123235.860107] [VFE_WARN] Nobody is waiting on this video buffer,buf = 0xee7922c0

[avafinger]

v4l2-cap was based on kernel 3.4.39/65 and 3.10.102/105.

This sequence earlier than your current code here:

input.index
xioctl(fd, VIDIOC_S_INPUT, &input.index)

was to get rid of error you have pointed here:
[123235.066255] [VFE_ERR]input index(1) > dev->dev_qty(1)-1 invalid!
maybe the kernel has been fixed already but there is some side effect with this workaround.

I will revisit the code for kernel 3.4.112/113, but can't do anything till weekend, i am busy at the moment and need to finish some stuffs.

[avafinger]

Just an update, i get same error on kernel 3.4.113 with my GC2035, oddly this very same code runs with OV5640 on the same kernel, suggesting is indeed something wrong with GC2035 code.
I will try to debug the driver and kernel and see if i find something. This can take some time. If you find something else let me know.

[hellochenwang]

Yeah, will do!

[hellochenwang]

https://www.youtube.com/watch?v=vBpxhfBlVLU