Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Camera with libcamera #55

Open
1 of 5 tasks
simonschmeisser opened this issue Nov 8, 2020 · 25 comments
Open
1 of 5 tasks

Camera with libcamera #55

simonschmeisser opened this issue Nov 8, 2020 · 25 comments

Comments

@simonschmeisser
Copy link
Contributor

simonschmeisser commented Nov 8, 2020

The big picture is that libcamera talks to the sun6i_csi and the camera drivers. It also provides a gstreamer source. This source can then be used by the normal sfos camera app.

Issues:

  • flash driver is broken, needs to be disabled in dts: sailfish-on-dontbeevil/kernel-megi@9429f77 thanks @piggz

  • libcamera as packaged by mal crashes due to (?) gnutls patch (workaround: set HAVE_GNUTLS to 0 in libcamera/meson.build)

  • missing firmware files: dmesg:

[ 1369.125711] ov5640 3-004c: ov5640_af_init: no autofocus firmware available (ov5640_af.bin)
[ 1369.125730] ov5640 3-004c: ov5640_set_ctrl_focus: no autofocus firmware loaded

and

[ 1580.581516] sun6i-csi 1cb0000.csi: Direct firmware load for gc2145-init.bin failed with error -2
[ 1580.581531] gc2145 3-003c: Failed to read firmware gc2145-init.bin, continuing anyway...

Update: gc2145-init.bin is not needed (thanks MartijnBraam)

for more see https://paste.ubuntu.com/p/hw5ZVpB2cr/

  • kernel driver for sun6i_csi is missing a feature:

"[0:26:20.623064785] [4767] ERROR V4L2 v4l2_videodevice.cpp:988 /dev/video0[cap]: Media bus code filtering not supported by the device" is an issue that needs to be solved
see commit d5f74a1eff9aef3ba23673c8ef74719ba8a105f7 in the mainline kernel for an example
( torvalds/linux@d5f74a1 )

will need to be added here: https://github.com/sailfish-on-dontbeevil/kernel-megi/blob/9429f77dc7e252a99f11cd0613913e5634a2e981/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c#L708

  • figuring out why opening /dev/video0 fails

thanks to @pinchartl and @MartijnBraam for infos

@simonschmeisser
Copy link
Contributor Author

Some logs (and documenting debug tools):
gstreamer first run: LIBCAMERA_LOG_LEVELS=*:DEBUG gst-launch-1.0 libcamerasrc https://paste.ubuntu.com/p/PQ84sFw7d6/
second: https://paste.ubuntu.com/p/Y6wfCKCmZV/

media-ctl -p -d /dev/media0 : https://paste.ubuntu.com/p/6Z5BWHyXMZ/

v4l2-ctl --all and LIBCAMERA_LOG_LEVELS=*:DEBUG cam -l https://paste.ubuntu.com/p/xQFjypXw3J/

another suggestion was yavta from https://git.ideasonboard.org/?p=yavta.git;a=summary not yet packaged however

@kbingham
Copy link

kbingham commented Nov 9, 2020

" figuring out why opening /dev/video0 fails"

This is often a permissions thing. Make sure the user is in the 'video' group.

libcamera as packaged by mal crashes due to (?) gnutls patch (workaround: set HAVE_GNUTLS to 0 in libcamera/meson.build)

This is curios. Can you post a report of this to the libcamera mailing list?

@simonschmeisser
Copy link
Contributor Author

I forgot to link to the patches for libcamera added due to the gnutls version on SailfishOS being old (?) https://git.sailfishos.org/mal/libcamera
https://git.sailfishos.org/mal/libcamera/blob/master/rpm/gnutls2.patch

@kbingham
Copy link

kbingham commented Nov 9, 2020

Ok, if that's all that is required to support the older gnutls, we can prepare a patch for that to be integrated in libcamera.

@mlehtima
Copy link

mlehtima commented Nov 9, 2020

The gnutls patch to libcamera is not tested and we had some issues during testing. It might be that something else could be needed also.

@pinchartl
Copy link

I forgot to link to the patches for libcamera added due to the gnutls version on SailfishOS being old (?) https://git.sailfishos.org/mal/libcamera
https://git.sailfishos.org/mal/libcamera/blob/master/rpm/gnutls2.patch

If I'm not mistaken, the latest gnutls 2.x version was released 4 years ago. Isn't this a pretty big security concern ?

@kbingham
Copy link

kbingham commented Nov 9, 2020

The gnutls patch to libcamera is not tested and we had some issues during testing. It might be that something else could be needed also.

Ok, it looks like we wouldn't really want to support an older version of GnuTLS anyway. It might be better to investigate upgrading the packages in your distribution.

@mlehtima
Copy link

mlehtima commented Nov 9, 2020

gnutls is not really used any more in Sailfish OS, after next release it is not needed at all by default.

@simonschmeisser
Copy link
Contributor Author

I'm fine with having this as a downstream patch ... but if it's not needed otherwise, it should be safe to upgrade to gnutls 3?

@simonschmeisser
Copy link
Contributor Author

The AF firmware currently does not get started correctly after uploading ... under investigation. This aborted the driver startup and might have been the reason for not being able to open /dev/video0

I also added | V4L2_CAP_IO_MC to https://github.com/sailfish-on-dontbeevil/kernel-megi/blob/9429f77dc7e252a99f11cd0613913e5634a2e981/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c#L708

This fixes (?) the error about media bus filtering not supported in cam -l and gives me the following new output:

[nemo@PinePhone ~]$ LIBCAMERA_LOG_LEVELS=*:DEBUG cam -l
[0:06:33.805212261] [3949] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyLinux"
[0:06:33.805583485] [3949] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyThread"
[0:06:33.806064039] [3949]  WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/lib/libcamera'
[0:06:33.806308315] [3949]  INFO Camera camera_manager.cpp:295 libcamera v0.0.0
[0:06:33.806830186] [3950] DEBUG Camera camera_manager.cpp:107 Starting camera manager
[0:06:33.813841541] [3950] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "cedrus" created from /dev/media1
[0:06:33.814059952] [3950] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media1 due to 1 missing dependencies
[0:06:33.815071249] [3950] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media1 found
[0:06:33.815211150] [3950] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media1: cedrus
[0:06:33.815881794] [3950] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "sun6i-csi" created from /dev/media0
[0:06:33.816889552] [3950] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media0 due to 1 missing dependencies
[0:06:33.817797059] [3950] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media0 found
[0:06:33.817901475] [3950] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media0: sun6i-csi
[0:06:33.818407478] [3950] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'SimplePipelineHandler'
[0:06:33.818553128] [3950] DEBUG DeviceEnumerator device_enumerator.cpp:311 Successful match for media device "sun6i-csi"
[0:06:33.818944094] [3950] DEBUG V4L2 v4l2_videodevice.cpp:573 /dev/video0[cap]: Opened device platform:csi: sun6i-video: sun6i-csi
[0:06:33.825634465] [3950] DEBUG MediaDevice media_device.cpp:808 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 1
[0:06:33.825808353] [3950] ERROR V4L2 v4l2_subdevice.cpp:430 'ov5640 3-004c': Unable to set format on pad 0: Invalid argument
[0:06:33.825927139] [3950] DEBUG SimplePipeline simple.cpp:273 Media bus code 0x1007 not supported for this pipeline
[0:06:33.826013895] [3950] ERROR V4L2 v4l2_subdevice.cpp:430 'ov5640 3-004c': Unable to set format on pad 0: Invalid argument
[0:06:33.826105816] [3950] DEBUG SimplePipeline simple.cpp:273 Media bus code 0x1008 not supported for this pipeline
[0:06:33.826185492] [3950] ERROR V4L2 v4l2_subdevice.cpp:430 'ov5640 3-004c': Unable to set format on pad 0: Invalid argument
[0:06:33.826264668] [3950] DEBUG SimplePipeline simple.cpp:273 Media bus code 0x2006 not supported for this pipeline
[0:06:33.826343886] [3950] ERROR V4L2 v4l2_subdevice.cpp:430 'ov5640 3-004c': Unable to set format on pad 0: Invalid argument
[0:06:33.826422854] [3950] DEBUG SimplePipeline simple.cpp:273 Media bus code 0x2008 not supported for this pipeline
[0:06:33.826499990] [3950] ERROR V4L2 v4l2_subdevice.cpp:430 'ov5640 3-004c': Unable to set format on pad 0: Invalid argument
[0:06:33.826579499] [3950] DEBUG SimplePipeline simple.cpp:273 Media bus code 0x3001 not supported for this pipeline
[0:06:33.826656843] [3950] ERROR V4L2 v4l2_subdevice.cpp:430 'ov5640 3-004c': Unable to set format on pad 0: Invalid argument
[0:06:33.826734769] [3950] DEBUG SimplePipeline simple.cpp:273 Media bus code 0x3002 not supported for this pipeline
[0:06:33.826810239] [3950] ERROR V4L2 v4l2_subdevice.cpp:430 'ov5640 3-004c': Unable to set format on pad 0: Invalid argument
[0:06:33.826889665] [3950] DEBUG SimplePipeline simple.cpp:273 Media bus code 0x3013 not supported for this pipeline
[0:06:33.826966551] [3950] ERROR V4L2 v4l2_subdevice.cpp:430 'ov5640 3-004c': Unable to set format on pad 0: Invalid argument
[0:06:33.827044144] [3950] DEBUG SimplePipeline simple.cpp:273 Media bus code 0x3014 not supported for this pipeline
[0:06:33.827119072] [3950] ERROR V4L2 v4l2_subdevice.cpp:430 'ov5640 3-004c': Unable to set format on pad 0: Invalid argument
[0:06:33.827205412] [3950] DEBUG SimplePipeline simple.cpp:273 Media bus code 0x4001 not supported for this pipeline
[0:06:33.827279132] [3950] ERROR SimplePipeline simple.cpp:323 No valid configuration found
[0:06:33.827366472] [3950] DEBUG MediaDevice media_device.cpp:808 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 0
[0:06:34.024118924] [3950] ERROR MediaDevice media_device.cpp:802 /dev/media0[sun6i-csi]: Failed to setup link: No such device or address
[0:06:34.024674699] [3950] DEBUG V4L2 v4l2_videodevice.cpp:1364 /dev/video0[cap]: Releasing buffers
[0:06:34.024818099] [3950] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
Available cameras:

[nemo@PinePhone ~]$ media-ctl -p -d /dev/media0
Media controller API version 5.10.0

Media device information
------------------------
driver          sun6i-csi
model           Allwinner Video Capture Device
serial          
bus info        platform:1cb0000.csi
hw revision     0x0
driver version  5.10.0

Device topology
- entity 1: sun6i-csi (1 pad, 2 links)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "ov5640 3-004c":0 []
                <- "gc2145 3-003c":0 []

- entity 5: ov5640 3-004c (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev0
        pad0: Source
                [fmt:YUYV8_2X8/1280x720@1/30 colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "sun6i-csi":0 []

- entity 7: gc2145 3-003c (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:YUYV8_2X8/1280x720@1/10 field:none colorspace:srgb]
                -> "sun6i-csi":0 []


@pinchartl
Copy link

I also added | V4L2_CAP_IO_MC to https://github.com/sailfish-on-dontbeevil/kernel-megi/blob/9429f77dc7e252a99f11cd0613913e5634a2e981/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c#L708

I'm afraid you need a bit more than that. The flag itself reports that the driver conforms with the related API requirements. From a libcamera point of view, this means that the driver should implement support for filtering formats during format enumeration through the mbus_code argument (see https://linuxtv.org/downloads/v4l-dvb-apis/userspace-api/v4l/vidioc-enum-fmt.html). Without that, the simple pipeline handler won't work properly.

@simonschmeisser
Copy link
Contributor Author

Thanks for the pointers! If I understand correctly I basically need to replicate/implement this https://github.com/sailfish-on-dontbeevil/kernel-megi/blob/9429f77dc7e252a99f11cd0613913e5634a2e981/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c#L50
here https://github.com/sailfish-on-dontbeevil/kernel-megi/blob/9429f77dc7e252a99f11cd0613913e5634a2e981/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c#L331

What I don't understand yet is why libcamera wouldn't work with a unfiltered list as well? Or does it give up after a certain number of tries?

@simonschmeisser
Copy link
Contributor Author

Hmm, I'm confused ... I tried implementing (a rough version of what I wrote above) but it does not seem to get called ever:

static int vidioc_enum_fmt_vid_cap(struct file *file, void *priv,
				   struct v4l2_fmtdesc *f)
{
	u32 index = f->index;

	if (index >= ARRAY_SIZE(supported_pixformats))
		return -EINVAL;

    struct sun6i_video *video = video_drvdata(file);
    dev_err(video->csi->dev, "fmt_vid_cap %d .. %d", f->mbus_code, f->index);
    
    if (f->mbus_code == 0) {
        f->pixelformat = supported_pixformats[index];

	return 0;
        
    }
    
    if (index > 0)
        return -EINVAL;
    
    
    switch (f->mbus_code) {
        case MEDIA_BUS_FMT_YUYV8_2X8:
            f->pixelformat = V4L2_PIX_FMT_YUYV;
            return 0;
        case MEDIA_BUS_FMT_YVYU8_2X8:
            f->pixelformat = V4L2_PIX_FMT_YVYU;
            return 0;
        case MEDIA_BUS_FMT_UYVY8_2X8:
            f->pixelformat = V4L2_PIX_FMT_UYVY;
            return 0;
        case MEDIA_BUS_FMT_VYUY8_2X8:
            f->pixelformat = V4L2_PIX_FMT_VYUY;
            return 0;
            
        default:
            return -EINVAL;
    }

	return 0;
}

The text message never gets printed to dmesg.

I also tried v4l2-compliance and it gives the following output

[root@PinePhone sun6i-csi]# v4l2-compliance -T
v4l2-compliance SHA: not available, 32 bits

                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_TRY_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
Compliance test for sun6i-video device /dev/video0:

Driver Info:
        Driver name      : sun6i-video
        Card type        : sun6i-csi
        Bus info         : platform:csi
        Driver version   : 5.10.0
        Capabilities     : 0xa4200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x24200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : sun6i-csi
        Model            : Allwinner Video Capture Device
        Serial           : 
        Bus info         : platform:1cb0000.csi
        Media version    : 5.10.0
        Hardware revision: 0x00000000 (0)
        Driver version   : 5.10.0
Entity Info:
        ID               : 1
        Name             : sun6i-csi
        Type             : V4L2 I/O
        Pad              : 0: Sink, Must Connect

Required ioctls:
        test MC information (see 'Media Driver Info' above): OK
                warn: v4l2-compliance.cpp(633): media bus_info 'platform:1cb0000.csi' differs from V4L2 bus_info 'platform:csi'
        test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_TRY_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
        test second /dev/video0 open: OK
                warn: v4l2-compliance.cpp(633): media bus_info 'platform:1cb0000.csi' differs from V4L2 bus_info 'platform:csi'
        test VIDIOC_QUERYCAP: OK
                VIDIOC_S_PRIORITY returned -1 (Device or resource busy)
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

                VIDIOC_CROPCAP returned -1 (Inappropriate ioctl for device)
                VIDIOC_CROPCAP returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
Debug ioctls:
                VIDIOC_DBG_G_REGISTER returned -1 (Inappropriate ioctl for device)
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
        test VIDIOC_LOG_STATUS: OK

Input ioctls:
                VIDIOC_G_STD returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_TUNER returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
                VIDIOC_G_FREQUENCY returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_FREQUENCY returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
                VIDIOC_S_HW_FREQ_SEEK returned -1 (Inappropriate ioctl for device)
        test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
                VIDIOC_ENUMAUDIO returned -1 (Inappropriate ioctl for device)
        test VIDIOC_ENUMAUDIO: OK (Not Supported)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_ENUMINPUT returned -1 (Invalid argument)
                VIDIOC_S_INPUT returned -1 (Invalid argument)
        test VIDIOC_G/S/ENUMINPUT: OK
                VIDIOC_G_AUDIO returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_AUDIO returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
                VIDIOC_G_MODULATOR returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S_MODULATOR: OK (Not Supported)
                VIDIOC_G_FREQUENCY returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_FREQUENCY returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
                VIDIOC_ENUMAUDOUT returned -1 (Inappropriate ioctl for device)
        test VIDIOC_ENUMAUDOUT: OK (Not Supported)
                VIDIOC_G_OUTPUT returned -1 (Inappropriate ioctl for device)
                VIDIOC_ENUMOUTPUT returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_OUTPUT returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
        test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
                VIDIOC_G_STD returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_STD returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_STD returned -1 (Inappropriate ioctl for device)
                VIDIOC_ENUMSTD returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYSTD returned -1 (Inappropriate ioctl for device)
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
                VIDIOC_G_DV_TIMINGS returned -1 (Inappropriate ioctl for device)
                VIDIOC_ENUM_DV_TIMINGS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERY_DV_TIMINGS returned -1 (Inappropriate ioctl for device)
        test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
                VIDIOC_DV_TIMINGS_CAP returned -1 (Inappropriate ioctl for device)
        test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
                VIDIOC_G_EDID returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_EDID returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK (Not Supported)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
        test VIDIOC_QUERYCTRL: OK (Not Supported)
                VIDIOC_G_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_CTRL returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S_CTRL: OK (Not Supported)
                VIDIOC_G_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK (Not Supported)
                VIDIOC_SUBSCRIBE_EVENT returned -1 (Invalid argument)
                VIDIOC_SUBSCRIBE_EVENT returned -1 (Invalid argument)
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK (Not Supported)
                VIDIOC_G_JPEGCOMP returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_JPEGCOMP returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
        Standard Controls: 0 Private Controls: 0

Format ioctls (Input 0):
                VIDIOC_ENUM_FMT returned -1 (Invalid argument)
                VIDIOC_ENUM_FMT returned -1 (Invalid argument)
                fail: v4l2-test-formats.cpp(312): Video Capture cap set, but no Video Capture formats defined
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: FAIL
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                warn: v4l2-test-formats.cpp(1320): S_PARM is supported for buftype 1, but not for ENUM_FRAMEINTERVALS
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
                VIDIOC_G_PARM returned -1 (Invalid argument)
                VIDIOC_S_PARM returned -1 (Invalid argument)
        test VIDIOC_G/S_PARM: OK
                VIDIOC_G_FBUF returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G_FBUF: OK (Not Supported)
                VIDIOC_G_FMT returned -1 (Invalid argument)
                fail: v4l2-test-formats.cpp(444): pixelformat 31384142 (BA81) for buftype 1 not reported by ENUM_FMT
        test VIDIOC_G_FMT: FAIL
                fail: v4l2-test-formats.cpp(444): pixelformat 31384142 (BA81) for buftype 1 not reported by ENUM_FMT
        test VIDIOC_TRY_FMT: FAIL
                fail: v4l2-test-formats.cpp(444): pixelformat 31384142 (BA81) for buftype 1 not reported by ENUM_FMT
        test VIDIOC_S_FMT: FAIL
                VIDIOC_G_SLICED_VBI_CAP returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
                VIDIOC_CROPCAP returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_CROP returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_SELECTION returned -1 (Inappropriate ioctl for device)
        test Cropping: OK (Not Supported)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_SELECTION returned -1 (Inappropriate ioctl for device)
        test Composing: OK (Not Supported)
                VIDIOC_S_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_S_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
        test Scaling: OK

Codec ioctls (Input 0):
                VIDIOC_ENCODER_CMD returned -1 (Inappropriate ioctl for device)
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
                VIDIOC_G_ENC_INDEX returned -1 (Inappropriate ioctl for device)
        test VIDIOC_G_ENC_INDEX: OK (Not Supported)
                VIDIOC_DECODER_CMD returned -1 (Inappropriate ioctl for device)
        test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_TRY_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_REQBUFS returned -1 (Invalid argument)
                VIDIOC_REQBUFS returned -1 (Invalid argument)
                VIDIOC_REQBUFS returned -1 (Invalid argument)
                VIDIOC_REQBUFS returned -1 (Cannot allocate memory)
                fail: v4l2-test-buffers.cpp(610): q.reqbufs(node, 1)
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: FAIL
                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_TRY_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_EXPBUF returned -1 (Invalid argument)
                fail: v4l2-test-buffers.cpp(747): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing or malfunctioning.
                fail: v4l2-test-buffers.cpp(748): VIDIOC_EXPBUF is supported, but the V4L2_MEMORY_MMAP support is missing, probably due to earlier failing format tests.
        test VIDIOC_EXPBUF: OK (Not Supported)
        test Requests: OK (Not Supported)
                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_TRY_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)

                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_TRY_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_TRY_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_TRY_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERY_EXT_CTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_TRY_EXT_CTRLS returned -1 (Inappropriate ioctl for device)
                VIDIOC_QUERYCTRL returned -1 (Inappropriate ioctl for device)
                VIDIOC_G_SELECTION returned -1 (Inappropriate ioctl for device)
Total for sun6i-video device /dev/video0: 45, Succeeded: 40, Failed: 5, Warnings: 3

@pinchartl
Copy link

Thanks for the pointers! If I understand correctly I basically need to replicate/implement this https://github.com/sailfish-on-dontbeevil/kernel-megi/blob/9429f77dc7e252a99f11cd0613913e5634a2e981/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c#L50
here https://github.com/sailfish-on-dontbeevil/kernel-megi/blob/9429f77dc7e252a99f11cd0613913e5634a2e981/drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c#L331

What I don't understand yet is why libcamera wouldn't work with a unfiltered list as well? Or does it give up after a certain number of tries?

libcamera needs to know what V4L2 pixel formats a video node supports for a given media bus code. The V4L2 API used to only support enumerating all supported pixel formats, until the recent addition of format enumeration filtering.

@pinchartl
Copy link

Format ioctls (Input 0):
VIDIOC_ENUM_FMT returned -1 (Invalid argument)
VIDIOC_ENUM_FMT returned -1 (Invalid argument)
fail: v4l2-test-formats.cpp(312): Video Capture cap set, but no Video Capture formats defined

This doesn't seem right. As you're using a non-mainline kernel I can't really help much to diagnose the issue, but it shouldn't be too difficult to trace the code paths to figure out why an error is returned.

@simonschmeisser
Copy link
Contributor Author

Some progress:

[root@PinePhone sun6i-csi]# LIBCAMERA_LOG_LEVELS=*:DEBUG cam -l
[2:21:50.335547036] [4489] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyLinux"
[2:21:50.336062849] [4489] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyThread"
[2:21:50.336532202] [4489]  WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/lib/libcamera'
[2:21:50.336780129] [4489]  INFO Camera camera_manager.cpp:295 libcamera v0.0.0
[2:21:50.337279983] [4490] DEBUG Camera camera_manager.cpp:107 Starting camera manager
[2:21:50.345082266] [4490] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "cedrus" created from /dev/media1
[2:21:50.345306609] [4490] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media1 due to 1 missing dependencies
[2:21:50.346292900] [4490] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media1 found
[2:21:50.346412530] [4490] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media1: cedrus
[2:21:50.347040014] [4490] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "sun6i-csi" created from /dev/media0
[2:21:50.348166270] [4490] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media0 due to 1 missing dependencies
[2:21:50.349087183] [4490] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media0 found
[2:21:50.349191646] [4490] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media0: sun6i-csi
[2:21:50.349746835] [4490] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'SimplePipelineHandler'
[2:21:50.349885341] [4490] DEBUG DeviceEnumerator device_enumerator.cpp:311 Successful match for media device "sun6i-csi"
[2:21:50.350254940] [4490] DEBUG V4L2 v4l2_videodevice.cpp:573 /dev/video0[cap]: Opened device platform:csi: sun6i-video: sun6i-csi
[2:21:50.361488158] [4490] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_BE
[2:21:50.361746627] [4490] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBR ]
[2:21:50.361844297] [4490]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format RGBR
[2:21:50.361998929] [4490] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_LE
[2:21:50.362167352] [4490] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBP ]
[2:21:50.362351943] [4490] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-UYVY8_2X8
[2:21:50.362637747] [4490] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, UYVY ]
[2:21:50.362726126] [4490]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[2:21:50.362939760] [4490] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-YUYV8_2X8
[2:21:50.363267857] [4490] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, YUYV ]
[2:21:50.363354610] [4490]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[2:21:50.363511283] [4490] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SBGGR8_1X8
[2:21:50.363624913] [4490] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ BA81 ]
[2:21:50.363775794] [4490] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGRBG8_1X8
[2:21:50.363886382] [4490] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GRBG ]
[2:21:50.364160185] [4490] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGBRG8_1X8
[2:21:50.364282982] [4490] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GBRG ]
[2:21:50.364445697] [4490] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SRGGB8_1X8
[2:21:50.364560577] [4490] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGGB ]
[2:21:50.364716500] [4490] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-0x4001
[2:21:50.364826880] [4490] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [  ]
[2:21:50.365049097] [4490] DEBUG MediaDevice media_device.cpp:808 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 0
[2:21:50.556104634] [4490] ERROR MediaDevice media_device.cpp:802 /dev/media0[sun6i-csi]: Failed to setup link: No such device or address
[2:21:50.558599780] [4490] DEBUG Camera camera_manager.cpp:160 Pipeline handler "SimplePipelineHandler" matched
Available cameras:
1: Internal front camera (/base/i2c-csi/rear-camera@4c)
[2:21:50.559235764] [4490] DEBUG V4L2 v4l2_videodevice.cpp:1364 /dev/video0[cap]: Releasing buffers
[2:21:50.559352769] [4490] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.

There is a bug in ov6540 where it uses the current frame rate to check if a certain format is possible but some (all?) formats are actually only available at a lower FPS. I changed that to 2FPS (0). Now I get a bit further. For the first half of the output I implemented the format filtering already, so it gets one step further.

It then complained about the resolution (VIDIOC_ENUM_FRAMESIZES) https://www.kernel.org/doc/html/v4.13/media/uapi/v4l/vidioc-enum-framesizes.html not being available for sun6i-csi so I implemented that (returning 4800x4800 as written in the header). ov5640 also seems to return the max resolution only ... future todo ...

Running libcamerasrc unfortunately crashes:

[nemo@PinePhone ~]$ LIBCAMERA_LOG_LEVELS=*:DEBUG gst-launch-1.0 libcamerasrc
[2:31:42.008347501] [4651] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyLinux"
[2:31:42.008503252] [4651] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyThread"
Setting pipeline to PAUSED ...
[2:31:42.011266103] [4651]  WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/lib/libcamera'
[2:31:42.011442729] [4651]  INFO Camera camera_manager.cpp:295 libcamera v0.0.0
[2:31:42.011787398] [4652] DEBUG Camera camera_manager.cpp:107 Starting camera manager
[2:31:42.015939216] [4652] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "cedrus" created from /dev/media1
[2:31:42.016087884] [4652] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media1 due to 1 missing dependencies
[2:31:42.016654929] [4652] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media1 found
[2:31:42.016778097] [4652] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media1: cedrus
[2:31:42.017205641] [4652] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "sun6i-csi" created from /dev/media0
[2:31:42.017749728] [4652] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media0 due to 1 missing dependencies
[2:31:42.018204356] [4652] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media0 found
[2:31:42.018255064] [4652] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media0: sun6i-csi
[2:31:42.018596858] [4652] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'SimplePipelineHandler'
[2:31:42.018658900] [4652] DEBUG DeviceEnumerator device_enumerator.cpp:311 Successful match for media device "sun6i-csi"
[2:31:42.018852443] [4652] DEBUG V4L2 v4l2_videodevice.cpp:573 /dev/video0[cap]: Opened device platform:csi: sun6i-video: sun6i-csi
[2:31:42.022368216] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_BE
[2:31:42.022501300] [4652] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBR ]
[2:31:42.022556383] [4652]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format RGBR
[2:31:42.022662551] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_LE
[2:31:42.022743718] [4652] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBP ]
[2:31:42.022854219] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-UYVY8_2X8
[2:31:42.023037553] [4652] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, UYVY ]
[2:31:42.023098595] [4652]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[2:31:42.023210596] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-YUYV8_2X8
[2:31:42.023386472] [4652] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, YUYV ]
[2:31:42.023443056] [4652]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[2:31:42.023550681] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SBGGR8_1X8
[2:31:42.023630474] [4652] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ BA81 ]
[2:31:42.023738683] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGRBG8_1X8
[2:31:42.023817433] [4652] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GRBG ]
[2:31:42.023928642] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGBRG8_1X8
[2:31:42.024049601] [4652] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GBRG ]
[2:31:42.024191644] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SRGGB8_1X8
[2:31:42.024274686] [4652] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGGB ]
[2:31:42.024465062] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-0x4001
[2:31:42.024539521] [4652] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [  ]
[2:31:42.024658522] [4652] DEBUG MediaDevice media_device.cpp:808 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 0
[2:31:42.218839383] [4652] ERROR MediaDevice media_device.cpp:802 /dev/media0[sun6i-csi]: Failed to setup link: No such device or address
[2:31:42.225296925] [4652] DEBUG Camera camera_manager.cpp:160 Pipeline handler "SimplePipelineHandler" matched
Pipeline is live and does not need PREROLL ...
[2:31:42.234143439] [4653] DEBUG Camera camera.cpp:754 streams configuration: (0) 2592x1944-NV21

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.271: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.271: gst_value_set_int_range_step: assertion 'start < end' failed
Setting pipeline to PLAYING ...

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.272: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.272: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.272: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.272: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.273: gst_value_set_int_range_step: assertion 'start < end' failed
New clock: GstSystemClock

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.273: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.273: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.273: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.273: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4651): GStreamer-CRITICAL **: 17:36:46.274: gst_value_set_int_range_step: assertion 'start < end' failed
[2:31:42.238309049] [4653]  INFO Camera camera.cpp:811 configuring streams: (0) 2592x1944-NV21
[2:31:42.238544759] [4652] DEBUG MediaDevice media_device.cpp:808 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 1
[2:31:42.238815052] [4652] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-YUYV8_2X8
[2:31:42.250113958] [4652] ERROR V4L2 v4l2_videodevice.cpp:1123 /dev/video0[cap]: Unable to request 3 buffers: Cannot allocate memory
[2:31:42.251468508] [4653] DEBUG Camera camera.cpp:942 Starting capture
[2:31:42.252165596] [4652] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 3 buffers requested.
[2:31:42.252327263] [4652] DEBUG V4L2 v4l2_videodevice.cpp:1349 /dev/video0[cap]: Prepared to import 3 buffers
free(): double free detected in tcache 2
Abgebrochen

with dmesg:

[ 9102.105091] cma: cma_alloc: alloc failed, req-size: 1846 pages, ret: -12
[ 9102.105150] sun6i-csi 1cb0000.csi: dma_alloc_coherent of size 7561216 failed

I'll clean up my changes and upload it somewhere for review/discussion next. Any ideas why the malloc fails?

@kbingham
Copy link

LIBCAMERA_LOG_LEVELS=*:DEBUG gst-launch-1.0 libcamerasrc

Can you use gstreamer like this? doesn't it need to connect to something?

Does qcam work?

I'll clean up my changes and upload it somewhere for review/discussion next. Any ideas why the malloc fails?

do the CMA reserved allocations need increasing somewhere?

@simonschmeisser
Copy link
Contributor Author

LIBCAMERA_LOG_LEVELS=*:DEBUG gst-launch-1.0 libcamerasrc
Can you use gstreamer like this? doesn't it need to connect to something?

I think so, but just to be save I also now tested
LIBCAMERA_LOG_LEVELS=*:DEBUG gst-launch-1.0 libcamerasrc ! jpegenc ! filesink location=test.jpg

Does qcam work?

Qt5 on SailfishOS is ancient (5.6) so we deactivated that. Will try if it actually compiles.

do the CMA reserved allocations need increasing somewhere?

Will try to google how to do that and what it means :-D

@simonschmeisser
Copy link
Contributor Author

Adding cma=64M to the kernel command line fixed the errors. I also managed to build and run qcam, it's black however ...

[root@PinePhone nemo]# LIBCAMERA_LOG_LEVELS=*:DEBUG gst-launch-1.0 libcamerasrc ! jpegenc ! filesink location=test.jpg
[3:25:02.027683928] [4930] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyLinux"
[3:25:02.027848432] [4930] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyThread"
Setting pipeline to PAUSED ...
[3:25:02.035911938] [4930]  WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/lib/libcamera'
[3:25:02.036142694] [4930]  INFO Camera camera_manager.cpp:295 libcamera v0.0.0
[3:25:02.036559705] [4931] DEBUG Camera camera_manager.cpp:107 Starting camera manager
[3:25:02.040638188] [4931] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "cedrus" created from /dev/media1
[3:25:02.040745691] [4931] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media1 due to 1 missing dependencies
[3:25:02.041304331] [4931] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media1 found
[3:25:02.041372916] [4931] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media1: cedrus
[3:25:02.041757802] [4931] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "sun6i-csi" created from /dev/media0
[3:25:02.042285982] [4931] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media0 due to 1 missing dependencies
[3:25:02.042735036] [4931] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media0 found
[3:25:02.042788454] [4931] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media0: sun6i-csi
[3:25:02.043195756] [4931] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'SimplePipelineHandler'
[3:25:02.043263675] [4931] DEBUG DeviceEnumerator device_enumerator.cpp:311 Successful match for media device "sun6i-csi"
[3:25:02.043455305] [4931] DEBUG V4L2 v4l2_videodevice.cpp:573 /dev/video0[cap]: Opened device platform:csi: sun6i-video: sun6i-csi
[3:25:02.046949314] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_BE
[3:25:02.047161945] [4931] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBR ]
[3:25:02.047225197] [4931]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format RGBR
[3:25:02.047351200] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_LE
[3:25:02.047442411] [4931] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBP ]
[3:25:02.047556581] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-UYVY8_2X8
[3:25:02.047749044] [4931] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, UYVY ]
[3:25:02.047816754] [4931]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[3:25:02.047934049] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-YUYV8_2X8
[3:25:02.048171889] [4931] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, YUYV ]
[3:25:02.048235307] [4931]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[3:25:02.048349852] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SBGGR8_1X8
[3:25:02.048432187] [4931] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ BA81 ]
[3:25:02.048541398] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGRBG8_1X8
[3:25:02.048622151] [4931] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GRBG ]
[3:25:02.048729612] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGBRG8_1X8
[3:25:02.048809322] [4931] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GBRG ]
[3:25:02.048916700] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SRGGB8_1X8
[3:25:02.048997910] [4931] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGGB ]
[3:25:02.049105163] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-0x4001
[3:25:02.049169790] [4931] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [  ]
[3:25:02.049326836] [4931] DEBUG MediaDevice media_device.cpp:808 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 0
[3:25:02.242920852] [4931] ERROR MediaDevice media_device.cpp:802 /dev/media0[sun6i-csi]: Failed to setup link: No such device or address
[3:25:02.249887037] [4931] DEBUG Camera camera_manager.cpp:160 Pipeline handler "SimplePipelineHandler" matched
Pipeline is live and does not need PREROLL ...
[3:25:02.258062712] [4932] DEBUG Camera camera.cpp:754 streams configuration: (0) 2592x1944-NV21

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.938: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.938: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.939: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.939: gst_value_set_int_range_step: assertion 'start < end' failed
Setting pipeline to PLAYING ...

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.939: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.939: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.939: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.939: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.939: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.940: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.940: gst_value_set_int_range_step: assertion 'start < end' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.940: gst_value_set_int_range_step: assertion 'start < end' failed
New clock: GstSystemClock

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.940: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.940: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.940: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.940: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.940: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.941: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.941: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.941: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.941: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.941: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.941: gst_value_subtract_int_range_int: assertion 'min < max' failed

(gst-launch-1.0:4930): GStreamer-CRITICAL **: 22:18:11.941: gst_value_subtract_int_range_int: assertion 'min < max' failed
[3:25:02.262162030] [4932]  INFO Camera camera.cpp:811 configuring streams: (0) 2592x1944-NV21
[3:25:02.262354910] [4931] DEBUG MediaDevice media_device.cpp:808 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 1
[3:25:02.262548582] [4931] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-YUYV8_2X8
[3:25:02.283783729] [4931] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 3 buffers requested.
[3:25:02.284168531] [4931] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
[3:25:02.284789797] [4932] DEBUG Camera camera.cpp:942 Starting capture
[3:25:02.284922092] [4931] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 3 buffers requested.
[3:25:02.284974844] [4931] DEBUG V4L2 v4l2_videodevice.cpp:1349 /dev/video0[cap]: Prepared to import 3 buffers
[3:25:02.285155932] [4931] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 0


pressing ctrl+c

Interrupt: Stopping pipeline ...
Execution ended after 0:01:24.323396494
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
[3:26:26.584570247] [4932] DEBUG Camera camera.cpp:974 Stopping capture
[3:26:26.584883094] [4931] DEBUG Request request.cpp:259 Request has completed - cookie: 0 [Cancelled]
[3:26:26.585085034] [4931] DEBUG V4L2 v4l2_videodevice.cpp:1364 /dev/video0[cap]: Releasing buffers
[3:26:26.585221980] [4931] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
Setting pipeline to NULL ...
[3:26:26.594712083] [4931] ERROR DeviceEnumerator device_enumerator.cpp:165 Removing media device /dev/media0 while still in use
Freeing pipeline ...
Caught SIGSEGV
#0  0xf7cac364 in ?? () from /lib/libpthread.so.0
#1  0xf7caad7e in waitpid () from /lib/libpthread.so.0
#2  0xf7d46656 in g_on_error_stack_trace () from /usr/lib/libglib-2.0.so.0
#3  0x00014882 in fault_spin () at ../tools/gst-launch.c:103
#4  fault_handler_sighandler (signum=<optimized out>)
#5  <signal handler called>
#6  0x00000000 in ?? ()
#7  0xf7a2774e in libcamera::Thread::postMessage (this=0x178ea00, 
#8  0xf7a21cfa in libcamera::Object::postMessage (this=this@entry=0xf6c03fc0, 
#9  0xf7a21d36 in libcamera::Object::deleteLater (this=0xf6c03fc0)
#10 0xf7a340c4 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release
#11 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (
#12 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::~__shared_count (
#13 std::__shared_ptr<libcamera::Camera, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr (this=0xf6201614, __in_chrg=<optimized out>)
#14 std::shared_ptr<libcamera::Camera>::~shared_ptr (this=0xf6201614, 
#15 libcamera::SimpleCameraConfiguration::~SimpleCameraConfiguration (
#16 libcamera::SimpleCameraConfiguration::~SimpleCameraConfiguration (
#17 0xf7a649fc in std::default_delete<libcamera::CameraConfiguration>::operator() (this=0x1790b10, __ptr=<optimized out>)
#18 std::unique_ptr<libcamera::CameraConfiguration, std::default_delete<libcamera::CameraConfiguration> >::~unique_ptr (this=0x1790b10, 
#19 GstLibcameraSrcState::~GstLibcameraSrcState (this=<optimized out>, 
#20 gst_libcamera_src_finalize (object=0x17ba028)
#21 0xf7ceb8cc in g_object_unref () from /usr/lib/libgobject-2.0.so.0
#22 0xf7e469f8 in gst_bin_remove_func (bin=0x17c6060, element=0x17ba028)
#23 0xf7e45df6 in gst_bin_remove (bin=bin@entry=0x17c6060, element=0x17ba028)
#24 0xf7e4604a in gst_bin_dispose (object=0x17c6060) at ../gst/gstbin.c:527
#25 0xf7ceb83e in g_object_unref () from /usr/lib/libgobject-2.0.so.0
#26 0x00013308 in main (argc=<optimized out>, argv=<optimized out>)


@pinchartl
Copy link

LIBCAMERA_LOG_LEVELS=*:DEBUG gst-launch-1.0 libcamerasrc ! jpegenc ! filesink location=test.jpg

Before you try gstreamer, I really, really advice using either the qcam or cam tools (cam would be best, it's simpler and lower-level). Otherwise you'll end up debugging kernel issues, libcamera issues and gstreamer issues all together, it will be messy. Once you can successfully capture images with the cam tool, you can move to testing with gstreamer.

@simonschmeisser
Copy link
Contributor Author

I had a look at the line
[3:25:02.242920852] [4931] ERROR MediaDevice media_device.cpp:802 /dev/media0[sun6i-csi]: Failed to setup link: No such device or address

Debugging a bit further, I got to understand that linking the main camera (ov5640) succeeds and actually only the front camera fails to link

[0:20:07.522568423] [4144] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 0
[0:20:07.716083130] [4144] ERROR MediaDevice media_device.cpp:802 /dev/media0[sun6i-csi]: Failed to setup link : gc2145 3-003c[0] -> sun6i-csi[0]: No such device or address
[0:20:07.716457873] [4144] DEBUG Camera camera_manager.cpp:160 Pipeline handler "SimplePipelineHandler" matched
Available cameras:
1: Internal front camera (/base/i2c-csi/rear-camera@4c)

Three questions:

  1. Do I need to fix the error with the gc2145 immediately or can I ignore it for the moment?

  2. linking the ov5640 actually gets a ENOIOCTLCMD for

ret = media_entity_call(link->source->entity, link_setup,
				link->source, link->sink, flags);
if (ret < 0 && ret != -ENOIOCTLCMD)
		return ret;

in mc-entity.c in __media_entity_setup_link_notify, should that be of concern?

  1. LIBCAMERA_LOG_LEVELS=*:DEBUG cam --capture=1 -c 1 -F test#.bin
    waits but does not write any files. How can I debug that further? Output:
[root@PinePhone nemo]# LIBCAMERA_LOG_LEVELS=*:DEBUG cam --capture=1 -c 1 -F test#.bin
[0:32:06.462508413] [4162] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyLinux"
[0:32:06.463016274] [4162] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyThread"
[0:32:06.463507803] [4162]  WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/lib/libcamera'
[0:32:06.463758921] [4162]  INFO Camera camera_manager.cpp:295 libcamera v0.0.0
[0:32:06.464316281] [4163] DEBUG Camera camera_manager.cpp:107 Starting camera manager
[0:32:06.471413713] [4163] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "cedrus" created from /dev/media1
[0:32:06.471624249] [4163] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media1 due to 1 missing dependencies
[0:32:06.472657595] [4163] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media1 found
[0:32:06.472784425] [4163] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media1: cedrus
[0:32:06.473352327] [4163] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "sun6i-csi" created from /dev/media0
[0:32:06.474181221] [4163] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media0 due to 1 missing dependencies
[0:32:06.475015948] [4163] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media0 found
[0:32:06.475107279] [4163] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media0: sun6i-csi
[0:32:06.475629931] [4163] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'SimplePipelineHandler'
[0:32:06.475763053] [4163] DEBUG DeviceEnumerator device_enumerator.cpp:311 Successful match for media device "sun6i-csi"
[0:32:06.476175083] [4163] DEBUG V4L2 v4l2_videodevice.cpp:573 /dev/video0[cap]: Opened device platform:csi: sun6i-video: sun6i-csi
[0:32:06.483090228] [4163] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 1
[0:32:06.483403511] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_BE
[0:32:06.483622839] [4163] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBR ]
[0:32:06.483787959] [4163]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format RGBR
[0:32:06.483980204] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_LE
[0:32:06.484214531] [4163] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBP ]
[0:32:06.484427650] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-UYVY8_2X8
[0:32:06.484793807] [4163] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, UYVY ]
[0:32:06.487285906] [4163]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[0:32:06.487496317] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-YUYV8_2X8
[0:32:06.487766351] [4163] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, YUYV ]
[0:32:06.487875182] [4163]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[0:32:06.488083759] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SBGGR8_1X8
[0:32:06.488217464] [4163] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ BA81 ]
[0:32:06.488372335] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGRBG8_1X8
[0:32:06.488487582] [4163] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GRBG ]
[0:32:06.488642786] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGBRG8_1X8
[0:32:06.488757324] [4163] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GBRG ]
[0:32:06.488913112] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SRGGB8_1X8
[0:32:06.489026942] [4163] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGGB ]
[0:32:06.489179479] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-0x4001
[0:32:06.489284727] [4163] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [  ]
[0:32:06.489509887] [4163] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 0
[0:32:06.676082104] [4163] ERROR MediaDevice media_device.cpp:802 /dev/media0[sun6i-csi]: Failed to setup link : gc2145 3-003c[0] -> sun6i-csi[0]: No such device or address
[0:32:06.676399429] [4163] DEBUG Camera camera_manager.cpp:160 Pipeline handler "SimplePipelineHandler" matched
Using camera /base/i2c-csi/rear-camera@4c
[0:32:06.679973581] [4162] DEBUG Camera camera.cpp:754 streams configuration: (0) 2592x1944-NV21
[0:32:06.680220241] [4162]  INFO Camera camera.cpp:811 configuring streams: (0) 2592x1944-NV21
[0:32:06.680456151] [4163] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 1
[0:32:06.680631855] [4163] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-YUYV8_2X8
[0:32:06.695173710] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 3 buffers requested.
[0:32:06.695574574] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
[0:32:06.696323428] [4162] DEBUG Camera camera.cpp:942 Starting capture
[0:32:06.696598546] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 3 buffers requested.
[0:32:06.696686543] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1349 /dev/video0[cap]: Prepared to import 3 buffers
Capture 1 frames
[0:32:06.696878080] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 0
[0:32:06.697021159] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 1
[0:32:06.697106657] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 2


^CExiting
[0:32:37.331881229] [4162] DEBUG Camera camera.cpp:974 Stopping capture
[0:32:37.333006199] [4163] DEBUG Request request.cpp:259 Request has completed - cookie: 0 [Cancelled]
[0:32:37.333136862] [4163] DEBUG Request request.cpp:259 Request has completed - cookie: 0 [Cancelled]
[0:32:37.333219902] [4163] DEBUG Request request.cpp:259 Request has completed - cookie: 0 [Cancelled]
[0:32:37.333340274] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1364 /dev/video0[cap]: Releasing buffers
[0:32:37.333501936] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
[0:32:37.341819508] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1364 /dev/video0[cap]: Releasing buffers
[0:32:37.342032127] [4163] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.

@simonschmeisser
Copy link
Contributor Author

I cleaned up my patches so far and uploaded them here on a fork of the megi kernel (based on 5.10rc2):
https://github.com/simonschmeisser/kernel-megi/commits/5.10-libcamera

Otherwise I'm unfortunately no further than in the comment above, ie. it tries to capture but nothing actually happens

@pinchartl
Copy link

Three questions:

1. Do I need to fix the error with the gc2145 immediately or can I ignore it for the moment?

Possibly not, but given that there's an issue here, and that you can't capture images from the other sensor, investigating this may unveil a hidden problem.

2. linking the ov5640 actually gets a `ENOIOCTLCMD` for
ret = media_entity_call(link->source->entity, link_setup,
				link->source, link->sink, flags);
if (ret < 0 && ret != -ENOIOCTLCMD)
		return ret;

in mc-entity.c in __media_entity_setup_link_notify, should that be of concern?

No, this is fine. If a driver doesn't care about links it doesn't have to implement the .link_setup() operation, in which case media_entity_call() returns -ENOIOCTLCMD. That's expected behaviour.

3. `LIBCAMERA_LOG_LEVELS=*:DEBUG cam --capture=1 -c 1 -F test#.bin`
   waits but does not write any files. How can I debug that further? Output:

I'd recommend configuring the MC pipeline manually (with media-ctl), and capturing with a V4L2 test application (yavta or v4l2-ctl) in order to ensure that the kernel drivers are operational. I suspect a misconfiguration that prevents the CSI-2 receiver from correctly receiving images. There may be debug information that the hardware can provide, but I don't have any experience with sun6i so I can't really help you there.

@simonschmeisser
Copy link
Contributor Author

After dropping some WIP changes to ov5640 I can now get it to manually capture frames using

media-ctl -d /dev/media0 --set-v4l2 '"ov5640 3-004c":0[fmt:YUYV8_2X8/1280x720]'

(already done via systemd)
followed by

v4l2-ctl --device /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=NV21

without the second command it will complain about unsupported pixelformat and/or resolution mismatch

capturing with cam or qcam still does not work and the v4l2-ctl will also not receive frames after letting libcamera configure things. But now I can compare it and will hopefully figure out the key difference.

 media-ctl -p -d /dev/media0
Media controller API version 5.10.0

Media device information
------------------------
driver          sun6i-csi
model           Allwinner Video Capture Device
serial          
bus info        platform:1cb0000.csi
hw revision     0x0
driver version  5.10.0

Device topology
- entity 1: sun6i-csi (1 pad, 2 links)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "ov5640 3-004c":0 [ENABLED]
                <- "gc2145 3-003c":0 []

- entity 5: ov5640 3-004c (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev0
        pad0: Source
                [fmt:YUYV8_2X8/1280x720@1/30 colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "sun6i-csi":0 [ENABLED]

- entity 7: gc2145 3-003c (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:YUYV8_2X8/1280x720@1/10 field:none colorspace:srgb]
                -> "sun6i-csi":0 []

v4l2-ctl --device /dev/video0 --all
Driver Info:
        Driver name      : sun6i-video
        Card type        : sun6i-csi
        Bus info         : platform:csi
        Driver version   : 5.10.0
        Capabilities     : 0xa4200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x24200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : sun6i-csi
        Model            : Allwinner Video Capture Device
        Serial           : 
        Bus info         : platform:1cb0000.csi
        Media version    : 5.10.0
        Hardware revision: 0x00000000 (0)
        Driver version   : 5.10.0
Interface Info:
        ID               : 0x03000003
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : sun6i-csi
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Sink, Must Connect
          Link 0x0200000d: from remote pad 0x1000006 of entity 'ov5640 3-004c': Data, Enabled
          Link 0x0200000f: from remote pad 0x1000008 of entity 'gc2145 3-003c': Data
Priority: 2
Video input : 0 (sun6i-csi: ok)
Format Video Capture:
        Width/Height      : 1280/720
        Pixel Format      : 'NV21' (Y/CrCb 4:2:0)
        Field             : None
        Bytes per Line    : 1920
        Size Image        : 1382400
        Colorspace        : Raw
        Transfer Function : Default (maps to None)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             : 
Streaming Parameters Video Capture:
        Capabilities     : timeperframe
        Frames per second: 30.000 (30/1)
        Read buffers     : 0


after capturing with cam

LIBCAMERA_LOG_LEVELS=*:DEBUG cam --capture=1 -s height=720,width=1280 -c 1 -F test#.bin  
[1:59:29.736767380] [4419] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyLinux"
[1:59:29.737147337] [4419] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyThread"
[1:59:29.739132120] [4419]  WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/lib/libcamera'
[1:59:29.739428244] [4419]  INFO Camera camera_manager.cpp:295 libcamera v0.0.0
[1:59:29.739940617] [4420] DEBUG Camera camera_manager.cpp:107 Starting camera manager
[1:59:29.747644334] [4420] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "cedrus" created from /dev/media1
[1:59:29.747863583] [4420] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media1 due to 1 missing dependencies
[1:59:29.749302952] [4420] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media1 found
[1:59:29.749439826] [4420] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media1: cedrus
[1:59:29.750050574] [4420] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "sun6i-csi" created from /dev/media0
[1:59:29.750932528] [4420] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media0 due to 1 missing dependencies
[1:59:29.752201773] [4420] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media0 found
[1:59:29.752322148] [4420] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media0: sun6i-csi
[1:59:29.753229144] [4420] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'SimplePipelineHandler'
[1:59:29.753384185] [4420] DEBUG DeviceEnumerator device_enumerator.cpp:311 Successful match for media device "sun6i-csi"
[1:59:30.356244173] [4420] DEBUG V4L2 v4l2_videodevice.cpp:573 /dev/video0[cap]: Opened device platform:csi: sun6i-video: sun6i-csi
[1:59:30.362972269] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_BE
[1:59:30.363217143] [4420] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBR ]
[1:59:30.363332726] [4420]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format RGBR
[1:59:30.363507600] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-RGB565_2X8_LE
[1:59:30.363633808] [4420] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGBP ]
[1:59:30.363829682] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-UYVY8_2X8
[1:59:30.364119890] [4420] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, UYVY ]
[1:59:30.364249389] [4420]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[1:59:30.364455805] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-YUYV8_2X8
[1:59:30.364684095] [4420] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, YUYV ]
[1:59:30.364801637] [4420]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[1:59:30.364987678] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SBGGR8_1X8
[1:59:30.365122510] [4420] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ BA81 ]
[1:59:30.365305343] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGRBG8_1X8
[1:59:30.365422384] [4420] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GRBG ]
[1:59:30.365593008] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SGBRG8_1X8
[1:59:30.365759133] [4420] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ GBRG ]
[1:59:30.365935424] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-SRGGB8_1X8
[1:59:30.366061256] [4420] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ RGGB ]
[1:59:30.366235339] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-0x4001
[1:59:30.366370463] [4420] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 2592x1944 in pixel formats [ JPEG ]
[1:59:30.366457671] [4420]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format JPEG
[1:59:30.480102732] [4420] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 0
[1:59:30.688075101] [4420] ERROR MediaDevice media_device.cpp:802 /dev/media0[sun6i-csi]: Failed to setup link : gc2145 3-003c[0] -> sun6i-csi[0]: No such device or address
[1:59:30.688405975] [4420] DEBUG Camera camera_manager.cpp:160 Pipeline handler "SimplePipelineHandler" matched
Using camera /base/i2c-csi/rear-camera@4c
[1:59:30.689064639] [4419] DEBUG Camera camera.cpp:754 streams configuration: (0) 2592x1944-NV21
[1:59:30.689169972] [4419] DEBUG SimplePipeline simple.cpp:471 Adjusting size from 1280x720 to 2592x1944
Camera configuration adjusted
[1:59:30.689356638] [4419]  INFO Camera camera.cpp:811 configuring streams: (0) 2592x1944-NV21
[1:59:31.388216056] [4420] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 1
[1:59:31.388548888] [4420] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 2592x1944-YUYV8_2X8
[1:59:31.420136421] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 3 buffers requested.
[1:59:31.420596711] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
[1:59:31.421191792] [4419] DEBUG Camera camera.cpp:942 Starting capture
[1:59:31.421390916] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 3 buffers requested.
[1:59:31.421481832] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1349 /dev/video0[cap]: Prepared to import 3 buffers
Capture 1 frames
[1:59:31.421697248] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 0
[1:59:31.421847705] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 1
[1:59:31.421952122] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 2
^CExiting
[2:00:17.389207266] [4419] DEBUG Camera camera.cpp:974 Stopping capture
[2:00:17.390372379] [4420] DEBUG Request request.cpp:259 Request has completed - cookie: 0 [Cancelled]
[2:00:17.390493599] [4420] DEBUG Request request.cpp:259 Request has completed - cookie: 0 [Cancelled]
[2:00:17.390576441] [4420] DEBUG Request request.cpp:259 Request has completed - cookie: 0 [Cancelled]
[2:00:17.390674408] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1364 /dev/video0[cap]: Releasing buffers
[2:00:17.390812422] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
[2:00:17.399142354] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1364 /dev/video0[cap]: Releasing buffers
[2:00:17.399350374] [4420] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
[root@PinePhone nemo]# v4l2-ctl --device /dev/video0 --all
Driver Info:
        Driver name      : sun6i-video
        Card type        : sun6i-csi
        Bus info         : platform:csi
        Driver version   : 5.10.0
        Capabilities     : 0xa4200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x24200001
                Video Capture
                Streaming
                Extended Pix Format
Media Driver Info:
        Driver name      : sun6i-csi
        Model            : Allwinner Video Capture Device
        Serial           : 
        Bus info         : platform:1cb0000.csi
        Media version    : 5.10.0
        Hardware revision: 0x00000000 (0)
        Driver version   : 5.10.0
Interface Info:
        ID               : 0x03000003
        Type             : V4L Video
Entity Info:
        ID               : 0x00000001 (1)
        Name             : sun6i-csi
        Function         : V4L2 I/O
        Pad 0x01000002   : 0: Sink, Must Connect
          Link 0x0200000d: from remote pad 0x1000006 of entity 'ov5640 3-004c': Data, Enabled
          Link 0x0200000f: from remote pad 0x1000008 of entity 'gc2145 3-003c': Data
Priority: 2
Video input : 0 (sun6i-csi: ok)
Format Video Capture:
        Width/Height      : 2592/1944
        Pixel Format      : 'NV21' (Y/CrCb 4:2:0)
        Field             : None
        Bytes per Line    : 3888
        Size Image        : 7558272
        Colorspace        : Raw
        Transfer Function : Default (maps to None)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             : 
Streaming Parameters Video Capture:
        Capabilities     : timeperframe
        Frames per second: 30.000 (30/1)
        Read buffers     : 0
[root@PinePhone nemo]# media-ctl -p -d /dev/media0        
Media controller API version 5.10.0

Media device information
------------------------
driver          sun6i-csi
model           Allwinner Video Capture Device
serial          
bus info        platform:1cb0000.csi
hw revision     0x0
driver version  5.10.0

Device topology
- entity 1: sun6i-csi (1 pad, 2 links)
            type Node subtype V4L flags 0
            device node name /dev/video0
        pad0: Sink
                <- "ov5640 3-004c":0 [ENABLED]
                <- "gc2145 3-003c":0 []

- entity 5: ov5640 3-004c (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev0
        pad0: Source
                [fmt:YUYV8_2X8/2592x1944@1/30 colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
                -> "sun6i-csi":0 [ENABLED]

- entity 7: gc2145 3-003c (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
        pad0: Source
                [fmt:YUYV8_2X8/1280x720@1/10 field:none colorspace:srgb]
                -> "sun6i-csi":0 []

Thinks I need to look at:

  • vidioc_enum_framesizes on sun6i-csi I had hardcoded to 1024x768 but that's ignored
  • vidioc_enum_framesizes on ov5640 looks somewhat twisted
  • framerate might be higher than what ov5640 can handle at full resolution?

@simonschmeisser
Copy link
Contributor Author

I limited the max resolution/frame size returned by ov5640 to 1280x720 and can now receive "data" with yavta, cam and qcam
However it's all garbage in the case of cam or qcam and all black with yavta

 LIBCAMERA_LOG_LEVELS=*:DEBUG cam --capture=5 -s height=720,width=1280 -c 1 -F test#.bin 
[3:59:30.934899709] [5521] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyLinux"
[3:59:30.935328612] [5521] DEBUG IPAProxy ipa_proxy.cpp:290 Registered proxy "IPAProxyThread"
[3:59:30.935810071] [5521]  WARN IPAManager ipa_manager.cpp:147 No IPA found in '/usr/lib/libcamera'
[3:59:30.936146034] [5521]  INFO Camera camera_manager.cpp:295 libcamera v0.0.0
[3:59:30.936726310] [5522] DEBUG Camera camera_manager.cpp:107 Starting camera manager
[3:59:30.943866048] [5522] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "cedrus" created from /dev/media1
[3:59:30.944193009] [5522] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media1 due to 1 missing dependencies
[3:59:30.945197146] [5522] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media1 found
[3:59:30.945316052] [5522] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media1: cedrus
[3:59:30.945908123] [5522] DEBUG DeviceEnumerator device_enumerator.cpp:223 New media device "sun6i-csi" created from /dev/media0
[3:59:30.946802189] [5522] DEBUG DeviceEnumerator device_enumerator_udev.cpp:95 Defer media device /dev/media0 due to 1 missing dependencies
[3:59:30.947600856] [5522] DEBUG DeviceEnumerator device_enumerator_udev.cpp:320 All dependencies for media device /dev/media0 found
[3:59:30.947702674] [5522] DEBUG DeviceEnumerator device_enumerator.cpp:251 Added device /dev/media0: sun6i-csi
[3:59:30.948290536] [5522] DEBUG Camera camera_manager.cpp:148 Found registered pipeline handler 'SimplePipelineHandler'
[3:59:30.948482670] [5522] DEBUG DeviceEnumerator device_enumerator.cpp:311 Successful match for media device "sun6i-csi"
[3:59:31.552073326] [5522] DEBUG V4L2 v4l2_videodevice.cpp:573 /dev/video0[cap]: Opened device platform:csi: sun6i-video: sun6i-csi
[3:59:31.558693797] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-RGB565_2X8_BE
[3:59:31.558935485] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1280x720 in pixel formats [ RGBR ]
[3:59:31.559052140] [5522]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format RGBR
[3:59:31.559229561] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-RGB565_2X8_LE
[3:59:31.559358428] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1280x720 in pixel formats [ RGBP ]
[3:59:31.559555604] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-UYVY8_2X8
[3:59:31.559774286] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1280x720 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, UYVY ]
[3:59:31.559886107] [5522]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[3:59:31.560162095] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-YUYV8_2X8
[3:59:31.560408576] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1280x720 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, YUYV ]
[3:59:31.560532400] [5522]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[3:59:31.560739495] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-SBGGR8_1X8
[3:59:31.560878990] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1280x720 in pixel formats [ BA81 ]
[3:59:31.561065622] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-SGRBG8_1X8
[3:59:31.561186070] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1280x720 in pixel formats [ GRBG ]
[3:59:31.561364575] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-SGBRG8_1X8
[3:59:31.561481397] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1280x720 in pixel formats [ GBRG ]
[3:59:31.561655442] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-SRGGB8_1X8
[3:59:31.561770722] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1280x720 in pixel formats [ RGGB ]
[3:59:31.561945351] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-0x4001
[3:59:31.562074759] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1280x720 in pixel formats [ JPEG ]
[3:59:31.562164199] [5522]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format JPEG
[3:59:31.672153546] [5522] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 0
[3:59:31.805973130] [5522] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: gc2145 3-003c[0] -> sun6i-csi[0]: 1
[3:59:31.806176266] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'gc2145 3-003c':0 -> 'sun6i-csi':0 configured with format 1600x1200-RGB565_2X8_LE
[3:59:31.806312927] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1600x1200 in pixel formats [ RGBP ]
[3:59:31.806407535] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'gc2145 3-003c':0 -> 'sun6i-csi':0 configured with format 1600x1200-UYVY8_2X8
[3:59:31.810793508] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1600x1200 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, UYVY ]
[3:59:31.815456137] [5522]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[3:59:31.815560206] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'gc2145 3-003c':0 -> 'sun6i-csi':0 configured with format 1600x1200-VYUY8_2X8
[3:59:31.815731000] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1600x1200 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, VYUY ]
[3:59:31.815815147] [5522]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[3:59:31.815892584] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'gc2145 3-003c':0 -> 'sun6i-csi':0 configured with format 1600x1200-YUYV8_2X8
[3:59:31.819089457] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1600x1200 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, YUYV ]
[3:59:31.819212531] [5522]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[3:59:31.819293677] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'gc2145 3-003c':0 -> 'sun6i-csi':0 configured with format 1600x1200-YVYU8_2X8
[3:59:31.819446258] [5522] DEBUG SimplePipeline simple.cpp:283 Adding configuration for 1600x1200 in pixel formats [ NV21, NV61, HM12, YU12, NV12, YV12, NV16, 422P, YVYU ]
[3:59:31.819525779] [5522]  WARN V4L2 v4l2_pixelformat.cpp:179 Unsupported V4L2 pixel format HM12
[3:59:31.819651645] [5522] DEBUG Camera camera_manager.cpp:160 Pipeline handler "SimplePipelineHandler" matched
Using camera /base/i2c-csi/rear-camera@4c
[3:59:31.820382627] [5521] DEBUG Camera camera.cpp:754 streams configuration: (0) 1280x720-NV21
[3:59:31.820596974] [5521]  INFO Camera camera.cpp:811 configuring streams: (0) 1280x720-NV21
[3:59:31.928118054] [5522] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: gc2145 3-003c[0] -> sun6i-csi[0]: 0
[3:59:32.516911554] [5522] DEBUG MediaDevice media_device.cpp:812 /dev/media0[sun6i-csi]: ov5640 3-004c[0] -> sun6i-csi[0]: 1
[3:59:32.517249934] [5522] DEBUG SimplePipeline simple.cpp:415 Link 'ov5640 3-004c':0 -> 'sun6i-csi':0 configured with format 1280x720-YUYV8_2X8
[3:59:32.523034563] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 3 buffers requested.
[3:59:32.523527441] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
[3:59:32.524201658] [5521] DEBUG Camera camera.cpp:942 Starting capture
[3:59:32.524467477] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 3 buffers requested.
[3:59:32.524589384] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1349 /dev/video0[cap]: Prepared to import 3 buffers
Capture 5 frames
[3:59:32.524838574] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 0
[3:59:32.525225966] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 1
[3:59:32.525367295] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 2
[3:59:32.526810336] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1507 /dev/video0[cap]: Dequeuing buffer 0
[3:59:32.530747735] [5522] DEBUG Request request.cpp:259 Request has completed - cookie: 0
14372.526697 (0.00 fps) stream0 seq: 000000 bytesused: 1382400
[3:59:32.574826575] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 0
[3:59:32.575060011] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1507 /dev/video0[cap]: Dequeuing buffer 1
[3:59:32.575134947] [5522] DEBUG Request request.cpp:259 Request has completed - cookie: 0
14372.560789 (29.33 fps) stream0 seq: 000001 bytesused: 1382400
[3:59:32.615565795] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 1
[3:59:32.626619659] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1507 /dev/video0[cap]: Dequeuing buffer 2
[3:59:32.626775783] [5522] DEBUG Request request.cpp:259 Request has completed - cookie: 0
14372.626492 (15.22 fps) stream0 seq: 000003 bytesused: 1382400
[3:59:32.651769674] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 2
[3:59:32.659601536] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1507 /dev/video0[cap]: Dequeuing buffer 0
[3:59:32.659753117] [5522] DEBUG Request request.cpp:259 Request has completed - cookie: 0
14372.659434 (30.36 fps) stream0 seq: 000004 bytesused: 1382400
[3:59:32.682958879] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1438 /dev/video0[cap]: Queueing buffer 0
[3:59:32.692611879] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1507 /dev/video0[cap]: Dequeuing buffer 1
[3:59:32.692838229] [5522] DEBUG Request request.cpp:259 Request has completed - cookie: 0
14372.692391 (30.34 fps) stream0 seq: 000005 bytesused: 1382400
[3:59:32.718560309] [5521] DEBUG Camera camera.cpp:974 Stopping capture
[3:59:32.719481506] [5522] DEBUG Request request.cpp:259 Request has completed - cookie: 0 [Cancelled]
[3:59:32.719609790] [5522] DEBUG Request request.cpp:259 Request has completed - cookie: 0 [Cancelled]
[3:59:32.719684267] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1364 /dev/video0[cap]: Releasing buffers
[3:59:32.719779959] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.
[3:59:32.721613684] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1364 /dev/video0[cap]: Releasing buffers
[3:59:32.721792438] [5522] DEBUG V4L2 v4l2_videodevice.cpp:1136 /dev/video0[cap]: 0 buffers requested.

[nemo@PinePhone yavta]$ ./yavta --capture=3 /dev/video0 -F test#.bin
Device /dev/video0 opened.
Device `sun6i-csi' on `platform:csi' (driver 'sun6i-video') supports video, capture, without mplanes.
Video format: NV21 (3132564e) 1280x720 (stride 1920) field none buffer size 1382400
8 buffers requested.
length: 1382400 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xf7af7000.
length: 1382400 offset: 1384448 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xf79a5000.
length: 1382400 offset: 2768896 timestamp type/source: mono/EoF
Buffer 2/0 mapped at address 0xf7853000.
length: 1382400 offset: 4153344 timestamp type/source: mono/EoF
Buffer 3/0 mapped at address 0xf7701000.
length: 1382400 offset: 5537792 timestamp type/source: mono/EoF
Buffer 4/0 mapped at address 0xf75af000.
length: 1382400 offset: 6922240 timestamp type/source: mono/EoF
Buffer 5/0 mapped at address 0xf745d000.
length: 1382400 offset: 8306688 timestamp type/source: mono/EoF
Buffer 6/0 mapped at address 0xf730b000.
length: 1382400 offset: 9691136 timestamp type/source: mono/EoF
Buffer 7/0 mapped at address 0xf71b9000.
0 (0) [-] none 0 1382400 B 14971.962335 14971.962365 3571.429 fps ts mono/EoF
1 (1) [-] none 1 1382400 B 14971.995956 14971.996567 29.743 fps ts mono/EoF
2 (2) [-] none 2 1382400 B 14972.029125 14972.069301 30.149 fps ts mono/EoF
Captured 3 frames in 0.107245 seconds (27.973159 fps, 38670095.579251 B/s).
8 buffers released.


I used imagemagick to convert: convert -size 1280x720 -depth 8 -sampling-factor 4:2:0 yuv:frame-000002.bin -colorspace rgb frame05.png

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants