Please upstream modules into mainline kernel

[bk2204]

I recently discovered that my ThinkPad X1 Carbon needs an out-of-tree driver (this one) for my camera. That seems atypical for Intel hardware, which is typically well supported upstream.

I've tried with both Debian sid's 5.18.0-2-amd64 and Ubuntu jammy's 5.15, but the driver is included in neither distro, probably because it isn't upstream. Since this hardware is considered important for most people who use an Alder Lake laptop, could you consider upstreaming this driver (and any dependent pieces) into the mainline kernel relatively soon, even if only in the staging area?

I did try to build this myself with DKMS, but due to #13 (both the missing header and then the modpost error), it's not possible to do so, so there is presently no way to make this hardware work on Debian.

[dennis-pries]

Fedora (and I think a lot of other distos) also will not include them, if not upstreamed.

Please do it.

[wbolster]

the new Dell XPS 13 Plus (9320) also requires this driver for the webcam, but building it fails on linux kernel 5.18.7 (arch linux) due to the issues in #13 (missing headers, and unresolved symbols after manually copying headers)

[naps62]

I also have a Dell XPS 13 plus, and just stumbled upon this issue. Also failed to build.
I'm not very profficient in kernel module compilation. Is there a way to get this working at the moment? I'm using Manjaro

[K-MeLeOn]

Dell XPS 13 Plus too, building error on modpost with message 'undefined!' on ov02c10.ko and ov01a10.ko

[paulmenzel]

I notified the Linux upstream folks about the issue. The current recommendation is to not buy such devices. From Greg KH:

On Thu, Aug 11, 2022 at 04:54:53PM +0300, Laurent Pinchart wrote:

For the time being, I agree with your recommendation to not buy these
devices if you care about camera support.

I second this, don't buy these devices if the vendor is not willing to
get their drivers upstreamed properly.

It also sounds like, that it won’t be easily be solved.

Those of you who can, I can only suggest to return the device to not further support vendors screwing up Linux support in this case, and save time fiddling with Linux driver issues.

No idea, who dropped the ball here, but the ChromiumOS/ChromeOS and Ubuntu/Canonical/Dell/Lenovo should have started the discussion with the upstream folks much earlier. It’s a very sad situation for the GNU/Linux ecosystem.

[kbingham]

We can certainly help over at https://libcamera.org if anyone from Intel wants to contact us.

[pinchartl]

@paulmenzel I share your disappointment, but I'd like to focus on how we could improve the situation instead of generating bad PR, as I'm sure the latter doesn't need my help :-)

I won't comment on what happened at Google, Canonical, Dell or Lenovo, as only them have a full picture. All I can say is that it's fairly common for discussions to start in private to decide how to support new product, especially before they are released. I have no reason to believe those discussions didn't start, but they obviously didn't bear fruits so far. The problem isn't easy to solve, and I do understand how development teams can be pressured with product deadlines. Still, this may explain the situation, but not excuse it.

Another point I'd like to make is that we have received technical support from multiple Intel developers when developing IPU3 support in libcamera. There are people with goodwill in their teams, I don't think it would be fair blaming all the developers for the lack of a high-level upstreaming strategy.

We won't be able to fix the mistakes that were made without a clear commitment from Intel, they need to engage directly with the Linux kernel and libcamera upstream developers. Given the amount of work that lies ahead, this has to happen now. My door is fully open.

[Thesola10]

hey, in the mean time I managed to create a patch for pure DKMS building in my AUR package intel-ipu6-dkms-git

[Vascro]

Doesn't appear there are any new updates regarding this ipu6 mipi camera issue. I just got my new XPS 9320 in the mail and noticed this issue immediately. Hopefully Intel fixes this ASAP.

[paulmenzel]

I can only recommend to return the system to not support such products. Otherwise vendors won’t change as long as they can sell such devices.

Additionally, Jonathan Corbet summarized the topic in the LWN article The growing image-processor unpleasantness.

[naps62]

I'm also stuck with a Dell XPS 13 with this issue. don't have a chance to return it unfortunately, although I do agree returning sends the more forceful message to vendors.

FYI @Thesola10 I tried your kernel module (although I'm not 100% sure what it would achieve) on my Manjaro installation. The boot hanged for a few minutes but ended up booting in the end. Stilll no working though, so I removed it again to fix the boot process

[Thesola10]

The lengthened boot process is due to including the intel_vsc drivers into the initramfs. Those can be excluded and they'll load after boot. Otherwise the driver works with icamerasrc on GStreamer so you can try installing that as well

[cjechlitschek]

This might be already known, but I still want to mention it. Canonical provides prebuilt packages for some Dell systems that enable the camera: https://wiki.ubuntu.com/Dell

[netllama]

@cjechlitschek those packages are completely useless if someone isn't running a Debian based distro. Also, see the previous comments which explain why those packages are flawed.

[corebots]

I also miss the webcam under Fedora, pleas fix who can!

[githubtefo]

This is crazy, I just bought the XPS 13 Plus because it was certified by Ubuntu 😩 and even under a Debian based distro, the drivers are not working on 5.17.0-1018-oem kernel.

[naps62]

@githubtefo For what it's worth, I've been running Ubuntu smoothly for a while now, with snapd and all the bloatware removed
I had already bought the laptop, after a 3-month wait, and shipped from US to Europe. so it's not like I had much of a choice

will happily switch back to Arch once possible. But all things considered, I'm quite happy with the performance of the laptop, including the webcam

[karolszk]

Hello, I would like also point out the need for camera drivers for the newest Lenovo Carbon X10 laptop. For 6 version of Linux kernel where are compilable and still lacking in the upstream.

[simisimis]

for my new job I was picking from Thinkpad X1 gen10 and Dell XPS Plus. Was afraid that I made a wrong choice. But from what you wrote I see I would have ended up in this thread regardless 😅

[corebots]

Hi, I've picked up the XPS 13 Plus 5 months ago, been using it with Fedora since then. Mostly good, except the webcam still not working after all that time :(

Please add the webcam drivers to the mainline kernel since it's a bummer needing to use an external usb webcam with such a great and expensive machine. Thanks!

[dmipx]

We are from RealSense working on a "webcam" support for IPU6.
This will support MIPI connected camera on IPU6 to act as "USB UVC", provide access to camera features through video node.
First of all, we want to support RealSense D457 MIPI camera on IPU6 platform.

The work is in progress - we estimate work done by endo of the year, you can observe PR's update.
#73
My repo with updates:
https://github.com/dmipx/ipu6-drivers/tree/iotg_ipu6_isys_video_ctrl

[netllama]

@dmipx that sounds like great news. Hopefully you can improve the README as well, so that there are clear build instructions, instead of the current outdated confusing mess.

[dmipx]

@netllama AFAIK the iotg_ipu6 branch is heavily developed so you can sync with it and see if there is any issue with build steps.
I'm not sure about all supported camera drivers but i know that IPU6 driver can be built without issues.

[pinchartl]

We are from RealSense working on a "webcam" support for IPU6. This will support MIPI connected camera on IPU6 to act as "USB UVC", provide access to camera features through video node. First of all, we want to support RealSense D457 MIPI camera on IPU6 platform.

The work is in progress - we estimate work done by endo of the year, you can observe PR's update. #73 My repo with updates: https://github.com/dmipx/ipu6-drivers/tree/iotg_ipu6_isys_video_ctrl

Are you planning to upstream this work, or just merge it in the out-of-tree IPU6 driver ?

[dmipx]

For now we have dispute about it with ipu6 team so we started to implement it as out-of-tree.
RealSense, as usual, will provide debian package for ubuntu distros with DKMS supporting d457 cameras.
If it will be demanded i will try to support laptops mipi cameras but for now - I have no hardware to work on it. Let's see how it goes.
In long term we want to upstream but this should be community demand.

[paulmenzel]

In long term we want to upstream but this should be community demand.

Please contact the Linux (subsystem) kernel mailing list with the involved developers in Cc notifying them about your efforts. This GitHub issue is not the right forum for this.

[MorganMLGman]

Hi, any updates regarding this topic? Is it possible to have this included in mainline kernel or people with alder lake laptops will be left with not working webcam on linux?

[chromer030]

Please upstream the module , people are suffering from lack of support.

[SmokinCaterpillar]

Hey is there any update? I bought a new X1Gen10 and, working in a remote setting without a camera, this machine is a bit of a very expensive paperweight.

[DemiMarie]

That sounds technically doable. The end result should provide images of similar quality as a cheap webcam, which is better than nothing.

Why can the PSYS produce better quality images? Is it because the PSYS has access to data that the CPU does not? Or would achieving equivalent quality be prohibitively expensive?

I believe there's some effort on a proof-of-concept, using the CPU to process data. I would then like to see how it could be accelerated with the GPU.

GPU acceleration would be great, but it should be optional.

[pinchartl]

Because it would be prohibitively expensive to apply the same level of processing using the CPU, or even the GPU.

[DemiMarie]

Would it be possible to use PCI passthrough to assign the ISYS and PSYS to a VM? Are the ISYS and PSYS in an IOMMU group with any other devices, except perhaps each other? Could firmware loading and authentication be done on the host before the device is assigned to the VM? Obviously the VM has no access to the CSE, so it cannot request firmware authentication itself.

[pinchartl]

For the PSYS, perhaps, I don't know. For the ISYS, likely not. You would need to also assign to the VM the I2C controllers for the buses on which the sensors are connected, and there could be other devices on those buses. Even if this could be done, you wouldn't be able in the VM to power the camera sensors (and other ancillary devices such as lens controllers) up and down, as that's handled by ACPI.

[jwrdegoede]

@netllama

After reading through most of the old comments in this issue, I discovered that the problem was that v4l2-relayd.service had failed (at boot?). After restarting it manually, the problem is fixed. I checked the logs, and the service was failing to start with the following output:

Hmm, so the grep: /sys/devices/virtual/video4linux/*/name: No such file or directory error suggests that the v4l2loopback kernel module has not loaded yet.

I guess this might hit on the first boot of a new kernel when rebooting directly after installing a kernel-update and thus before akmod has build the v4l2loopback module. Then the module will be build on boot which I guess may cause this.

Are you still seeing this?

[netllama]

yes, that is exactly the scenario !

…

On November 22, 2023 4:51:56 PM GMT+01:00, Hans de Goede ***@***.***> wrote: @netllama > After reading through most of the old comments in this issue, I discovered that the problem was that v4l2-relayd.service had failed (at boot?). After restarting it manually, the problem is fixed. I checked the logs, and the service was failing to start with the following output: Hmm, so the `grep: /sys/devices/virtual/video4linux/*/name: No such file or directory` error suggests that the v4l2loopback kernel module has not loaded yet. I guess this might hit on the first boot of a new kernel when rebooting directly after installing a kernel-update and thus before akmod has build the v4l2loopback module. Then the module will be build on boot which I guess may cause this.

[jwrdegoede]

yes, that is exactly the scenario !

Ok, there is not much which can be done about this. akmod will try to build the modules for the new kernel in the background after installing the new kernel. But if you reboot before akmod has completed the build then the module will be build at boot triggering this.

I believe that dnf / akmod even inhibits normal reboots through the UI while the modules are being build. But if you do "sudo reboot" from a terminal that does not honor the inhibit and then you may hit this.

[SmokinCaterpillar]

Small question / reminder, first post of this issue was on Jun 21, 2022 that is now over a year and a half ago. What has happened in the meantime? Any update from intel?

[arpad-m]

There is ongoing upstreaming efforts for various parts of the webcam driver stack:

• https://lore.kernel.org/linux-media/20240111065531.2418836-1-bingbu.cao@intel.com/
• https://lore.kernel.org/linux-media/20231204123947.5754-1-hdegoede@redhat.com/
• https://lists.libcamera.org/pipermail/libcamera-devel/2024-January/040113.html

jwrdegoede will also have a talk at fosdem about it. See also their recent blog post.

[paulmenzel]

Thank you Red Hat and Linaro, and, therefore, to all organizations buying RHEL licenses and not being parasites by using CentOS or other clones. Unfortunately that’s often banks and insurance companies. I am curious, if Dell or Google ChromeOS participated in this effort.

Anyway, big kudos again to Red Hat (@jwrdegoede) and Linaro.

[pinchartl]

I have good hopes that the ISYS driver will land upstream in a reasonably near future.

The PSYS is a different story. It seems much more likely to me that IPU6-based platforms will be supported by libcamera using a software-based ISP (possibly GPU-accelerated at some point). That will incur some performance and power consumption costs, but should provide decent-enough images for basic video conferencing needs.

[DemiMarie]

I have good hopes that the ISYS driver will land upstream in a reasonably near future.

The PSYS is a different story. It seems much more likely to me that IPU6-based platforms will be supported by libcamera using a software-based ISP (possibly GPU-accelerated at some point). That will incur some performance and power consumption costs, but should provide decent-enough images for basic video conferencing needs.

What is the reason for this? Is it because Linux has no framework for PSYS drivers, or because Intel is not willing to provide a driver or documentation, and therefore extensive reverse engineering would be required?

[pinchartl]

Only Intel can officially answer that question. You may be able to form your personal opinion on the matter by looking at how much information the existing PSYS driver exposes (or doesn't expose) about the parameters the ISP receives, the statistics it produces, and how it operates in general.

[DemiMarie]

Ah, so this is a situation where the actual driver runs in userspace, with the kernel driver just being a pipe between userspace and the on-device firmware.

[paulmenzel]

Slide 7 of the FOSDEM 2024 presentation slides says:

Reluctance/refusal of some vendors to disclose “secret sauce”

Doesn’t that contradict Intel’s pledge to openness in 2021?

[DemiMarie]

I have good hopes that the ISYS driver will land upstream in a reasonably near future.

The PSYS is a different story. It seems much more likely to me that IPU6-based platforms will be supported by libcamera using a software-based ISP (possibly GPU-accelerated at some point). That will incur some performance and power consumption costs, but should provide decent-enough images for basic video conferencing needs.

Does that mean that for those who need high-quality images or good battery life, an external high-end USB webcam will be needed for the forseeable future?

[pinchartl]

For those who need high-quality images and good battery life with upstream drivers, yes. That is my understanding, but I would be very happy if Intel proved me wrong :-)

[acegallagher]

See here for discussion and status on kernel mailing list:

https://lwn.net/ml/linux-media/008f30be-7d1b-498c-88fa-d8cf061e19fa@redhat.com/#t

[oblique]

The v6 of the patch was posted for review two days ago: https://lwn.net/ml/linux-media/20240426095822.946453-1-sakari.ailus@linux.intel.com/

[arpad-m]

Also, v8 of the softisp patch has been posted recently https://lists.libcamera.org/pipermail/libcamera-devel/2024-April/041448.html

the patches I've linked above have made it into Linux 6.8.

[chromer030]

the driver upstreamed in linux 6.10.

https://lore.kernel.org/lkml/20240516080159.76e8b45d@sal.lan/

EDIT: relative commit which contains codes & documentation about how driver works:

torvalds/linux@6fd600d

[prochac]

the driver upstreamed in linux 6.10.

https://lore.kernel.org/lkml/20240516080159.76e8b45d@sal.lan/

My for-sure-last Intel-based NTB will have a working webcam in 2024, yay.

[pinchartl]

the driver upstreamed in linux 6.10.
https://lore.kernel.org/lkml/20240516080159.76e8b45d@sal.lan/

While it's an important part of the puzzle, the ISYS driver alone isn't enough. The PSYS driver is still missing, and that will take more time. In the meantime, libcamera has a software-based ISP implementation (contributed by Redhat and Linaro, kudos to them) that will be usable as a stop-gap measure.

My for-sure-last Intel-based NTB will have a working webcam in 2024, yay.

You will also need a driver for the camera sensor in your device. With some luck, there will already be one in the kernel. It may also need integration in libcamera, if not done yet.

TL;DR: Avoiding the extra CPU and power consumption of the software ISP will have to wait for next year, but the good news is that there's a chance you'll get images out of your camera in 2024 :-)

[DemiMarie]

the driver upstreamed in linux 6.10.
https://lore.kernel.org/lkml/20240516080159.76e8b45d@sal.lan/

While it's an important part of the puzzle, the ISYS driver alone isn't enough. The PSYS driver is still missing, and that will take more time.

Does this mean that a PSYS driver is at least planned? I thought it was completely blocked by Intel not being willing to release any userspace for the PSYS and upstream (rightly) refusing to take a driver with no working userspace.

(...)

My for-sure-last Intel-based NTB will have a working webcam in 2024, yay.

You will also need a driver for the camera sensor in your device. With some luck, there will already be one in the kernel. It may also need integration in libcamera, if not done yet.

Does Linux have drivers for the sensors most commonly found in laptops?

[pinchartl]

Does this mean that a PSYS driver is at least planned? I thought it was completely blocked by Intel not being willing to release any userspace for the PSYS and upstream (rightly) refusing to take a driver with no working userspace.

I'm not sure I would say "planned". The effort isn't dead, but the relevant upstream kernel communities have not blessed any architecture and technical solution yet. Whether or not a driver could land upstream, and at what time, are not questions that can be answered with a complete guarantee at this point.

(...)

My for-sure-last Intel-based NTB will have a working webcam in 2024, yay.

You will also need a driver for the camera sensor in your device. With some luck, there will already be one in the kernel. It may also need integration in libcamera, if not done yet.

Does Linux have drivers for the sensors most commonly found in laptops?

Some of them are supported. Intel has upstreamed sensor drivers in the past few of years, so I would say they have done work in that area, but I don't know what shares of the Intel-based laptops that would cover, or what models.

[DemiMarie]

Does this mean that a PSYS driver is at least planned? I thought it was completely blocked by Intel not being willing to release any userspace for the PSYS and upstream (rightly) refusing to take a driver with no working userspace.

I'm not sure I would say "planned". The effort isn't dead, but the relevant upstream kernel communities have not blessed any architecture and technical solution yet. Whether or not a driver could land upstream, and at what time, are not questions that can be answered with a complete guarantee at this point.

I know that drivers for other ISPs have been accepted upstream. Is the lack of an open userspace for the PSYS still the main blocker, or are there other problems that also need to be resolved?

[Vascro]

the driver upstreamed in linux 6.10.

https://lore.kernel.org/lkml/20240516080159.76e8b45d@sal.lan/

EDIT: relative commit which contains codes & documentation about how driver works:

torvalds/linux@6fd600d

Is there any talk of backporting the driver to 6.9? I'm aware that the PSYS driver would still be needed, but with 6.9 being so new, it makes sense to me to try supporting it.