New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Camera support #91
Comments
I agree |
Right, makes sense to track this here. |
Copying my old comment here: I found libcamera based on this bug report, it seems to have the required userspace code to have ipu3 working on 5.0. libcamera FOSDEM slides |
Unlikely to get anywhere without someone stepping up and putting some work into it though. I was hoping to work on the cameras before buying my Surface Laptop 3, but since the newest generation's cameras are available via USB it just worked out of the box -- I don't have hardware to develop against. I think @GrayHatter was taking a look at things a while ago. |
Also Surface go has not support for camera. It helps to me a lot because I use Surface Go basically for working outside the office and, sometime I have to do some Skype videocall... And I should do in my mobile. |
Well, I managed to find datasheets for the two missing drivers at least. Never done kernel dev, but I mean, how hard could it be? (sarcasm). |
I wish you the best of luck! For reference, here are the (I think) most relevant links from the other discussion:
Getting started with the ovXXXX drivers is probably the best idea. |
@Doridian Feel free to message us on IRC (freenode @ ##linux-surface) or the Matrix bridge (might be easier to join), we can always help out! I also recommend starting with #91 (comment) and doing the OV drivers first. Once we have those we can figure out how to integrate with ipu3/libcamera. I was searching for comparable drivers by looking at the product brief's functional diagram. I see that you also found the datasheet for both which is pretty great. |
@Doridian highlight me with nearly anything you think is relevant, I'm currently working on getting it working, but haven't had the spare cycles lately. That's awesome to find the datasheets for the cameras, I'm pouring through them now. Currently, my understanding, and thus next step, is to figure out what magic windows is using to load the cameras after boot, the DSDT is incomplete on the SB2. I originally thought you'd have to load a custom DSDT at boot up, but https://www.kernel.org/doc/html/v5.2/firmware-guide/acpi/method-customizing.html seems to imply we can do what windows does and inject our fixes at runtime. Then IIRC we write an i2c driver for the cameras to control them, and then wire them up to ipu3, like @archseer said It's all been done before, so it shouldn't be too much actual work, once we find the right magic to apply. |
@GrayHatter Good ideas. I'm currently just poking around the Windows driver package. There seems to be quite a bit of hardcoding some things going on in there... |
Based on the DSDT it looks like |
@qzed Ah, makes sense. Is there and HWIDs for the ipu3 interface or other things I could try searching for in the driver to see how it fetches those devices? |
I haven't really figured out much more than that. There are also the |
What's the approach like to get the camera working on the Surface Go? I ain't no pro but is there some kind of support that I can provide? Also the readme mentions partial support - any idea how it's enabled? |
Happy to fund this if someone wants to get it resolved speedily... |
The process on the Go isn't any different than for the other devices, except that it uses a different sensor (OV2740). |
@olantrust IssueHunt or Bountysource are all fine. IssueHunt seems to do some reward matching on their end as well to increase the bounty. |
I've copied the Now I see the following error message
but at least the |
Great. |
$75 Dollar bounty provided: |
Contributed! |
@PhilDevProg https://src.fedoraproject.org/rpms/libcamera/blob/rawhide/f/libcamera.spec doesn't seem to mention it so I guess it's not packaged by Fedora. I'll ask for it to be included. |
@kbingham I tried building libcamera but I get this error:
This is the output of
|
@PhilDevProg You're building with GCC-13 which needs fixes ... that are about to be merged in the release I'm trying to make.
|
@kbingham Thanks! Your branch works |
For building with meson you might want to add |
How can i capture an IR image on the surface pro 6 again? I've done it before, but I can't find the script anymore |
@djrscally Since the image quality of the cameras (especially the rear camera) isn't really good at the moment, I have a question: Is there a way to improve the image quality? Maybe via some configs in libcamera? Or will it have to stay this way forever since it's just not possible on Linux without the optimizations done by Microsoft? |
The quality can be improved. It requires someone to do work on tuning and possibly improvements to the algorithms. I had started exploring adding lens shading calibration to the IPU3 quite some time ago but I haven't ever got round to continuing it. If someone's interested, that's a good starting point for exploring the algorithm development on the IPU3. There's some early development I've just rebased here https://github.com/kbingham/libcamera/tree/kbingham/ipu3/lsc that might be interesting to play with - but it's a complete WIP development branch - don't expect it to work out of the box. |
I still have issue #906 with debian bookworm. imo there are two issues here:
This leads to an error, naturally. Symlinking
Please consider linking against the newer libssl so the module can be signed. |
@werdahias Unfortunately, it's not that easy. We build our kernels on Ubuntu 20.04, so any library versions required for the kernel's user-space tooling are based on that. If we would switch to something newer, we would have problems with DKMS not being usable at all (see e.g. here). The long-term solution would be to compile the kernel images for each distro specifically, like we already do for Fedora. |
I juat put in to say that I appreciate your work and understand that supporting newer gcc compilers and libraries must be a pain. For that reason I want to confirm that Ubuntu 22.04 with 6.2.14-surface kernel (installed via extrepo) has got both cameras working either in chrome and in firefox via V4L loopback. I just had to compile libcamera from source and install it and place the ipu firmware to the intel folder. Please note that I don't need any longer to add quees to let the gstreaming go, as you may see in my script so it's definitely a step forward, comparing to 6 months ago (when running Ubuntu 21).
Let's say that enabling cameras is slowly becoming easy. Thanks. Luca |
Getting there! - Once Firefox 114 is released you can also use the cameras I believe with just I'm still awaiting full integration with pipewire for true seamless integration with both chrome/chromium and firefox though. |
@qzed thanks, that what I'd feared. Appreciate all you guys did. Looks like I can't use cameras for the time being :( |
You could try modifying the sign_helper script to call the Or disable secureboot and don't sign the DKMS modules at all. |
Hello, I'm trying to follow the instructions in the guide to create the loopback device and use the camera in other applications. When running the command |
I would guess that your build/configuration has either not built the gstreamer component or not installed it at the location gstreamer expects to find it. Check the output when you run meson setup, to verify gstlibcamerarc is being built and check if you needto set -Dprefix=/usr or such to make sure you install on paths that gstreamer will look for plugins. |
The capture plugin gstreamer looks for should be here
|
I just experienced something very weird on my Go 3 (I'm on Fedora): In Qcam I can switch between the front and back camera without any issues but in Cheese, I get an error message and have to reopen Cheese every time I change between the front and back camera... This wasn't an issue before. Snapshot has a similar issue: It opens and I can use the front camera but if I want to switch to the back camera, I get an error message. Update: It works now again. |
I'm getting this now too; it looks like a format negotiation error. I'll try to get to the bottom of it, but I'm a bit busy over the next week so might be a while. Bit of news, as Kieran mentioned integration between Pipewire and Chrome/Firefox through WebRTC has been in the works for a while. As of about a week ago the support for WebRTC in Firefox is in Nightly. I tried it (which on my Ubuntu 22.04 meant first getting mainline pipewire for the libcamera support) and it worked with some fiddling. So not quite there, but extremely close. |
Wow - I didn't think Firefox would win that race - I was expecting Chromium to be first - but that's great news - finally 'hack/workaround' free video calling on IPU3 without performance impacts from copying through v4l2-loopbacks and software format conversions! \o/ |
Hi @kbingham! I'm trying to programmatically set exposure time on the IPU3 SB2 rear camera, but trying |
@lukexi The Surface cameras and IPU3 remain 'untuned' - and there's not been a lot of development gone on with the algorithms there. Someone with a keen interest and device needs to work through and look at the tuning processes for the surface cameras and I suspect the manual controls aren't yet implemented in the IPA for the IPU3 - so that also needs exploration. |
Thanks much Kieran! I'll poke around and see if I can figure out improvements with the hardcoded parameters in the IPU3 IPA |
Hi all, do we still need to compile libcamera from source on Jammy (22.04) or can we use apt: I would like to try the "libamerify firefox" thing... Also - was there any luck with IR camera and Howdy tooling? Thanks! |
Hey everyone, the latest build of Firefox (116) introduced experimental support for libcamera over pipewire, and I am glad to report that it works almost out of the box :) After a quick test, the back camera works, the front camera is pitch black (but works on Qcam), and switching cameras crashes Firefox. Far from perfect but still, a huge improvement :) |
Sadly, the |
It is only a plugin in the sense that it is a loadable module. It is built together with pipewire, and Arch does have it enabled in their package. Fedora just decided to put the resulting files into a subpackage so that not every install has to pull in libcamera. |
So do I have to install because I enabled the option in Firefox(116), no camera were found. |
Judging from https://archlinux.org/packages/extra/x86_64/pipewire/ the official package already enables it. |
OK, I tried it. Firefox instantly crashed when I gave camera access permission.
|
I see that there is some activity here: jakeday/linux-surface#145
I think it's better to trace it here isn't it ?
Best,
Jerome
Maintainer note: There is a BountySource bounty available for this issue. If you'd like to see a camera driver consider donating to incentivise the work.
The text was updated successfully, but these errors were encountered: