Feature Request: libcamera support

[rothn]

Is your feature request related to a problem? Please describe.
Waydroid currently supports mainline Linux cameras by passing low-level V4L2 camera devices through to Android, which sets them up as extremely limited external webcams. This is not ideal, e.g. for Android apps that use the front-facing camera to capture QR codes.

Describe the solution you'd like

• Include libcamera as an optional dependency
• Install the compiled libcamera Android HAL in the container
• Set "ro.hardware.camera=<libcamera-so-path>" instead of "ro.hardware.camera=v4l2" on supported hardware

Describe alternatives you've considered

• One-off camera implementations per platform: Infeasible due to required effort, but support would be betteer
• Vanilla v4l2-- Generally unusable because lacks color calibration support, autofocus, etc. found in complex cameras (e.g., PinePhone Pro, Librem 5)

Additional context

• libcamera has basic support for Librem 5, with more thorough support coming soon
• libcamera fully supports PinePhone Pro
• libcamera supports the vanilla PinePhone

[rothn]

@aleasto @erfanoabdi I'm probably going to try to knock these out over the weekend. Does the scope I suggested under "Describe the solution you'd like" sound right to you? Anything I'm missing that I may want to consider?

[aleasto]

libcamera already implements the android HAL interface?

[rothn]

Yes tree link, although it's based on camera3.h, which is only supported on Android < 8 according to the docs. So part of the scope might also have to include porting libcamera's HAL linkage from standard C++ to AIDL with XML metadata. Not a weekend project anymore.

[aleasto]

No it's ok. The default hidl implementations already wrap around the legacy hal format. Everybody uses that.

[aleasto]

However it doesn't look like libcamera builds inline with the aosp build system

[rothn]

No, it doesn't. Think that will be an issue? I was thinking I'd build a .so file, find the right place to drop it (e.g. /vendor), and hopefully it would be picked up correctly.

[aleasto]

The right place would be /vendor/lib[64]/hw/camera.libcamera.so and ro.hardware.camera=libcamera

There's a way to include a prebuilt blob in aosp but it's less than ideal. We don't do this for any other native component.

You also need to figure out how to build against the android NDK rather than your host's glibc & co. Since the android part seems to be supported upstream, they might already have tools for this.

[aleasto]

kbingham/libcamera#26
https://github.com/GloDroid/glodroid_forks/commits/libcamera-next

Here's an effort to build inline with aosp following the same approach as freedesktop/mesa

[rothn]

@aleasto I'm stumped... I've got libcamera building inline now and updated Waydroid to use libcamera (see WIP section of this guide.

The files are there:

• /vendor/lib64/hw/camera.libcamera.so
• /vendor/lib64/libcamera.so
• /vendor/lib64/libcamera-base.so

And yet, according to Android, based on lshal and dumpsys media.camera, this HAL does not exist!

EDIT: I was able to get a little farther, to where the .so seems to load but still no cameras:

10-20 00:44:41.142    71    71 I CamPrvdr@2.4-legacy: Loaded "libcamera camera HALv3 module" camera module
10-20 00:44:41.142    71    71 I CamPrvdr@2.4-legacy: setUpVendorTags: No vendor tags defined for this device.

I'm working with the libcamera team to support a full inline build, but I need to be able to validate that this works first. Any ideas or guesses as to what I might be missing? I'm really stumped and would appreciate your help.

libcamera_debug.log
logcat.log
logcat.log

Thanks in advance!

[aleasto]

It looks correctly hooked up.

BTW I'm not sure anybody has verified that the camera stack still works in lineage-18.1 builds. Have you verified on a compatible camera that the current V4L2 hal works prior to this work?

[kbingham]

Can you try to set LIBCAMERA_LOG_LEVELS=:0 as an environment variable for whatever process space is loading the libcamera HAL module please? (and share the logs)

[rothn]

@aleasto The V4L2 HAL doesn't work, which isn't surprising. The RK3399 SoC my phone is based on is kind of weird in that it needs a userland control loop to really work. libcamera has a special pipeline (rkisp2 IIRC) that implements this.

Though the camera technically implements V4L2, a quick read-through of Google's V4L2 implementation and TODO(b/*) comments makes it pretty clear that Google's V4L2 HAL will reject cameras from that SoC.

[aleasto]

on a compatible camera was the key there

[rothn]

@kbingham I believe that the libcamera logger is broken, and have filed a ticket with more details: https://bugs.libcamera.org/show_bug.cgi?id=161
Working around that, I get a much more informative logcat: logcat.log

It looks like this expects (but does not find) /dev/media[0-3], which looks like a pretty good clue. Investigating...

Key output:

10-20 14:57:11.467    76    76 W android.hardware.camera.provider@2.4-service: LIBCAMERA_LOG_LEVELS is 0
10-20 14:57:11.467    76    76 W android.hardware.camera.provider@2.4-service: Initialising Android camera HAL
10-20 14:57:11.467    76    76 W android.hardware.camera.provider@2.4-service: Failed to open configuration file /vendor/etc/libcamera/camera_hal.yaml: No such file or directory
10-20 14:57:11.468    76    76 W android.hardware.camera.provider@2.4-service: Public key not valid
10-20 14:57:11.468    76    76 W android.hardware.camera.provider@2.4-service: No IPA found in '/usr/local/lib/libcamera'
10-20 14:57:11.468    76    76 W android.hardware.camera.provider@2.4-service: libcamera v0.0.1
10-20 14:57:11.469    76   102 W android.hardware.camera.provider@2.4-service: Starting camera manager
10-20 14:57:11.470    76   102 W android.hardware.camera.provider@2.4-service: Device node /dev/media2 should exist but doesn't
10-20 14:57:11.470    76   102 W android.hardware.camera.provider@2.4-service: Device node /dev/media0 should exist but doesn't
10-20 14:57:11.470    76   102 W android.hardware.camera.provider@2.4-service: Device node /dev/media3 should exist but doesn't
10-20 14:57:11.471    76   102 W android.hardware.camera.provider@2.4-service: Device node /dev/media1 should exist but doesn't
10-20 14:57:11.471    76   102 W android.hardware.camera.provider@2.4-service: Found registered pipeline handler 'SimplePipelineHandler'
10-20 14:57:11.471    76   102 W android.hardware.camera.provider@2.4-service: Found registered pipeline handler 'PipelineHandlerRkISP1'
10-20 14:57:11.472    76    76 I CamPrvdr@2.4-legacy: Loaded "libcamera camera HALv3 module" camera module
10-20 14:57:11.472    76    76 I CamPrvdr@2.4-legacy: setUpVendorTags: No vendor tags defined for this device.
10-20 14:57:11.472   101   101 I HidlServiceManagement: Registered android.hardware.gatekeeper@1.0::IGatekeeper/default
10-20 14:57:11.474   101   101 I HidlServiceManagement: Removing namespace from process name android.hardware.gatekeeper@1.0-service to gatekeeper@1.0-service.
10-20 14:57:11.474   101   101 I LegacySupport: Registration complete for android.hardware.gatekeeper@1.0::IGatekeeper/default.
10-20 14:57:11.475    76    76 I HidlServiceManagement: Registered android.hardware.camera.provider@2.4::ICameraProvider/legacy/0
10-20 14:57:11.475    76    76 I HidlServiceManagement: Removing namespace from process name android.hardware.camera.provider@2.4-service to provider@2.4-service.
10-20 14:57:11.476    76    76 I LegacySupport: Registration complete for android.hardware.camera.provider@2.4::ICameraProvider/legacy/0.
...
10-20 14:57:13.499   144   144 I CameraProviderManager: Connecting to new camera provider: legacy/0, isRemote? 1
10-20 14:57:13.500   144   144 I CameraProviderManager: Camera provider legacy/0 ready with 0 camera devices
10-20 14:57:13.500   144   182 W CameraProviderManager: addProviderLocked: Camera provider HAL with name 'legacy/0' already registered
10-20 14:57:13.503   144   144 I cameraserver: Waiting for activity service