Feature request: Enable DOL-HDR support for Sony IMX477 on Raspberry Pi 5

[Bra1nsen]

Hello libcamera devs,

I'm working with the Sony IMX477 sensor on Raspberry Pi 5 and I'm interested in enabling its built-in DOL-HDR mode (2- or 3-frame).

According to the sensor datasheet, the IMX477 supports Digital Overlap HDR (e.g. 2-frame DOL at 30 fps), which uses multiple staggered exposures with overlapping readouts and transmits them over CSI-2 virtual channels. From what I understand, these capabilities are not currently exposed or supported in the Raspberry Pi libcamera pipeline.

I’d like to ask:

Are there any plans to support DOL-HDR mode in the IMX477 camera driver and libcamera pipeline handler?

Would enabling this require changes to the kernel sensor driver and CSI-2 virtual channel handling in the pipeline?

If not planned yet, would you be open to contributions or testing support if someone works toward this?

We’d be happy to test or contribute patches if this is on the roadmap or if you can provide some guidance.

Thanks in advance for your time and for your great work on the Pi camera stack!

Best regards,
Paul

[6by9]

It would need significant changes to both kernel driver and libcamera, and I don't believe any of it is planned.

We don't have the full datasheet for DOL-HDR support on IMX477, but the basics we do have allow setting for 2 or 3 frame DOL-HDR.
Each frame has separate settings for integration time, analog gain, and digital gain - V4L2 currently has no way to expose those multiple settings.
Each frame uses a separate CSI virtual channel, so would be received into independent buffers by the CFE. Currently the image stream is fed directly into the ISP-FE, but that only accepts a single data type so would need significant rework in the pipeline handler. (Note that this can not be supported on Pi0-4).
Handling of multiple streams across a multiplexed bus (eg CSI2) requires the streams API changes that still haven't been fully merged into mainline.

You then need all the necessary IPA changes to compute the multiple exposure times and gains that should be used.

The further complication is to try and handle this in a manner that isn't specific to the imx477.

Summary - HDR (whether DOL or QBC) hasn't been discounted, but it is not a trivial task.

[naushir]

Unfortunately nobody is actively working on this issue, nor do we (RPi) have any future planes for it. I'm going to close it down