OverlayShaderProgram does not support HDR colors yet

[steelbytes]

Compiled and installed demo-transformer onto my Pixel 7 Pro.
Selected a video recorded on my pixel.
Choose video effect overlay logo & timer.
Clicked export.
Failed.

VERY surprised you don't support videos created on your own phone

Caused by: java.lang.IllegalArgumentException: OverlayShaderProgram does not support HDR colors yet.
                                                               at androidx.media3.common.util.Assertions.checkArgument(Assertions.java:55)
                                                               at androidx.media3.effect.OverlayShaderProgram.<init>(OverlayShaderProgram.java:58)
                                                               at androidx.media3.effect.OverlayEffect.toGlShaderProgram(OverlayEffect.java:44)
                                                               at androidx.media3.effect.OverlayEffect.toGlShaderProgram(OverlayEffect.java:27)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor.getGlShaderProgramsForGlEffects(DefaultVideoFrameProcessor.java:707)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor.createOpenGlObjectsAndFrameProcessor(DefaultVideoFrameProcessor.java:606)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor.access$100(DefaultVideoFrameProcessor.java:70)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor$Factory.lambda$create$0$androidx-media3-effect-DefaultVideoFrameProcessor$Factory(DefaultVideoFrameProcessor.java:258)
                                                               at androidx.media3.effect.DefaultVideoFrameProcessor$Factory$$ExternalSyntheticLambda0.call(Unknown Source:20)
                                                               at java.util.concurrent.FutureTask.run(FutureTask.java:264)
                                                               at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
                                                               at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
                                                               at java.lang.Thread.run(Thread.java:1012)

[droid-girl]

OverlayShaderProgram does not currently support HDR colors.
You can use setHdrMode(HDR_MODE_TONE_MAP_HDR_TO_SDR_USING_OPEN_GL) to tone-map your HDR video to SDR

[steelbytes]

thanks. probably should update the sample app

[steelbytes]

tried that setting (found it was unhelpfully hidden in demo app) and tried both the opengl and the mediacodec choices and both failed to make a video resembles the original.

so still a no go for me and my needs :-(

[dway123]

Unfortunately, tone-mapping does involve converting HDR to SDR, which also removes the extra brightness information. Until OverlayShaderProgram supports HDR colors, I think tone-mapping and applying an overlay are the best Transformer can do out of the box. You could also implement your own custom Overlay implementation that supports HDR overlays, to avoid needing to tone map.

Adding HDR support for overlays isn't currently planned, but when we do add that support, I suspect your needs could be met through that too.

[steelbytes]

it's not the accuracy of the overlay colours that are bugging me. its the colours of the base video as it appears in the output are so dead looking vs original.

[dway123]

Yes, what I meant is that because OverlayShaderProgram only supports SDR and not HDR, you must input SDR colors to use it. From there, tone-mapping in general converts HDR colors to SDR, which clips out the brighter and darker parts of the base video (which might be contributing to the "dead-looking" look, relative to the original HDR input video)

[droid-girl]

Transformer now supports:

• UltraHDR image as overlay. You need to enable it by setting experimentalSetRetainHdrFromUltraHdrImage to true
• HDR support for TextOverlay. You can use luminanceMultiplier option in OverlaySettings