[Android] Implement 'HDR display enable' for ffmpeg decoded streams #16557

peak3d · 2019-09-01T16:54:02Z

Description

This PR implements switching HDR capable displays into HDR mode when playing HDR streams using LinuxRendererGLES GPU rendering.
When using this path, our internal HDR implementation is overriden and HDR calulation is done on GPU / display, its compareable to audio passthrough.

Motivation and Context

VP9 profile 2 HDR streams are currently not supported on NVIDIA Shield TV, using this PR its from theory possible to play them. 1080p streams don't play smooth yet, but there are still ptions to optimize the render chain (e.g. YUV scanout instead our own YUV2RGB shader)

How Has This Been Tested?

NVIDIA Shield TV + youtube addon
enable vp9 dash in youtube addon settings
search for 'vp9 hdr' and play one stream.
AFTV 4K (no GPU HDR extension)
HDR setting is not visible and internal HDR pipeline is used
WETEK HUB (SDK version does not support HDR)
HDR setting is not visible and internal HDR pipeline is used

For discussion

Solved: Should there be an settings option to disable this feature and use our shader HDR calulation instead?
There is now a setting which is only visible if display supports HDR and let users switch between display HDR and internal GL(ES) shader HDR

Types of change

Bug fix (non-breaking change which fixes an issue)
Clean up (non-breaking change which removes non-working, unmaintained functionality)
Improvement (non-breaking change which improves existing functionality)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that will cause existing functionality to change)
Cosmetic change (non-breaking change that doesn't touch code)
None of the above (please explain below)

a1rwulf · 2019-09-01T18:18:34Z

xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecFFmpeg.cpp

-    pVideoPicture->color_range = 0;
+    pVideoPicture->color_range = m_hints.colorRange == AVCOL_RANGE_JPEG ? 1 : 0;
+
+  CLog::Log(LOGWARNING, "XXX PRI: %d TRC: %d SPC: %d RNG: %d", pVideoPicture->color_primaries, pVideoPicture->color_transfer,  pVideoPicture->color_space, (int)pVideoPicture->color_range);


Are you sure you want this to be LOGWARNING?
Looks more like a candidate for LOGDEBUG to me.

use C++ style cast

a1rwulf · 2019-09-01T18:19:27Z

xbmc/cores/VideoPlayer/VideoRenderers/LinuxRendererGLES.cpp

@@ -116,6 +116,7 @@ bool CLinuxRendererGLES::ValidateRenderTarget()

 bool CLinuxRendererGLES::Configure(const VideoPicture &picture, float fps, unsigned int orientation)
 {
+  CLog::Log(LOGDEBUG, "LinuxRendererGLES: Configire with HDR metadad: %s", (picture.hasDisplayMetadata && picture.hasLightMetadata) ? "true" : "false");


LinuxRendererGLES: Configure with HDR metadata

Rechi · 2019-09-03T12:31:45Z

xbmc/windowing/android/WinSystemAndroidGLESContext.cpp

@@ -7,12 +7,14 @@
 */

 #include "WinSystemAndroidGLESContext.h"
-


add back the empty line, https://github.com/xbmc/xbmc/blob/master/docs/CODE_GUIDELINES.md#7-headers

Rechi · 2019-09-03T12:32:15Z

xbmc/windowing/android/WinSystemAndroidGLESContext.cpp

 #include "VideoSyncAndroid.h"
+


remove the empty line, https://github.com/xbmc/xbmc/blob/master/docs/CODE_GUIDELINES.md#7-headers

Rechi · 2019-09-03T12:32:45Z

xbmc/windowing/android/WinSystemAndroidGLESContext.cpp

 #include "VideoSyncAndroid.h"
+
+#include "cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodec.h"
+#include "platform/android/activity/XBMCApp.h"


wrong section, https://github.com/xbmc/xbmc/blob/master/docs/CODE_GUIDELINES.md#7-headers

Rechi · 2019-09-03T12:33:58Z

xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecFFmpeg.cpp

-    pVideoPicture->color_range = 0;
+    pVideoPicture->color_range = m_hints.colorRange == AVCOL_RANGE_JPEG ? 1 : 0;
+
+  CLog::Log(LOGWARNING, "XXX PRI: %d TRC: %d SPC: %d RNG: %d", pVideoPicture->color_primaries, pVideoPicture->color_transfer,  pVideoPicture->color_space, (int)pVideoPicture->color_range);


use C++ style cast

xbmc/windowing/android/WinSystemAndroidGLESContext.cpp

peak3d · 2019-09-04T07:11:29Z

Jenkins build this please

peak3d · 2019-09-04T10:33:07Z

Jenkins build this please

peak3d · 2019-09-04T12:43:35Z

Jenkins build this please

ksooo · 2019-09-04T12:54:31Z

jenkins build this please

ksooo · 2019-09-04T14:10:42Z

system/settings/settings.xml

+              <condition on="property" name="ishdrdisplay" />
+            </dependency>
+          </dependencies>
+          <!--requirement>ishdrdisplay</requirement-->


There is now a setting which is only visible if display supports HDR and let users switch between display HDR and internal GL(ES) shader HDR

Really? With this commented out the setting will always be visible.

nvm, sorry for the noise.

Thx, I'll remove the comment - leftover from testing

xbmc/utils/EGLUtils.h

xbmc/windowing/android/AndroidUtils.h

xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecFFmpeg.cpp

xbmc/settings/SettingConditions.cpp

peak3d · 2019-09-05T09:39:04Z

Jenkins build this please

peak3d · 2019-09-05T13:33:19Z

Jenkins build this please

CiNcH83 · 2019-09-06T21:57:18Z

I am not seeing this setting on BRAVIA. Suppose it does not support the Android/EGL HDR APIs.

ksooo · 2019-09-07T06:24:56Z

Working fine on Shield.

CiNcH83 · 2019-09-07T09:54:53Z

On BRAVIA:
2019-09-07 11:41:16.187 T:2487245168 DEBUG: CWinSystemAndroidGLESContext::InitWindowSystem: HDRConfig: 0, HDRExtensions: 0

peak3d · 2019-09-07T10:01:17Z

@CiNcH83 yes, there has to be a EGL extension. Not sure if we will arrive this extension on devices other than Nvidia Shield, because the Shield is the only one which focusses on Game development, too.

Could be that we have a chance for other devices if we use the Vulkan API, but - your HDR videos should look ok if you use our HDR implementation. The display switch to HDR could save some GPU resources, but results should match.

CiNcH83 · 2019-09-07T10:08:22Z

your HDR videos should look ok if you use our HDR implementation.

On my BRAVIA, SW decoding in Kodi isn't feasible anyway. HDR video playback via MediaCodec Surface is fine (and has been fine before).

The only problem is the Kodi menu/OSD. When the TV detects an HDR video and switches into the respective HDR mode, there is always a distinct color and luminance shift in OSDs. Artwork is way too dark for example. Haven't checked yet whether this is still the case.

MilhouseVH · 2019-09-07T18:34:05Z

This PR has broken the RPi4 with Kodi 19 (latest master), which is using mesa.

kod.bin now exits with the following error (it does not crash):

ERROR: Unable to create GUI. Exiting

In the kodi debug log:

2019-09-07 19:31:31.417 T:3006934624  NOTICE: EGL_VERSION = 1.4
2019-09-07 19:31:31.417 T:3006934624  NOTICE: EGL_VENDOR = Mesa Project
2019-09-07 19:31:31.418 T:3006934624  NOTICE: EGL_EXTENSIONS = EGL_EXT_buffer_age EGL_EXT_image_dma_buf_import EGL_EXT_image_dma_buf_import_modifiers EGL_KHR_cl_event2 EGL_KHR_config_attribs EGL_KHR_create_context EGL_KHR_create_context_no_error EGL_KHR_fence_sync EGL_KHR_get_all_proc_addresses EGL_KHR_gl_colorspace EGL_KHR_gl_renderbuffer_image EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_3D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_image EGL_KHR_image_base EGL_KHR_image_pixmap EGL_KHR_no_config_context EGL_KHR_reusable_sync EGL_KHR_surfaceless_context EGL_EXT_pixel_format_float EGL_KHR_wait_sync EGL_MESA_configless_context EGL_MESA_drm_image EGL_MESA_image_dma_buf_export
2019-09-07 19:31:31.418 T:3006934624  NOTICE: EGL_CLIENT_EXTENSIONS = EGL_EXT_client_extensions EGL_EXT_device_base EGL_EXT_device_enumeration EGL_EXT_device_query EGL_EXT_platform_base EGL_KHR_client_get_all_proc_addresses EGL_KHR_debug EGL_MESA_platform_gbm
2019-09-07 19:31:31.418 T:3006934624   DEBUG: failed to find EGL config with EGL_NATIVE_VISUAL_ID=875713089
2019-09-07 19:31:31.418 T:3006934624   DEBUG: failed to find EGL config with EGL_NATIVE_VISUAL_ID=875713112
2019-09-07 19:31:31.418 T:3006934624   DEBUG: failed to find EGL config with EGL_NATIVE_VISUAL_ID=875713089
2019-09-07 19:31:31.418 T:3006934624   DEBUG: failed to find EGL config with EGL_NATIVE_VISUAL_ID=875713112
2019-09-07 19:31:31.418 T:3006934624   FATAL: CApplication::Create: Unable to init windowing system

Reverting this PR (and also #16579) restores working Kodi with RPi4.

MilhouseVH · 2019-09-07T18:41:04Z

From @Kwiboo on LibreELEC Slack:

cool, I suspect the EGL_PBUFFER_BIT change to be the most likely culprit, the driver probably do not supports creating pixel buffer surfaces

and

that is just my best guess, could also be the EGL_ALPHA_SIZE change, the rest of the changes do mostly look to be isolated to when hdr = true

ksooo · 2019-09-08T17:04:12Z

addons/resource.language.en_gb/resources/strings.po

+#. Description of setting with label #13436 "Use display HDR capabilities"
+#: system/settings/settings.xml
+msgctxt "#36299"
+msgid "Switch display into HDR mode if media with HDR information is played.\nIf disabled, HDR information are applied using kodi's internal HDR path"


@peak3d typo: kodi's => Kodi's (capital K, please) and a period is missing to end the second sentence.

peak3d added Type: Feature non-breaking change which adds functionality Component: Video Component: Video rendering Platform: Android Platform: Linux v19 Matrix Component: GLES rendering labels Sep 1, 2019

peak3d added this to the Matrix 19.0-alpha 1 milestone Sep 1, 2019

a1rwulf reviewed Sep 1, 2019

View reviewed changes

Rechi reviewed Sep 3, 2019

View reviewed changes

peak3d force-pushed the vp9hdr branch from 4c3735b to be65230 Compare September 3, 2019 22:09

ksooo reviewed Sep 4, 2019

View reviewed changes

peak3d force-pushed the vp9hdr branch from 67b0be8 to d717201 Compare September 4, 2019 15:34

Rechi reviewed Sep 4, 2019

View reviewed changes

xbmc/utils/EGLUtils.h Outdated Show resolved Hide resolved

xbmc/windowing/android/AndroidUtils.h Outdated Show resolved Hide resolved

xbmc/windowing/android/AndroidUtils.h Outdated Show resolved Hide resolved

xbmc/windowing/android/AndroidUtils.h Outdated Show resolved Hide resolved

peak3d force-pushed the vp9hdr branch from d717201 to c8c12c7 Compare September 4, 2019 18:40

Rechi reviewed Sep 5, 2019

View reviewed changes

xbmc/cores/VideoPlayer/DVDCodecs/Video/DVDVideoCodecFFmpeg.cpp Outdated Show resolved Hide resolved

xbmc/settings/SettingConditions.cpp Outdated Show resolved Hide resolved

[Android] Implement HDR display enable for ffmpeg decoded streams

c40d3ce

peak3d force-pushed the vp9hdr branch from c8c12c7 to c40d3ce Compare September 5, 2019 08:21

peak3d merged commit f4ead59 into xbmc:master Sep 6, 2019

peak3d deleted the vp9hdr branch September 6, 2019 11:52

peak3d mentioned this pull request Sep 7, 2019

[EGL Render] Fix accidentially changed egl config #16585

Merged

7 tasks

ksooo reviewed Sep 8, 2019

View reviewed changes

peak3d mentioned this pull request Sep 9, 2019

[Settings] Fix typo (s) introduced with HDR settings PR #16592

Merged

7 tasks

CastagnaIT mentioned this pull request May 20, 2021

Frame dropping playing 1080p (Netflix, Disney) xbmc/inputstream.adaptive#460

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Android] Implement 'HDR display enable' for ffmpeg decoded streams #16557

[Android] Implement 'HDR display enable' for ffmpeg decoded streams #16557

peak3d commented Sep 1, 2019 •

edited

a1rwulf Sep 1, 2019

Rechi Sep 3, 2019

a1rwulf Sep 1, 2019

Rechi Sep 3, 2019

Rechi Sep 3, 2019

Rechi Sep 3, 2019

Rechi Sep 3, 2019

peak3d commented Sep 4, 2019

peak3d commented Sep 4, 2019

peak3d commented Sep 4, 2019

ksooo commented Sep 4, 2019

ksooo Sep 4, 2019 •

edited

ksooo Sep 4, 2019

peak3d Sep 4, 2019

peak3d commented Sep 5, 2019

peak3d commented Sep 5, 2019

CiNcH83 commented Sep 6, 2019

ksooo commented Sep 7, 2019

CiNcH83 commented Sep 7, 2019

peak3d commented Sep 7, 2019

CiNcH83 commented Sep 7, 2019 •

edited

MilhouseVH commented Sep 7, 2019

MilhouseVH commented Sep 7, 2019

ksooo Sep 8, 2019 •

edited

[Android] Implement 'HDR display enable' for ffmpeg decoded streams #16557

[Android] Implement 'HDR display enable' for ffmpeg decoded streams #16557

Conversation

peak3d commented Sep 1, 2019 • edited

Description

Motivation and Context

How Has This Been Tested?

For discussion

Types of change

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peak3d commented Sep 4, 2019

peak3d commented Sep 4, 2019

peak3d commented Sep 4, 2019

ksooo commented Sep 4, 2019

ksooo Sep 4, 2019 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peak3d commented Sep 5, 2019

peak3d commented Sep 5, 2019

CiNcH83 commented Sep 6, 2019

ksooo commented Sep 7, 2019

CiNcH83 commented Sep 7, 2019

peak3d commented Sep 7, 2019

CiNcH83 commented Sep 7, 2019 • edited

MilhouseVH commented Sep 7, 2019

MilhouseVH commented Sep 7, 2019

ksooo Sep 8, 2019 • edited

Choose a reason for hiding this comment

peak3d commented Sep 1, 2019 •

edited

ksooo Sep 4, 2019 •

edited

CiNcH83 commented Sep 7, 2019 •

edited

ksooo Sep 8, 2019 •

edited