[Windows] Software Renderer Protection and 10 bit Output

[CrystalP]

Description

• Turn on D3D11 multithreading protection for dxva2 decoding + software renderer path because the main thread and the video player video thread sometimes use the non-thread safe D3D immediate context simultaneously

  • protection is disabled when playback ends, unless protection was already on (could happen if separate D3D device could not be created for DXVA decode)
  • a critical section to protect the immediate context would be better and would cover all situations, however that's a lot of changes for a case that's likely not used a lot. So, using the same solution as when a separate D3D device cannot be created for the dxva decoding.
  • no need to protect the software decode path because all operations happen on the main (=rendering) thread

• Make swscale use the output format AV_PIX_FMT_X2BGR10 when Kodi is using a 10 bit swap chain (HDR screen&media and SDR media with "Use 10 bit for SDR" setting.

  • the rest of the code doesn't need to change as the intermediate texture is already created as DXGI_FORMAT_R10G10B10A2_UNORM in those situations

Motivation and context

The D3D debug layer triggers exceptions when using dxva2 decoding with software renderer. Can take a while, or very quickly when some breakpoints are set that significantly slow down the program and alter the usual timing.

Wrong colors on 10 bit output of software renderer.

How has this been tested?

Windows 10 x64, debug and release mode, UWP on Windows 10 as well.
Software renderer with DXVA2 decoding on and off. HDR and SDR material with HDR and SDR 10 or 8 bit output.

There are pre-existing unrelated issues with dxva2 decode + software renderer: on UWP decoded frames are presented in wrong order (improves when OSD is displayed), for x64 there are partially updated frames when playing 4k or when the OSD is displayed,

What is the effect on users?

Software renderer displays correctly HDR and works in SDR 10 bit mode.

Avoids some corruption when using dxva2 with software renderer. I didn't witness it but it would be more likely to happen on slow computers or more intensive loads (4k).

Screenshots (if appropriate):

Exception for simultaneous usage of immediate context from 2 threads:

Corrupted 10 bit output:

Fixed output:

Types of change

• Bug fix (non-breaking change which fixes an issue)
• Clean up (non-breaking change which removes non-working, unmaintained functionality)
• Improvement (non-breaking change which improves existing functionality)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that will cause existing functionality to change)
• Cosmetic change (non-breaking change that doesn't touch code)
• None of the above (please explain below)

Checklist:

• My code follows the Code Guidelines of this project
• My change requires a change to the documentation, either Doxygen or wiki
• I have updated the documentation accordingly
• I have read the Contributing document
• I have added tests to cover my change
• All new and existing tests passed

[thexai]

Good findings! 👍

The truth is that I knew the wrong pixel format in render SW + 10 bit but I had completely forgotten.

[CrystalP]

Changes applied.
Further testing showed another problem with the software renderer: it has no primaries conversion so colors are a bit off when tonemapping HDR to SDR (compared with DXVA and Pixel Shaders). Not sure what to do about it.

[thexai]

Further testing showed another problem with the software renderer: it has no primaries conversion so colors are a bit off when tonemapping HDR to SDR (compared with DXVA and Pixel Shaders).

Seems this happens, probably topic for another PR...

DXVA

Software