Software decoding looks low quality

[TianZerL]

Bug report

Describe the bug

Here is a clear and concise description of what the problem is:

it looks low quality if I use software decoding (ff-hevc) for hevc video, and will be fine if I enable hardware acceleration.
Same video software decoding in VLC or MPC-HC is OK.

Expected Behavior

Here is a clear and concise description of what was expected to happen:

The edges of the lines should be smooth.

Actual Behavior

Line edges are not smooth, it's especially evident in Anime.
There is no scale, so it's not because of nearest upscaling.

Possible Fix

Use hardware decoder instead of ff-hevc software decoder.

To Reproduce

Steps to reproduce the behavior:

1. Make sure ff-hevc decoder is currently using.
2. Play a hevc video, both 8 or 10bit is ok.

Debuglog

The debuglog can be found here:

kodi.old.log

Screenshots

Here are some links or screenshots to help explain the problem:

Please right click to see the original image. Notice the edges of the text, and the pattern in the middle of the first group of images

HEVC 10bit KODI SW (low quality)

HEVC 10bit VLC SW (OK)

HEVC 8bit KODI SW (low quality)

HEVC 8bit KODI HW (OK)

Additional context or screenshots (if appropriate)

Here is some additional context or explanation that might help:

Your Environment

Used Operating system:

• Android

• iOS

• tvOS

• Linux

• OSX

• Windows

• Windows UWP

• Operating system version/name: Windows 11

• Kodi version: 20 Alpha3

note: Once the issue is made we require you to update it with new information or Kodi versions should that be required.
Team Kodi will consider your problem report however, we will not make any promises the problem will be solved.

[thexai]

Totally useless as you not provided none info (video resolution, render resolution, output resolution, debug log, etc.).

fine if I enable hardware acceleration.

This is the key point.

Some context:
Best upscaling method is DXVA (and only can be used is is used DVXA render method).
And by default DXVA only is used if is used DXVA2 video decoding.

DXVA2 = OFF ----> PixelShaders are used ----> DXVA upscaling cannot be used.

[TianZerL]

@thexai

video resolution, render resolution, output resolution

I have said that there was no scale, all 1080p, 100% point-to-point.
I don't think this is a problem with the renderer, PixelShaders or software render is same result. output from the ff-hevc decoder should be bad.
I updated the log of playing the video, if it is helpful, but this problem appeared on many of my machines, which I think may not be called as a 'bug', so I didn't upload it before.

[thexai]

Debug log is mandatory

[TianZerL]

@thexai

After more tests, I found that this does not only happen with hevc, but also with H264. Therefore, I suspect that all software decoders have this problem.

It doesn't look like a scaling problem. I used MPC-HC in different scaling algorithm for comparison. These unsmooth edges are neither the nearest nor bilinear.

The log of h264 is as follows: kodi.log

[thexai]

Intel(R) HD Graphics 4400

INFO <general>: DX::DeviceResources::CheckNV12SharedTexturesSupport: extended NV12 shared textures is NOT supported

DX::DeviceResources::GetVideoDriverVersion: video driver version is Intel 20.19.15.5166

I'm sorry, but I'm not going to waste time looking at something that happens in outdated graphics (10 years old)

[TianZerL]

@thexai
it's definitely not the point!
Same in Nvidia GTX1660super or AMD vega8.

This happens all of my PC with different GPUs: Intel HD4400, AMD Vega8, AMD 5700xt and Nvidia GTX1660Super. I don't think all of them are out of date.

[thexai]

Debug log?

[TianZerL]

@thexai
Debug log from AMD Vega8: kodi.log

[sarbes]

It looks to me that the Chroma siting is 0.5px off in both directions. I.E. sampling Type 2 4:2:0 media as Type 1.

[malvinas2]

Does the issue still occur under version 21 using FFmpeg 6.0 ?

[CrystalP]

Yes and dxva is affected sometimes, has to do with 8 or 10 bit but not clear yet how.
Not related to the decoding but the yuv to rgb conversion. It just happens that by default dvxa decoding yields dxva conversion, and software decoding to shader / ffmpeg swscale conversion.
The fact that the picture looks better with dxva doesn't necessarily mean that it's accurate, the vendor controls that path and could have added some " enhancement" processing. More digging needed.

[CrystalP]

After some tests, with dxva the issue is the AMD dxva processor with 10 bit output (same results with 8 or 10 bit input). There is a pixellation of the chroma that doesn't exist with 8 bit output.

Intel and NVIDIA have no such issue.

The pixel shaders (Kodi color conversion) and software render method (ffmpeg color conversion) show a similar chroma pixellation.

dxva may have more advanced chroma upscaling algorithms, explaining the better results. To be further investigated.

[CrystalP]

If you're still around and can test with v21 beta2 + AMD other than the vega 8 (like 5700XT), you may find an issue with dxva render method when the new setting " High quality processing" is on. Please let us know.
(see discussion in the linked PR to fix, it's not clear if all AMD are impacted or only some)

[TianZerL]

@CrystalP it seems ok with 5700xt.
kodi.log
I'm using a 10bit monitor, do I need to change anything?

[CrystalP]

Thanks!
Please adjust some settings, that did not use the path that causes issues. Per the log the dxva processor output was 8 bit, which was fine on my 5600G as well:

CRendererDXVA::Configure: chosen conversion: NV12 / YCBCR_STUDIO_G22_LEFT_P709 to B8G8R8A8_UNORM / RGB_FULL_G22_NONE_P709
...
DX::DeviceResources::ResizeBuffers: 8 bit swapchain is used with 6 flip discard buffers and SDR output (format B8G8R8A8_UNORM)

Set the following options:

• "Use 10 bit for SDR": Always (System > Display)
• "Use high precision processing": ON (Player > Videos)
• "Render method": DXVA (Player > Videos)

Play an SDR video.

On my 5600G (contemporary with your vega 8) that makes the chroma pixillated even with latest driver.

[TianZerL]

@CrystalP

• "Use 10 bit for SDR": Always (System > Display)
• "Use high precision processing": ON (Player > Videos)
• "Render method": DXVA (Player > Videos)

Did those and got same result.
kodi.log

Maybe it is for GCN only? New Xbox and 5700XT is RDNA.

[CrystalP]

Something is wrong, it still picked 8 bit

CRendererDXVA::Configure: chosen conversion: NV12 / YCBCR_STUDIO_G22_LEFT_P709 to B8G8R8A8_UNORM / RGB_FULL_G22_NONE_P709

Maybe the processor doesn't have support for 10 bit.

Please turn on additional logging in System > Logging.
Enable debug logging ON
Enable component-specific logging ON
"Specify component-specific logging..." turn on for the "Video component"

Then play the video.

[TianZerL]

@CrystalP
Do I need to play a 10bit HEVC video?

Tested with 10 bit HEVC video, it still looks ok.

10bit video log:
kodi.log

[CrystalP]

Thanks. I could grab your log file with 8 bit before you replaced it and have a clear picture now for the 5700XT,

For 8 bit source the dxva processor doesn't claim support for rgb10 output
For 10 bit source, the dxva processor claims support for rgb10 output and it's used with good visual result.

So yeah it's likely architecture related. Could you try your vega 8? (both 8 and 10 bit input)

[TianZerL]

@CrystalP
I can confirm this issue with Vega8 (Ryzen 3500U).
It happens with both 8bit and 10 bit source, only if I enabled "Use high precision processing".

Vega8 log: kodi.log

[CrystalP]

Great, now we have something. Thanks for your help, it confirms an issue with the GCN family. Now how to identify it...

Looks like AMD put GCN and RDNA in different branches recently: https://www.anandtech.com/show/21126/amd-reduces-ongoing-driver-support-for-polaris-and-vega-gpus

Looking at the driver version numbers, the 3500U has 31.0.21902.5, the 5600G has 31.0.21905.1001 (same for a couple months now, matches the timeline of the article)
The RDNA 5700 XT has 31.0.22023.1014 in your log and most recent available is 31.0.23013.1023.

We could probably filter with version < 31.0.22000.0.
@thexai could you please confirm driver version on xbox?

[thexai]

There is not easy way to know driver version on Xbox because apps not have access to Windows Registry (not reported on Kodi logs) and driver version also not seems present in Xbox system settings. Anyway is easy exclude/include Xbox from workarounds with this code:

if (CSysInfo::GetWindowsDeviceFamily() == CSysInfo::Xbox)