Vulkan: Use compute shader for RealXFB decoding

[stenzek]

Ends up being slightly faster (38% in a purely XFB upload bound scenario), but mainly trying to find a solution to the black screen/incorrect colours issue.

[JMC47]

@dolphin-emu-bot rebuild

[JMC47]

I didn't notice much of a performance difference, but if it's a performance up and realXFB works fine still, that's nice for me.

[degasus]

Is there a good way to share more code with the texture decoder of the vulkan backend?

[iwubcode]

@degasus - what if we shared this once AbstractTextures was merged? We could have a function called ConvertDataToRGBFormat(const void* src_ptr, u32 src_width, u32 src_stride, u32 src_height, ConversionFormat) on the texture? I think we'd have to fallback to CPU if there weren't any compute-shaders though.

This would fall in line with the conversation on IRC yesterday about hybrid-xfb and doing YCbCr to RGB on the GPU if possible.

So we'd support converting both formats (YCbCr and YUYV) to RGB using either GPU or CPU?

[stenzek]

@degasus Yup, I don't see why it couldn't be added as another format for the texture decoding shaders.

[phire]

Hybrid XFB requires a XFB decoding shader anyway.

[iwubcode]

@phire - isn't that just another 'flavor' of converting some data to RGB format?

[psennermann]

Is it possible to have a build link to test if this PR really fixes the black screen on Vulkan XFB?
Thanks

[JMC47]

Click show all tests and find the buildbot for whatever version you're using (Windows x64 for most people) and click details. On that page you'll be able to find a download link to the test build.

[psennermann]

Oh thanks, unfortunately the black screen is still there...

[mimimi085181]

For me it did not fix the black screen issue. I have high hopes for #5498 though.

[stenzek]

Irrelevant now that Hybrid XFB has been merged.