[Draft] Convert input media type from RBG to YUV

[federicoveronesi]

In this proof of concept code change we remove the implicit conversion from RGB32 to a format supported by H264 MF encoder.
MF H246 encoder support MFVideoFormat_IYUV format
https://docs.microsoft.com/en-us/windows/win32/medfound/h-264-video-encoder

In this code change the frame acquired from screen is converted to YUV and the passed to video encoder.
The result has the same image quality but CPU usage is reduced by ~50% and GPU usage is also reduced by ~30%.

Note: Code seems to be copied from https://github.com/descampsa/yuv2rgb

[forderud]

Wow! That was a surprisingly large speedup. I guess we can live with a bit more complex implementation if reducing CPU & GPU load as much as 30-40%.

Have you checked if the speedup is consistent across Nvidia vs. Intel vs. no-GPU back-ends, or is it only one of them that is slow without this optimization?

[federicoveronesi]

It seems consistent across all the GPUs, but I guess it will not impact significantly the CPU encoding performance. Also we hope to get even better results with RGB2YUV implementation on GPU. Il Gio 30 Set 2021, 15:36 Fredrik Orderud ***@***.***> ha scritto:

…

Wow! That was a surprisingly large speedup. I guess we can live with a bit more complex implementation if reducing CPU & GPU load as much as 30-40%. Have you checked if the speedup is consistent across Nvidia vs. Intel vs. no-GPU back-ends, or is it only one of them that is slow without this optimization? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#17 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADVKKHOLWNQFHY6CECORU73UERRUVANCNFSM5FCKJHLA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[forderud]

Could it be an idea to combine this PR with a ENABLE_RGB2YUV preprocessor define, so that it becomes simple to compare results with vs. without YUV conversion moving forward?

It seems to me like the code bloat associated with a ENABLE_RGB2YUV define would be quite small. This could be useful both for performance profiling and for color "correctness" comparison.

[forderud]

Would you mind adding a comment that explains where the yuv_rgb.h/cc files are copied from, so that we get some traceability for future updates of this code? You can either add the comment at the top of the files or in the commit comment.