libobs: Separate textures for YUV output, fix chroma

[jpark37]

Description

The shaders to pack YUV information into the same texture were rather
complicated and suffering precision issues. Breaking them up into
separate textures makes the shaders much simpler and avoids having to
compute large integer offsets. Unfortunately, the code to handle
multiple textures is not as pleasant, but at least the NV12 rendering
path is no longer separate.

In addition, write chroma samples to "standard" offsets. For I444,
there's no difference, but I420/NV12 formats now have chroma shifted to
the left as 4:2:0 is shown in the H.264 specification.

Motivation and Context

YUV shaders are not operating as quickly as they should, and we can out knock two precision-related bugs in the process.
Fixes https://obsproject.com/mantis/view.php?id=624
Fixes https://obsproject.com/mantis/view.php?id=1512

Also nice to do chroma samples the H.264 way although it should probably be a user setting in a perfect world.

How Has This Been Tested?

• Verified output video correct, and correct shaders used in D3D.
• Verified output video correct, and correct shaders used in GL.
• Verify Mantis issues are fixed.

Intel GPA, SetStablePowerState, Intel HD Graphics 530

Expect speed incrase:
I420: 844 us -> 493 us (254 us + 190 us + 274 us)
I444: 837 us -> 747 us (258 us + 276 us + 272 us)
NV12: 450 us -> 368 us (319 us + 168 us)

Expect no change:
NV12 (HW): 580 (481 us + 166 us) us -> 588 us (468 us + 247 us)
RGB: 359 us -> 387 us

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.

[energizerfellow]

i422?

[jpark37]

I422 is not supported by OBS as an output format.

[energizerfellow]

True, but should it be added at some point with 4:2:2's prevalence in the pro video world?

[jpark37]

That sounds like a fine idea, but it's not on my personal hit list.

[jpark37]

Finished testing, and I verified both Mantis bugs are fixed.

[Xaymar]

I'm pretty sure there are 4:2:2 options for use in encoders:

	/* packed 422 formats */
	VIDEO_FORMAT_YVYU,
	VIDEO_FORMAT_YUY2, /* YUYV */
	VIDEO_FORMAT_UYVY,

I'm using these in obs-ffmpeg-encoder and enc-vfw.

Edit: Not saying these are required to be supported, just that they exist and currently likely fall back to software conversion.

[jpark37]

OBS has shaders to convert 4:2:2 YCbCr to RGB, but not the reverse. There are only 4 options in the output settings: NV12, I420, I444, and RGB.

[jpark37]

This would also make #1655 unnecessary because those shader functions have been deleted.

[SuslikV]

... I420/NV12 formats now have chroma shifted to
the left as 4:2:0 is shown in the H.264 specification

Document needs clarification with year of the last redaction used as base source for the implementation.