DolphinQt / VideoCommon / Core - Add support for "split screen" using freelook

[iwubcode]

Imagine you're hanging out at a friend's house and they pull up a single player game. Wouldn't it be nice to do more than watch them? Well now you can by using this PR (and in conjunction with this one for easier controls - #8747 ).

Introducing split screen freelook!

• Interact with your friend's game, while they are playing
• Get an advantage by looking from a different perspective while still playing the game normally
• More interesting reasons in the future :)

You can play in horizontal split screen:

or vertical split screen:

Here's the current UI:

I felt like valuing screen real-estate made more sense (despite the weird aspect ratio) but open to suggestions.

---

General testing and fixing the side-by-side / top-bottom 3d configurations are what is keeping this in draft.

[Bearborg]

Rotating the camera seems to cause extreme distortion of the geometry. Also, when using this in conjunction with stereoscopic rendering, the stereoscopic offset is applied incorrectly- both of the non-freelook screens render one eye's perspective, while both of the freelook screens render the other.

[iwubcode]

@Bearborg - please try again at your earliest convenience. I'm sill working on the 3d but believe the distortion should be fixed.

[Bearborg]

I'm no longer seeing any further distortion when I rotate the camera, but geometry seems to be weird in general now. It seems like it might be depth-related?

[iwubcode]

Thanks for the additional testing @Bearborg . I will look into the fix and then go through more rigorous testing to ensure the solution works. I only ran a couple of games last night. I seem to have messed up something after I took the screenshots in the PR description. My apologies for the busy-work.

[iwubcode]

So I went back to my old implementation @Bearborg . But that once again flips the horizontal movement and zoom. @stenzek suggested modifying the vertex shader to not apply the projection matrix in that case. I guess that's what I'll try next.

Still not sure what is wrong with SBS/TAB stereo 3d logic. Or rather, I feel like I fixed the problem where each eye wasn't rendering properly but the geometry shader isn't executing in the way I expect. The "instancing" isn't happening and this if statement I have eye = (layer_view < 2) ? 0 : 1 seems to always take the true path. layer_view is essentially the InstanceID for geometry shader instancing. I'm setting instancing to 4, so I'm not sure what is going wrong. Anyone that might have an idea, please let me know!

[Bearborg]

Geometry now looks correct and rotates correctly. However, I've noticed that geometry behind the camera plane is completely absent, even when that same geometry is normally visible in freelook.

[iwubcode]

The issues @Bearborg reported should be fixed to a degree (thank you for the testing Bearborg!) as well as the movement being backward. @stenzek 's change was the solution to not use the vertex shader when applying the projection matrix but also ended up moving a lot of stuff to the geometry shader and increased complexity.

There is a lot more work that needs to be done. Vulkan and likely OpenGL have graphical errors on many games. And the following issues are known:

• Per-pixel lighting fails because it needs the projected position in the vertex shader; it needs to be moved to the geometry shader
• "Emboss map" texture gen fails because it needs the projected position in the vertex shader; it needs to be moved to the geometry shader
• "Vertex rounding" needs to be moved to the geometry shader

I've also noticed some game errors even on D3D (the "best" backend at the moment, since I've been primarily using it for testing):

• Last Story - for some reason, the freelook does no take affect at all in split screen view
  

• Mario Kart Wii - some odd transparent images of the game that randomly gets put into the game world (even odder this doesn't happen on Vulkan!)
  

• Spiderman Shattered Memories - the issue that @stenzek fixed, is back in splitscreen. Not immediately obvious what needs to be changed
  

Ubershaders have not been tested yet and the stereo SBS/TAB are still busted.

Lots to be done, my biggest concern is this makes shader management more difficult by requiring us to maintain two pathes with very similar code. I'm not sure what to do to fix that.

Additionally FWIW #8380 also used an inverse projection matrix, so it would need to adopt these same changes to avoid the "black behind" issue Bearborg mentioned.

[Bearborg]

Can confirm that this fixes all the issues I was seeing with camera transformations.

[iwubcode]

Maybe some day I will revisit but for now, this is dead. Closing.