Vulkan: Simplify MultiFenceHolder and managing them

[riperiperi]

MultiFenceHolder is currently absolutely reliant on all users submitting to the same command buffer pool, which means they must all come from the same thread. This is because waitiers register the command buffer index, which is only valid for entries of the same pool. The locking and dictionary structures greatly slow down the process of binding and ref counting for all types of buffer, while also being unnecessary when these rules are followed.

This PR simplifies MultiFenceHolder in a few ways:

• ReservedCommandBuffer.Waitables is now a List for less overhead. Enforcing uniqueness is done by MultiFenceHolder.AddFence returning true only when the command buffer index is not yet registered on the waitable. Similarly, the call to Contains has been replaced with asking the waitable if it has the current fence, rather than checking the set on the command buffer.
• MultiFenceHolder now maintains an array of fences rather than a dictionary. Each entry in the array represents a command buffer in the pool. If the value in the array at that command buffer index is null, there is no registered fence yet. This removes all cost for trying to add and trying to remove fences, which was pretty considerable.
  • The WaitForFences methods have been updated to extract active fences from this array, and properly handle a fence already being disposed by the time it's iterated.
  • Everything is completely lockless, which is good for a method this hot.
• Added "Try" methods to Auto<> and FenceHolder to properly handle cases where we get a resource but it might already be disposed. These are a bit slower as they use a compare exchange to increment the value only if it's not 0, but they are at least correct if a bunch of threads are contending with each other.

There was an existing case where the rules about keeping Get(cbs) uses to the same pool are broken, which is the copy source when flushing data from a device local buffer. The destination is fine, because it is only used by the command buffer pool owned by the background thread. This will probably cause issues right now, but to avoid creating new ones I decided it's worth fixing.

I've changed this case to manage keeping the source buffer alive outside the copy method. Since it waits for the copy to complete anyways, it can just increment and decrement the reference count around the use manually.

---

Improves performance in Vulkan bottlenecked games such as BOTW. (at least on my system)

Testing various games would be helpful to ensure nothing ends up broken.

[github-actions]

Download the artifacts for this pull request:

• ryujinx-Release-1.1.0+b4924ce-linux_x64
• ryujinx-Release-1.1.0+b4924ce-osx_x64
• ryujinx-Release-1.1.0+b4924ce-win_x64

Experimental GUI (Avalonia)

• ava-ryujinx-Release-1.1.0+b4924ce-linux_x64
• ava-ryujinx-Release-1.1.0+b4924ce-osx_x64
• ava-ryujinx-Release-1.1.0+b4924ce-win_x64

GUI-less (SDL2)

• sdl2-ryujinx-headless-Release-1.1.0+b4924ce-linux_x64
• sdl2-ryujinx-headless-Release-1.1.0+b4924ce-osx_x64
• sdl2-ryujinx-headless-Release-1.1.0+b4924ce-win_x64

Only for Developers

• ava-ryujinx-Debug-1.1.0+b4924ce-linux_x64
• ava-ryujinx-Debug-1.1.0+b4924ce-osx_x64
• ava-ryujinx-Debug-1.1.0+b4924ce-win_x64
• ryujinx-Debug-1.1.0+b4924ce-linux_x64
• ryujinx-Debug-1.1.0+b4924ce-osx_x64
• ryujinx-Debug-1.1.0+b4924ce-win_x64
• sdl2-ryujinx-headless-Debug-1.1.0+b4924ce-linux_x64
• sdl2-ryujinx-headless-Debug-1.1.0+b4924ce-osx_x64
• sdl2-ryujinx-headless-Debug-1.1.0+b4924ce-win_x64

[gdkchan]

lgtm, thanks.

[Bjorn29512]

Tested about 20 titles :

Looked for any performance changes compared to master or any visual oddities that can be reliably spotted in limited playtime for each of the following titles.

Key observations :

• No Regression/change was noted compared to master in any of the titles tested :

  (Note for users: The changes are tangible and do infact improve performance in BOTW but it depends on your setup, gains are mostly noted on PCs with weaker CPUs.)

  • Bayonetta 3
  • FE: Three Hopes
  • FE: Engage
  • FE: Three Houses
  • Advance Wars 1+2 Re-Boot Camp
  • Hyrule Warriors : Age of Calamity
  • Mario + Rabbids: Sparks of Hope
  • Metroid Dread
  • Metroid Prime Remastered
  • Pokemon Violet
  • Pokemon Sword
  • SMT 5
  • Super Mario Odyssey
  • TloZ: Link's Awakening
  • TloZ: Skyward Sword HD
  • TloZ: Breath of The Wild
  • Xenoblade Chronicles DE
  • Xenoblade Chronicles 2
  • Xenoblade Chronicles 3
  • Yo Kai Watch 4