[WIP] D3D12: Handle descriptor pool re-use, PSO disk cache issues

[stenzek]

D3D12 does not handle running out of descriptors when the temporary pool is exhausted, which resulted in graphical corruption or driver resets when this occurred within a single frame.

This also sorts out the huge memory usage on recent AMD drivers (2-3GB memory usage after starting a game).

On AMD, the PSO cache seems to have issues when recreating the pipeline objects after reloading, in the earlier driver releases it caused bugchecks, and in the latest release seems to crash within the driver. I've double-checked that the state matches exactly when re-creating the PSO, which it does, but it is possible I've missed something, so I'm marking this WIP until I get a chance to investigate further.

Relevant issues:
https://bugs.dolphin-emu.org/issues/9701
https://bugs.dolphin-emu.org/issues/9652
https://bugs.dolphin-emu.org/issues/9640 (possibly)
https://bugs.dolphin-emu.org/issues/9461 (possibly)

---

This change is

[lioncash]

Review status: 0 of 15 files reviewed at latest revision, 3 unresolved discussions.

---

Source/Core/VideoBackends/D3D12/D3DDescriptorHeapManager.cpp, line 112 [r1] (raw file):

  while (!m_fences.empty())
  {
    auto& iter = m_fences.begin();

This can likely also be const.

---

Source/Core/VideoBackends/D3D12/D3DDescriptorHeapManager.cpp, line 189 [r1] (raw file):

  if (!this_ptr->m_fences.empty() && this_ptr->m_fences.back().second == current_index)
    return;

Unnecessary spaces

---

Source/Core/VideoBackends/D3D12/D3DDescriptorHeapManager.cpp, line 272 [r1] (raw file):

{
  // D3D12TODO: Do something more efficient than this.
  return (!memcmp(&lhs, &rhs, sizeof(D3DSamplerHeapManager::SamplerStateSet)));

Please use something like std::tie()or compare the members sequentially instead of using memcmp. Transient padding bits shouldn't be a factor in constituting equality. It also inlines better than memcmp.

---

Comments from Reviewable

[RisingFog]

Any updates on this PR?

[Fallcrest]

I guess this should be closed now that D3D12 has been removed as of #4424.

[leoetlino]

Closing as D3D12 was dropped