D3D12 Bindless Compute PSO replay crash

[yunmnmn]

Description

Since yesterday, I've been getting a crash when I click on a ComputePSO Dispatch in D3D12. The shader uses ResourceDescriptorHeap[] to access resources. If I don't access resources through ResourceDescriptorHeap[], no crash occurs.

I happen to have an old capture (captured prior to when the error occurred), and tried to view the same Dispatch pass, but the crash also occurred there. It seems that the crash isn't dependent on the RenderDoc version or capture.

On my secondary machine everything works fine with the latest stable RenderDoc builds (v1.26 & v1.27)

I dipped my toe into RenderDoc's source to hopefully identify the issue(all code references are from the latest v1.x branch):

• IDxcValidator outputs validation warnings and errors in ~ProgramEditor() (dxil_bytecode_editor.cpp, line 103)

dxil_bytecode_editor.cpp( 188) - Warning - DXIL validation failed: error: Validator version in metadata (1.7) is not supported; maximum: (1.6).
dxil_bytecode_editor.cpp( 188) - Warning - error: Named metadata '<null>' is unknown.
dxil_bytecode_editor.cpp( 188) - Warning - Function: dx.op.binary.i32: error: External function 'dx.op.binary.i32' is unused.
dxil_bytecode_editor.cpp( 188) - Warning - Validation failed.

• The "error: Named metadata '<null>' is unknown" caught my attention. When looking into which functions add entries to m_NamedMeta, there is a helper function (ProgramEditor::CreateNamedMetadata()) that adds entries without setting the name
• Setting the name resolved the issue for me locally, and the validation error is also gone:

(dxil_bytecode_editor.cpp, line 352)
NamedMetadata *ProgramEditor::CreateNamedMetadata(const rdcstr &name)
{
  for(NamedMetadata *m : m_NamedMeta)
    if(m->name == name)
      return m;

  m_NamedMeta.push_back(new(alloc) NamedMetadata);

  // <edit
  m_NamedMeta.back()->name = name;
  // edit>
  return m_NamedMeta.back();
}

I'm unsure if this is a proper fix, or just luck. The opcode injection is pretty involved.

Steps to reproduce

1. Create a bindless Compute PSO with a shader that accesses resources through ResourceDescriptorHeap[]
2. Take a capture and replay
3. Click on the Compute PSO Dispatch

I can provide a capture if necessary

Environment

• RenderDoc version: v1.24, v1.26, v1.27 and local build
• Operating System: Windows 11 (Nvidia driver version: 536.40, latest as of writing)
• Graphics API: D3D12 (with DX12 Agility (tried with 600 & 610), Nvidia RTX 3070 and AMD RX 580)

[baldurk]

I do have a test that accesses resources through ResourceDescriptorHeap[] which doesn't hit this issue, but looking at it I suspect that's because there are some normal cbuffers used so the dx.resources metadata exists already.

Your investigation looks bang on to me and the fix is easily correct, the bug definitely present before. Since this works for you I'm happy to commit it without a direct repro. In future I might need to expand the test cases to include a "pure" direct heap access shader with no normal resources at all to stress this path.

[yunmnmn]

That makes a lot of sense. Thanks for the quick response!