Shader Debugging

UI interface

The shader debugging is requested from the UI layer by use of these functions:

bool ReplayRenderer_VSGetDebugStates(ReplayRenderer *rend, uint32_t vertid, uint32_t instid, uint32_t idx, uint32_t instOffset, uint32_t vertOffset, ShaderDebugTrace *trace);
bool ReplayRenderer_PSGetDebugStates(ReplayRenderer *rend, uint32_t x, uint32_t y, ShaderDebugTrace *trace);
bool ReplayRenderer_CSGetDebugStates(ReplayRenderer *rend, uint32_t groupid[3], uint32_t threadid[3], ShaderDebugTrace *trace);

Each of these functions takes the relevant specific inputs and runs the currently bound shader with the currently bound state to completion.

These functions are implemented by the IRemoteDriver currently running via DebugVertex, DebugPixel, DebugThread and return a ShaderDebugTrace.

Each ShaderDebugTrace contains a list of all the inputs (if relevant), constants, as well as a list of states. The list of states begins with the state of the shader before the first instruction is executed, and one state exists for every instruction that is executed thereafter, containing the state of all mutable registers as well as which instruction is next to be executed (to account for loops and other control flow).

In the UI currently these get launched from TextureViewer, BufferViewer or PipelineStateViewer depending on the shader stage. When the trace comes back, a new ShaderViewer is created and passed the trace which it then displays and allows the user to step forward or backwards freely without any further simulation.

D3D11

The D3D11 interface is in d3d11_analyse.cpp, where D3D11DebugManager implements each of the Debug??? functions.

These functions prepare the initial state. They call out to CreateShaderDebugState to create all the mutable registers in the state and fetch the cbuffer contents, then each function fills in the inputs in a stage-specific way. CreateShaderGlobalState is also used to prefetch contents of bound buffer resources.

DebugThread is easy as it just fills in the thread IDs and such, there aren't any other inputs to compute shaders.

DebugVertex examines the inputs bound to the IA and interprets them by matching up the input layout to the vertex shader.

DebugPixel re-runs the drawcall on the GPU with a custom bespoke pixel shader bound. This pixel shader checks SV_Positon against the desired x/y. For each matching pixel it allocates a structure from a UAV via an atomic counter, then writes all of the inputs for the desired pixel as well as the derivatives for all the inputs into that structure. After this draw we map the contents of the UAV back onto the CPU and find the pixel drawn that passes the depth test last and generate a quad of pixels from the derivatives. We have to simulate 4 pixels so that the pixel we want to debug can calculate derivatives.

Each of the debug functions then sits in a loop, taking the latest state that it has and calling into GetNext which calculates the state after the next instruction executes, and pushes it back onto its list. Once this list is complete we can return it in the trace.

State::GetNext inside dxbc_debug.cpp fetches out the source operands and does a switch statement to actually implement each opcode. I won't go into the details here of each opcode but I will highlight a couple of things:

• GetSrc and SetDst are utility functions that handle swizzling, saturates, absolutes, etc. This keeps those details out of the actual opcode implementations for the most part.
• Note that when an opcode is being done scalar, the result (and temporaries) are expected to be in .x of the ShaderVariable in spite of any swizzling that goes on.
• Texture lookups, samples, etc are done by creating a shader that hard-codes the derivatives and texture co-ordinates in-line and does a single texture sample, we then render a quad to a 1x1 pixel target and map it out to get the result. This way we re-use the GPUs texture units rather than having to implement them ourselves. For a sample_c there isn't a good equivalent that lets us specify derivatives as well as everything else so we create a vertex shader that hard-codes the derivative values to each corner.