Home

General things to take into account:

In the bind_pipeline we get the shader handle that's to be bound. We have to store that with the current active pipeline. DX12 can have multiple bound shaders.

If a drawcall (draw or drawindexed event) is received, check if the bound shaders are in the list to be blocked. If so, return true from the event to make the drawcall a no-op.

DX12: A pipeline can contain multiple shader stages (as is the case in D3D12, though it in each possible stage can only occur once). To reverse back what shaders it contains, you can register for the init_pipeline event, which is called after a pipeline was created and gives you both the handle of that pipeline + the shader stages and code it contains (via multiple pipeline_subobject items, one for each shader stage). Then create a map from pipeline handle to that shader list and back in bind_pipeline you can lookup the pipeline in that map to know which shaders it binds.

struct my_pipeline_info {
    uint32_t vs_hash = 0;
    uint32_t ps_hash = 0;
};
// Better not make this global, but part of a struct stored per device via device->create/get_private_data()
std::unordered_map<uint64_t, my_pipeline_info> g_pipelines;

void on_init_pipeline(reshade::api::device *, reshade::api::pipeline_layout, uint32_t subobject_count, const reshade::api::pipeline_subobject *subobjects, reshade::api::pipeline pipeline) {
    my_pipeline_info info;
    for (uint32_t i = 0; i < subobject_count; ++i) {
        switch (subobjects[i].type) {
        case reshade::api::pipeline_subobject_type::vertex_shader:
            // Pipeline contains a vertex shader, get shader binary and calcuate its hash
            const reshade::api::shader_desc *vertex_shader = static_cast<const reshade::api::shader_desc *>(subobjects[i].data);
            info.vs_hash = calc_hash(vertex_shader->code, vertex_shader->code_size);
            break;
        case reshade::api::pipeline_subobject_type::pixel_shader:
            // Pipeline contains a pixel shader, get shader binary and calcuate its hash
            const reshade::api::shader_desc *vertex_shader = static_cast<const reshade::api::shader_desc *>(subobjects[i].data);
            info.vs_hash = calc_hash(vertex_shader->code, vertex_shader->code_size);
            break;
        // ...
        }
    }
    g_pipelines[pipeline.handle] = info;
}

void on_bind_pipeline(reshade::api::command_list *cmd_list, reshade::api::pipeline_stage stages, reshade::api::pipeline pipeline) {
    my_pipeline_info info = g_pipelines[pipeline.handle];
    if (info.vs_hash != 0) {
        // Pipeline contains a vertex shader
    }
    if (info.ps_hash != 0) {
        // Pipeline contains a pixel shader
    }
}

Lock ideas: You probably won't get around some locking. But Windows has very lighweight locks that are pretty much unnoticeable if used correctly (https://docs.microsoft.com/en-us/windows/win32/sync/slim-reader-writer--srw--locks). Better to use those instead of Critical Sections for these purposes, especially since they also allow multiple readers to enter simultaneously! And if you prefer STL over raw Win32 (like me): std::shared_mutex uses these internally and is much less code to write with std::shared_lockstd::shared_mutex for reads and std::unique_lockstd::shared_mutex for writes (https://en.cppreference.com/w/cpp/thread/shared_mutex, https://en.cppreference.com/w/cpp/thread/shared_lock + https://en.cppreference.com/w/cpp/thread/unique_lock).

Also, a tip, instead of thread local storage for command list specific data, might also be able to use private data stored per command list:

struct __declspec(uuid("A GUID, e.g. created via Visual Studio - TOOLS - Create GUID")) my_per_command_list_data {
    std::vector<int> stuff;
};

void on_create_command_list(reshade::api::command_list *cmd_list) {
    // Attach default initialized "my_per_command_list_data" struct to the command list
    cmd_list->create_private_data<my_per_command_list_data>();
}
void on_destroy_command_list(reshade::api::command_list *cmd_list) {
    cmd_list->destroy_private_data<my_per_command_list_data>();
}

void on_bind_pipeline(reshade::api::command_list *cmd_list, reshade::api::pipeline_stage stages, reshade::api::pipeline pipeline) {
    // Get the "my_per_command_list_data" data for this command list
    my_per_command_list_data &data = cmd_list->get_private_data<my_per_command_list_data>();
    // An application must access a command list from a single thread only, so this is safe without locking
    data.stuff.push_back(1337);
}