Document best practices for debugging/profiling.

[dvc94ch]

These are things that I keep getting hung up on. It is clear how gdb/perf work for example, by using the ptrace/open_perf_event syscalls in the linux kernel. However when it comes to gpu programming there seems to be few tools that aren't vendor specific and it is completely unclear how they work to a non-gpu expert. To open up gpu programming to a larger audience, how to perform debugging/profiling tasks should be documented better.

I presume there is some sort of synchronizing primitive in vulkan that could be used for debugging to step through the code, but how would you sample the gpu stack for example?

[follower]

Tool 1: Renderdoc

In case it's helpful, I'll mention that I've recently (again) found Renderdoc to be helpful with regard to debugging--in my case, specifically in relation to Vulkan Compute shaders.

It supports multiple platforms & multiple APIs: https://github.com/baldurk/renderdoc/blob/v1.x/README.md#api-support

The home page also specifically mentions a focus on usability:

Usability Focus

Usability matters. Tools should have a low barrier to entry and be easy to use and understand.
RenderDoc makes the process of getting started as smooth as possible, and simplifies common workflows.

One useful starting point is probably the Window (i.e. UI) reference

Apparently the Vulkan Compute shader debugging support is "experimental" but I found it useful when trying to understand some unexpected behaviour & could e.g. single-step & live edit/reload: https://renderdoc.org/docs/how/how_debug_shader.html#debugging-a-compute-thread

Tool 2: printf

Also, Renderdoc supports the e.g. Vulkan "debug printf" functionality and one thing that wasn't obvious is that it can even provide printf support even if the underlying driver/GPU doesn't support all the extensions which the Vulkan Validation layers printf version requires. (Because Renderdoc has its own implementation.)

The Renderdoc documentation also makes reference to performance counters, callstacks & timeline/timing.

And while Renderdoc provides a tool-focused approach to problem solving initially, I imagine the source code would also go some way to showing you how it interacts with the GPU in order to do so.

Tool 3: Vulkan Layers, especially Validation

As a general Vulkan observation, my impression is that there's a focus on the use of their "layers" architecture to provide necessary debugging support, e.g. for Validation, performance or other properties and insight into what is happening on the GPU, via various layers configured e.g. via the Vulkan SDK's vkconfig GUI: https://github.com/LunarG/VulkanTools/blob/a7ab1708549a11bbff3eddc0d70d63859ca93948/vkconfig/README.md#using-the-vulkan-configurator

Tool 4: Vulkan "debug utils" extension

And, then, building on that, an approach of encouraging use of their more recent "debug utils" extension by applications in order to annotate objects & automatically "mark" progress through various pipelines, e.g. see PDF linked from: https://www.lunarg.com/new-tutorial-for-vulkan-debug-utilities-extension/

Although also, based on their ecosystem survey analysis it does seem like Vulkan/Khronos/LunarG do acknowledge there's a lot more that could be done to make Vulkan development more accessible.

Wrapping up

Anyway, I've read far too much about all this over the past couple of days, so hopefully there's something useful in these pointers for someone. :D

Definitely check out Renderdoc if you haven't already though--it can display debugging information in many pretty colours. :)