Max number for dynamic resources

[shaoboyan]

Current spec support dynamic resources. The number of dynamic resources needs to be limited because they may consume special resources , which are not unlimit in native backends, to achieve performance improvement through dynamic offsets.

Here are current state in Metal, Vulkan and D3D12 :

Metal

In metal, resources can achieve dynamic offsets through SetVertexBufferOffset in vertex shader, SetFragmentBufferOffset in fragment shader and SetBufferOffset in compute shader.

No specific limitation found in Metal for applying these API to buffers.

D3D12

In D3D12, resources can achieve dynamic offsets and gain performance improvement through root descriptor.

Each root descriptor takes 2 DWORDs in root signature, and the maximum size of a root signature is 64 DWORDs.

In current spec, the maxBindGroup in WebGPU is 4 which means that they will take 4 DWORDs in root signature(if they're root descriptor heaps). So the max number of dynamic resources cannot exceed 30.

Vulkan

Vulkan has dynamic resources types. It has dynamic uniform buffer and dynamic storage buffer to achieve dynamic offsets.

The min-max numbers of dynamic uniform buffer and dynamic storage buffer that can be set are 8 and 4 (Vulkan spec Table 30).

Proposal

According to above information, Vulkan is the one of all three native backends that has specific max number limitation for dynamic uniform buffer and dynamic storage buffer. The total number of dynamic resources cannot exceed 12 in Vulkan backend. This limitation is compatible with D3D12, which requests number of dynamic resources cannot exceed 30 and Metal, which has no specific limitation.

So WebGPU can defines two max numbers for whole pipeline :

• maxUniformBuffersDynamic is 8
• maxStorageBuffersDynamic is 4

[kvark]

Thank you for the investigation!

The total number of dynamic resources cannot exceed 12 in Vulkan backend.

Note that we can't just have the total number limit, we'll need to follow the limit of storage and non-storage separately, and those are per bind group:

• maxDescriptorSetStorageBuffersDynamic starts at 4
• maxDescriptorSetUniformBuffersDynamic starts at 8

No specific limitation found in Metal for applying these API to buffers.

It comes down to a question of how dynamic offsets are implemented for Metal argument buffers. Our plan for wgpu-native was to have them bound as separate buffer bindings (thus being able to use SetVertexBufferOffset and SetFragmentBufferOffset that you mentioned. I think this is the most straightforward path, and it would be nice if WebGPU API allow for such an implementation to be valid. Question to Apple and Google - are you planning on providing the dynamic offsets differently? I can imagine multiple ways, but none of them are pretty.

So, let's see how many dynamic buffers we can bind separately from the argument buffers. Metal has a limit on the total number of buffers bound per stage, which is 31. If we divide this by the number of the bind groups, we get 7 buffers per group. One binding is taken by the argument buffer of a group itself (if we assume that argument buffers are required), so 6 is left.

This number does not take into account the vertex buffers, which we want to be 16 if I'm recalling correctly... so for the vertex shaders, we have (31 - 16 - 1) / 4 = 3 unoccupied buffer bindings per bind group.

Thus, we end up with at most 3 dynamic buffers per bind group that is visible in the vertex shader, and up to 6 dynamic buffers for other bind groups.

[shaoboyan]

@kvark thanks for your comments!

Note that we can't just have the total number limit, we'll need to follow the limit of storage and non-storage separately, and those are per bind group:

I think the maximum numbers are pipeline layout level according to Vulkan spec in VkPipelineLayoutCreateInfo:

The total number of descriptors of the type VK_DESCRIPTOR_TYPE_UNIFORM_BUFFER_DYNAMIC accessible across all shader stages and across all elements of pSetLayouts must be less than or equal to VkPhysicalDeviceLimits::maxDescriptorSetUniformBuffersDynamic

So, let's see how many dynamic buffers we can bind separately from the argument buffers. Metal has a limit on the total number of buffers bound per stage, which is 31.

Great analysis ! One thing I'd like to discuss is that whether we should define different max numbers for dynamic resources per bind group (e.g. 3 as max number of dynamic resrouces per bind group) or use the same max number of dynamic resrources the whole pipelineLayout has(e.g. 8 dynamic uniform buffers and 4 dynamic storage buffers per bind group).

Take Metal as example, I think, for developers, maybe make sure the sum of all max numbers of buffer resources shouldn't exceed 31 is more flexible than average these resources into bind groups and define a max number per bind group.

But it seems that current max values for dynamic resources need to be discussed more, because 8 + 4 + 4(argument buffer) + 16(max vertex buffer) = 32 > 31

WDYT?

[kvark]

I think the maximum numbers are pipeline layout level according to Vulkan spec in VkPipelineLayoutCreateInfo:

Indeed, thanks for correction! That makes our life much simpler. Defining the limit as 8 dynamic uniform buffers and 4 dynamic storage buffers total per pipeline layout appears to fit all the backends 👍

[Kangz]

Closing since this investigation has been implemented in the spec.