Poor performance

[janekb04]

The performance overhead imposed by Diligent Engine seems to be much larger than expected.
For comparison, here are the frame times for the Asteroid demo (compiled with Visual Studio 2019 Release x64) from my computer (AMD Ryzen 7 2700X Eight-Core Processor, NVIDIA GeForce GTX 1660 Ti):
Native Direct3D11 - 6.5ms
Native Direct3D12 - 2ms
Diligent Engine (Direct3D11 backend) - 5.5ms
Diligent Engine (Direct3D12 backend) - 3.5ms
Diligent Engine (Vulkan backend) - 3ms

Native Direct3D12 version is implemented differently: instead of binding every texture before every draw call, it binds all textures once and then uses bindless approach where the texture is selected in the shader, so it is not apple-to-apple comparison. You can do exactly the same in Diligent, but the goal here was to stress test resource binding system.
This distinction should be made more clear. Or rather, the same bindless approach should be shown in Diligent D3D12 and Vulkan modes.

[janekb04]

Ok, thank you for clarification. So basically it’s the “fault” of the demo implementation and not of the engine itself.

I guess you can say so. The demo is primarily used as performance testbed. There are multiple modes that can be toggled with the keyboard, but the fact the native d3d12 mode is not identical is indeed confusing. Will fix that.
Speaking about performance in general, you should not expect miracles. The API gives you tools to create very efficient applications, but when not used properly, the performance will suffer, much like with D3D12, Vulkan or any other native API.

@janekb04 I reworked the demo so that by default it runs in bindless mode + made few performance improvements and updates to the API. As a result, both Diligent D3D12 and Vulkan modes now run even faster than original native D3D12 implementation, which is not bad given that below is the entire main loop of the native implementation:

auto constantsPointer = frame->mDrawConstantBuffersGPUVA + sizeof(DrawConstantBuffer) * drawStart;
for (UINT drawIdx = drawStart; drawIdx < drawEnd; ++drawIdx)
{
    auto staticData = &staticAsteroidData[drawIdx];
    auto dynamicData = &dynamicAsteroidData[drawIdx];

    XMStoreFloat4x4(&drawConstantBuffers[drawIdx].mWorld, dynamicData->world);
    XMStoreFloat4x4(&drawConstantBuffers[drawIdx].mViewProjection, viewProjection);

    // Set root cbuffer
    //cmdLst->SetGraphicsRootDescriptorTable(RP_TEX_SRV, mSRVDescs->GPU(0));
    cmdLst->SetGraphicsRootConstantBufferView(RP_DRAW_CBV, constantsPointer);
    constantsPointer += sizeof(DrawConstantBuffer);
                    
    cmdLst->DrawIndexedInstanced(dynamicData->indexCount, 1, dynamicData->indexStart, staticData->vertexStart, 0);
}

Can you get the latest and try it on your side? (Don't forget to recursively update all submodules)

[janekb04]

After the update I performed the benchmark again, but this time in two different views:

    View from above
Backend |  Old  |  New
------------------------
  D3D11 |  6.5  |   7
  D3D12 |   2   |   2
DED3D11 |  5.5  |   6
DED3D12 |  3.5  |  2.5
   DEVK |   3   |   2

Close up look (default)
Backend |  Old  |  New
------------------------
  D3D11 |   -   |  6.8
  D3D12 |   -   |  2.5 
DED3D11 |   -   |  5.7
DED3D12 |   -   |  2.5
   DEVK |   -   |   2

Last time I only bench-marked only the view from above.
It seems that the overall performance has improved and in the close up look the native implementation and the DE version are almost equally fast.
Unfortunately, in the view from above, the native implementations are still faster, but the overhead has decreased from 75% to 25%.
As a side-note, it seems that the Direct3D11 back-end speed has decreased (both native and DE) and that Vulkan is both the fastest and has the most stable frame-time of all of the implementations.

Thank you for benchmarking this and sharing the numbers!
Few thoughts on performance:

• Two years ago when I first published the demo, its native D3D12 implementation was running at 320 fps on my NV GTX970 and Intel Core i7-4790K 4-core 8-thread CPU. Over time, with Windows and driver updates, the performance was steadily decreasing and now it is running at 240 fps. I have very little idea of what MSFT and NVidia were doing

• The performance of the demo may interfere with other processes as it is trying to load all CPU cores. Sometimes it just drops and I have to reboot my machine to restore it.

• There should be no dependence on the view, and I don't know how to explain it. The demo does not do any view frustum culling and every frame issues the same 50K draw calls no matter what.

• The demo reports three numbers: X (Y / Z). X is the overall frame time. Y is the asteroid update time and it should be more or less the same for all implementations except for native D3D11 that does not support multithreading. Z is the rendering command recording time alone, which reflects only draw command issuing.

• You can toggle multi-threaded mode with 'm' and change the number of threads with '+' and '-'. One fact that has concerned me from the beginning is that parallel command recording never scales as much as asteroid update. In particular, I see almost no scaling of command list recording time for native D312 implementation. This suggests that there are still some dependencies somewhere in D3D12/driver

Few other thoughts regarding the perfomance:

• From my experiments I estimate the pure overhead that Diligent imposes over the same series of native commands as 25%-30%, which I believe is totally acceptable
• The overhead Diligent introduces should scale perfectly with the number of cores because it has no dependencies on driver.
• In absolute numbers, Diligent easily lets you issue 50000 draw calls in ~1 ms command recording time so it can barely be called low-performing. On the other hand, just because you can issue that many draw calls I believe you absolutely shouldn't. Especially when D3D12 and Vulkan allow many ways to reduce the draw call number with bindless resources and other capabilities.