Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds push const ranges in pipelinelayout to fix #168 #174

Merged
merged 18 commits into from Mar 6, 2021

Conversation

axsaucedo
Copy link
Member

@axsaucedo axsaucedo commented Mar 4, 2021

Fixes #168
Fixes #175

@axsaucedo axsaucedo added bug Something isn't working c++ labels Mar 4, 2021
@axsaucedo axsaucedo added this to In progress in 0.7.0 via automation Mar 4, 2021
@axsaucedo axsaucedo added this to In progress in 0.6.1 via automation Mar 4, 2021
@unexploredtest
Copy link
Contributor

Ok um, this is getting more and more confusing. I no longer get a segfault with AMDVLK. However, the tests fail with both drivers:

[alipmpaint@archlinux test]$ ./test_kompute
Running main() from /home/alipmpaint/Documents/github/vulkan-kompute/external/googletest/googletest/src/gtest_main.cc
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from TestPushConstants
[ RUN      ] TestPushConstants.TestConstantsAlgoDispatchOverride
[2021-03-04 14:47:35.746] [info] [Shader.cpp:68] Kompute Shader Information: 

WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-04 14:47:35.898] [info] [Manager.cpp:269] Using physical device index 1 found AMD Radeon HD 7700 Series
[2021-03-04 14:47:35.919] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 1 and spirv size: 292
[2021-03-04 14:47:35.919] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-04 14:47:35.919] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 1, Y: 1, Z: 1
[2021-03-04 14:47:35.927] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-04 14:47:35.928] [info] [Sequence.cpp:63] Kompute Sequence command recording END
/home/alipmpaint/Documents/github/vulkan-kompute/test/TestPushConstant.cpp:46: Failure
Expected equality of these values:
  tensor->data()
    Which is: { 0, 0, 0 }
  kp::Constants({ 0.4, 0.4, 0.4 })
    Which is: { 0.4, 0.4, 0.4 }
[2021-03-04 14:47:35.928] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-04 14:47:35.928] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-04 14:47:35.928] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-04 14:47:35.928] [info] [Manager.cpp:102] Destroying device
[2021-03-04 14:47:35.931] [warning] [Sequence.cpp:196] Kompute Sequence destroy called with null Device pointer
[  FAILED  ] TestPushConstants.TestConstantsAlgoDispatchOverride (278 ms)
[ RUN      ] TestPushConstants.TestConstantsAlgoDispatchNoOverride
[2021-03-04 14:47:36.020] [info] [Shader.cpp:68] Kompute Shader Information: 


WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-04 14:47:36.157] [info] [Manager.cpp:269] Using physical device index 1 found AMD Radeon HD 7700 Series
[2021-03-04 14:47:36.158] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 1 and spirv size: 292
[2021-03-04 14:47:36.158] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-04 14:47:36.158] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 1, Y: 1, Z: 1
[2021-03-04 14:47:36.158] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-04 14:47:36.158] [info] [Sequence.cpp:63] Kompute Sequence command recording END
/home/alipmpaint/Documents/github/vulkan-kompute/test/TestPushConstant.cpp:89: Failure
Expected equality of these values:
  tensor->data()
    Which is: { 0, 0, 0 }
  kp::Constants({ 0.4, 0.4, 0.4 })
    Which is: { 0.4, 0.4, 0.4 }
[2021-03-04 14:47:36.159] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-04 14:47:36.159] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-04 14:47:36.159] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-04 14:47:36.159] [info] [Manager.cpp:102] Destroying device
[2021-03-04 14:47:36.162] [warning] [Sequence.cpp:196] Kompute Sequence destroy called with null Device pointer
[  FAILED  ] TestPushConstants.TestConstantsAlgoDispatchNoOverride (230 ms)
[ RUN      ] TestPushConstants.TestConstantsWrongSize
[2021-03-04 14:47:36.251] [info] [Shader.cpp:68] Kompute Shader Information: 


WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-04 14:47:36.380] [info] [Manager.cpp:269] Using physical device index 1 found AMD Radeon HD 7700 Series
[2021-03-04 14:47:36.381] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 1 and spirv size: 292
[2021-03-04 14:47:36.381] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-04 14:47:36.381] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 1, Y: 1, Z: 1
[2021-03-04 14:47:36.389] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-04 14:47:36.389] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-04 14:47:36.389] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-04 14:47:36.389] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-04 14:47:36.389] [info] [Manager.cpp:102] Destroying device
[2021-03-04 14:47:36.392] [warning] [Sequence.cpp:196] Kompute Sequence destroy called with null Device pointer
[       OK ] TestPushConstants.TestConstantsWrongSize (230 ms)
[----------] 3 tests from TestPushConstants (739 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 1 test suite ran. (739 ms total)
[  PASSED  ] 1 test.
[  FAILED  ] 2 tests, listed below:
[  FAILED  ] TestPushConstants.TestConstantsAlgoDispatchOverride
[  FAILED  ] TestPushConstants.TestConstantsAlgoDispatchNoOverride

 2 FAILED TESTS

(the tests pass with Intel's GPU)
However, this is not the end of the confusion. With the code provided in the new README, I get the correct results with AMDVLK:

WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-04 14:53:22.044] [info] [Manager.cpp:269] Using physical device index 1 found AMD Radeon HD 7700 Series
[2021-03-04 14:53:22.139] [info] [Shader.cpp:68] Kompute Shader Information: 


[2021-03-04 14:53:22.141] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 4 and spirv size: 480
[2021-03-04 14:53:22.141] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-04 14:53:22.141] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 3, Y: 1, Z: 1
[2021-03-04 14:53:22.141] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-04 14:53:22.142] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-04 14:53:22.142] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-04 14:53:22.142] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-04 14:53:22.142] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-04 14:53:22.142] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-04 14:53:22.142] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-04 14:53:22.142] [info] [Sequence.cpp:63] Kompute Sequence command recording END
4  8  12  10  10  10  [2021-03-04 14:53:22.142] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-04 14:53:22.142] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-04 14:53:22.142] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-04 14:53:22.142] [info] [Manager.cpp:102] Destroying device

but with AMD Mesa's vulkan, I get these weird results:

WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-04 14:54:47.582] [info] [Manager.cpp:269] Using physical device index 2 found AMD RADV VERDE (ACO)
[2021-03-04 14:54:47.683] [info] [Shader.cpp:68] Kompute Shader Information: 


[2021-03-04 14:54:47.686] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 4 and spirv size: 480
[2021-03-04 14:54:47.686] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-04 14:54:47.686] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 3, Y: 1, Z: 1
[2021-03-04 14:54:47.687] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-04 14:54:47.687] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-04 14:54:47.687] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-04 14:54:47.687] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-04 14:54:47.687] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-04 14:54:47.687] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-04 14:54:47.687] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-04 14:54:47.687] [info] [Sequence.cpp:63] Kompute Sequence command recording END
2  4  6  6  4  6  [2021-03-04 14:54:47.687] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-04 14:54:47.687] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-04 14:54:47.687] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-04 14:54:47.687] [info] [Manager.cpp:102] Destroying device

It's definitely an improvement I guess

@axsaucedo
Copy link
Member Author

axsaucedo commented Mar 4, 2021

Interesting... ok it does seem much more positive - looking at the results given that you have {6, 4, 6} it suggests that there may be a race condition... Ok, maybe could you try the following edit to the readme for your AMD mesa vulkan driver:

    mgr.sequence()->eval<kp::OpTensorSyncDevice>(params)
    mgr.sequence()->eval<kp::OpAlgoDispatch>(algorithm);
    mgr.sequence()->eval<kp::OpAlgoDispatch>(algorithm, pushConstsB);

This would basically add enough time for the driver to avoid any potential race condition. Not sure why there is a race condition as I do believe there are memory barriers, but perhaps there are not the right memory barriers between the first dispatch and the second dispatch...

This doesn't fully explain why the tests are failing tho, are you not able to make the tests pass with either of your drivers? I'm wondering if it indeed is a race condition it may be that we'd need to add memory barriers to ensure the push constants are in memory before the dispatch takes place.

@axsaucedo
Copy link
Member Author

It seems actualyl like we could get more insights on what is going under the hood using the VK_LAYER_LUNARG_api_dump , could you share the output as well?

@unexploredtest
Copy link
Contributor

unexploredtest commented Mar 4, 2021

Ok, maybe could you try the following edit to the readme for your AMD mesa vulkan driver:

Yeah that worked, now I get the correct results

It seems actualyl like we could get more insights on what is going under the hood using the

How to enable this layer?

@axsaucedo
Copy link
Member Author

Thank you for trying that - wow it does seem like there is some potential race condition happening... From initial research it seems the push constants shouldn't require memory barriers so not really sure how this could be ensured.

In regards to how to enable layers I need to have a deeper look for this

@axsaucedo
Copy link
Member Author

Ok I've added new functionality that allows you to add line separated desired debug layer names, so if you run the following before:

export KOMPUTE_ENV_DEBUG_LAYERS="VK_LAYER_LUNARG_api_dump"

And then run one of the tests, preferrably only the TestConstantsAlgoDispatchNoOverride then you should be able to share the logs of the full dump.

@axsaucedo
Copy link
Member Author

axsaucedo commented Mar 5, 2021

Ok it seems I have finally found the issue @aliPMPAINT (at least the latest one) - basically there seems to be a race condition in the shader as I am using the += operator which for some reason doesn't appear in my case but it does in yours - the solution is to use atomicAdd, which requires a GL extension, so I am going to update the code to allow for vulkan extensions to be added on manager initialisation so we can test this again

@unexploredtest
Copy link
Contributor

unexploredtest commented Mar 5, 2021

Sorry for the late reply

Ok I've added new functionality that allows you to add line separated desired debug layer names, so if you run the following before:

Nice! Gonna install it it.

Ok it seems I have finally found the issue @aliPMPAINT (at least the latest one) - basically there seems to be a race condition in the shader as I am using the += operator which for some reason doesn't

Aha I see....

ode to allow for vulkan extensions to be added on manager initialisation so we can test this again

Gonna test it

@axsaucedo
Copy link
Member Author

Ok I have now added a new iteration with atomicAdd, it seems that it does pass on my local NVIDIA with the atomicAdd, as the float atomicAdd seems to be part of the supported extensions by default, but unfortunately it seems that the swiftshader ICD that we use for the integration tests doesn't support this extension, so the test won't pass. What I will have to do is to change the tests to be run in multiple sequences to avoid the race condition, but it woudl still be good to confirm whether the tests work for you on the current push that contains the atomicAdd @aliPMPAINT

@unexploredtest
Copy link
Contributor

unexploredtest commented Mar 5, 2021

@axsaucedo Unfortunately, none of my Vulkan drivers support both extentions - AMDVLK supports neither of them and Mesa's vulkan has just VK_EXT_shader_atomic_float, and with both cases I get this:

[alipmpaint@archlinux test]$ ./test_kompute
Running main() from /home/alipmpaint/Documents/github/vulkan-kompute/external/googletest/googletest/src/gtest_main.cc
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from TestPushConstants
[ RUN      ] TestPushConstants.TestConstantsAlgoDispatchOverride
[2021-03-05 11:57:48.887] [info] [Shader.cpp:68] Kompute Shader Information: 


WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-05 11:57:49.049] [info] [Manager.cpp:298] Using physical device index 0 found AMD Radeon HD 7700 Series
[2021-03-05 11:57:49.049] [error] [Manager.cpp:367] Kompute Manager not all extensions were added: {}
[2021-03-05 11:57:49.051] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 1 and spirv size: 282
[2021-03-05 11:57:49.051] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-05 11:57:49.051] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 1, Y: 1, Z: 1
Segmentation fault (core dumped)

I'm gonna test the new layer

@axsaucedo
Copy link
Member Author

axsaucedo commented Mar 5, 2021

Ah that is unfortunate, it seems there is a lot of uncertainty in regards to the supported extensions across each GPU driver - the latest push basically updates such that the operations are evaluated on separate sequence runs, so they should be synchronous given they are managed by a fence. At least this PR did add two very useful additions which is basically the ability to add extra debug layers via env var, and add extra extensions via manager parameters.

@unexploredtest
Copy link
Contributor

The tests have now passed:

Running main() from /home/alipmpaint/Documents/github/vulkan-kompute/external/googletest/googletest/src/gtest_main.cc
[==========] Running 3 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 3 tests from TestPushConstants
[ RUN      ] TestPushConstants.TestConstantsAlgoDispatchOverride
[2021-03-05 12:09:20.490] [info] [Shader.cpp:68] Kompute Shader Information: 


WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-05 12:09:20.654] [info] [Manager.cpp:298] Using physical device index 0 found AMD Radeon HD 7700 Series
[2021-03-05 12:09:20.655] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 1 and spirv size: 292
[2021-03-05 12:09:20.655] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-05 12:09:20.655] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 1, Y: 1, Z: 1
[2021-03-05 12:09:20.655] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-05 12:09:20.655] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 12:09:20.655] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 12:09:20.656] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-05 12:09:20.656] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 12:09:20.656] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 12:09:20.656] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-05 12:09:20.656] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 12:09:20.656] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 12:09:20.656] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-05 12:09:20.656] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 12:09:20.656] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 12:09:20.656] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-05 12:09:20.656] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-05 12:09:20.656] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-05 12:09:20.656] [info] [Manager.cpp:107] Destroying device
[2021-03-05 12:09:20.659] [warning] [Sequence.cpp:196] Kompute Sequence destroy called with null Device pointer
[       OK ] TestPushConstants.TestConstantsAlgoDispatchOverride (269 ms)
[ RUN      ] TestPushConstants.TestConstantsAlgoDispatchNoOverride
[2021-03-05 12:09:20.754] [info] [Shader.cpp:68] Kompute Shader Information: 


WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-05 12:09:20.875] [info] [Manager.cpp:298] Using physical device index 0 found AMD Radeon HD 7700 Series
[2021-03-05 12:09:20.877] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 1 and spirv size: 292
[2021-03-05 12:09:20.877] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-05 12:09:20.877] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 1, Y: 1, Z: 1
[2021-03-05 12:09:20.877] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 12:09:20.877] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 12:09:20.877] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 12:09:20.877] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-05 12:09:20.877] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-05 12:09:20.877] [info] [Manager.cpp:107] Destroying device
[2021-03-05 12:09:20.880] [warning] [Sequence.cpp:196] Kompute Sequence destroy called with null Device pointer
[       OK ] TestPushConstants.TestConstantsAlgoDispatchNoOverride (221 ms)
[ RUN      ] TestPushConstants.TestConstantsWrongSize
[2021-03-05 12:09:20.975] [info] [Shader.cpp:68] Kompute Shader Information: 


WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-05 12:09:21.093] [info] [Manager.cpp:298] Using physical device index 0 found AMD Radeon HD 7700 Series
[2021-03-05 12:09:21.095] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 1 and spirv size: 292
[2021-03-05 12:09:21.095] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-05 12:09:21.095] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 1, Y: 1, Z: 1
[2021-03-05 12:09:21.095] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 12:09:21.095] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-05 12:09:21.095] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-05 12:09:21.095] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-05 12:09:21.095] [info] [Manager.cpp:107] Destroying device
[2021-03-05 12:09:21.098] [warning] [Sequence.cpp:196] Kompute Sequence destroy called with null Device pointer
[       OK ] TestPushConstants.TestConstantsWrongSize (218 ms)
[----------] 3 tests from TestPushConstants (708 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 1 test suite ran. (708 ms total)
[  PASSED  ] 3 tests.

thanks!

@axsaucedo
Copy link
Member Author

Great! Ok I will merge, it seems it was actually a combination of issues - but glad we got to the bottom of thsi, thank you for your input @aliPMPAINT , it's great to be able to have also the ability to get an insight on AMD cards as I only have an NVIDIA. Now also will only be a thing of being aware of race conditions, as well as tackling these with either atomicAdd or other supported approaches.

@unexploredtest
Copy link
Contributor

Thanks, it works fine now
Just one thing, about the extra layer, I'm not able to see the logs of the full dump after I export it...

@axsaucedo
Copy link
Member Author

Interesting, ok could you share the debug logs? I addeed a print statement that shows exactly which layers are the ones added at the end so this could give us confirmation whether its being provided

@unexploredtest
Copy link
Contributor

here we go

[alipmpaint@archlinux Build]$ ./kompute_array_mult
[2021-03-05 13:06:46.813] [debug] [Manager.cpp:142] Kompute Manager creating instance
[2021-03-05 13:06:46.960] [debug] [Manager.cpp:223] Kompute Manager Instance Created
[2021-03-05 13:06:46.960] [debug] [Manager.cpp:275] Kompute Manager creating Device
WARNING: radv is not a conformant vulkan implementation, testing use only.
[2021-03-05 13:06:46.962] [info] [Manager.cpp:298] Using physical device index 1 found AMD RADV VERDE (ACO)
[2021-03-05 13:06:46.962] [debug] [Manager.cpp:350] Kompute Manager desired extension layers {}
[2021-03-05 13:06:46.962] [debug] [Manager.cpp:359] Kompute Manager available extensions {"VK_AMD_buffer_marker", "VK_AMD_device_coherent_memory", "VK_AMD_draw_indirect_count", "VK_AMD_gcn_shader", "VK_AMD_memory_overallocation_behavior", "VK_AMD_mixed_attachment_samples", "VK_AMD_shader_ballot", "VK_AMD_shader_core_properties", "VK_AMD_shader_core_properties2", "VK_AMD_shader_explicit_vertex_parameter", "VK_AMD_shader_fragment_mask", "VK_AMD_shader_image_load_store_lod", "VK_AMD_shader_info", "VK_AMD_shader_trinary_minmax", "VK_AMD_texture_gather_bias_lod", "VK_EXT_4444_formats", "VK_EXT_buffer_device_address", "VK_EXT_calibrated_timestamps", "VK_EXT_conditional_rendering", "VK_EXT_custom_border_color", "VK_EXT_depth_clip_enable", "VK_EXT_depth_range_unrestricted", "VK_EXT_descriptor_indexing", "VK_EXT_discard_rectangles", "VK_EXT_display_control", "VK_EXT_extended_dynamic_state", "VK_EXT_external_memory_dma_buf", "VK_EXT_external_memory_host", "VK_EXT_global_priority", "VK_EXT_host_query_reset", "VK_EXT_image_robustness", "VK_EXT_inline_uniform_block", "VK_EXT_line_rasterization", "VK_EXT_memory_budget", "VK_EXT_memory_priority", "VK_EXT_pci_bus_info", "VK_EXT_pipeline_creation_cache_control", "VK_EXT_pipeline_creation_feedback", "VK_EXT_private_data", "VK_EXT_queue_family_foreign", "VK_EXT_robustness2", "VK_EXT_sample_locations", "VK_EXT_sampler_filter_minmax", "VK_EXT_shader_atomic_float", "VK_EXT_shader_demote_to_helper_invocation", "VK_EXT_shader_image_atomic_int64", "VK_EXT_shader_stencil_export", "VK_EXT_shader_subgroup_ballot", "VK_EXT_shader_subgroup_vote", "VK_EXT_shader_viewport_index_layer", "VK_EXT_subgroup_size_control", "VK_EXT_texel_buffer_alignment", "VK_EXT_transform_feedback", "VK_EXT_vertex_attribute_divisor", "VK_EXT_ycbcr_image_arrays", "VK_GOOGLE_decorate_string", "VK_GOOGLE_hlsl_functionality1", "VK_GOOGLE_user_type", "VK_KHR_16bit_storage", "VK_KHR_8bit_storage", "VK_KHR_bind_memory2", "VK_KHR_buffer_device_address", "VK_KHR_copy_commands2", "VK_KHR_create_renderpass2", "VK_KHR_dedicated_allocation", "VK_KHR_depth_stencil_resolve", "VK_KHR_descriptor_update_template", "VK_KHR_device_group", "VK_KHR_draw_indirect_count", "VK_KHR_driver_properties", "VK_KHR_external_fence", "VK_KHR_external_fence_fd", "VK_KHR_external_memory", "VK_KHR_external_memory_fd", "VK_KHR_external_semaphore", "VK_KHR_external_semaphore_fd", "VK_KHR_get_memory_requirements2", "VK_KHR_image_format_list", "VK_KHR_imageless_framebuffer", "VK_KHR_incremental_present", "VK_KHR_maintenance1", "VK_KHR_maintenance2", "VK_KHR_maintenance3", "VK_KHR_multiview", "VK_KHR_pipeline_executable_properties", "VK_KHR_push_descriptor", "VK_KHR_relaxed_block_layout", "VK_KHR_sampler_mirror_clamp_to_edge", "VK_KHR_sampler_ycbcr_conversion", "VK_KHR_separate_depth_stencil_layouts", "VK_KHR_shader_atomic_int64", "VK_KHR_shader_clock", "VK_KHR_shader_draw_parameters", "VK_KHR_shader_float16_int8", "VK_KHR_shader_float_controls", "VK_KHR_shader_non_semantic_info", "VK_KHR_shader_subgroup_extended_types", "VK_KHR_shader_terminate_invocation", "VK_KHR_spirv_1_4", "VK_KHR_storage_buffer_storage_class", "VK_KHR_swapchain", "VK_KHR_swapchain_mutable_format", "VK_KHR_timeline_semaphore", "VK_KHR_uniform_buffer_standard_layout", "VK_KHR_variable_pointers", "VK_KHR_vulkan_memory_model", "VK_NV_compute_shader_derivatives"}
[2021-03-05 13:06:46.964] [debug] [Manager.cpp:381] Kompute Manager device created
[2021-03-05 13:06:46.964] [debug] [Manager.cpp:395] Kompute Manager compute queue obtained
[2021-03-05 13:06:46.964] [debug] [Manager.cpp:401] Kompute Manager tensor creation triggered
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:11] Kompute Tensor constructor data length: 3, and type: 0
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:34] Kompute Tensor rebuilding with size 3
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:326] Kompute Tensor creating buffer
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:335] Kompute Tensor creating primary buffer and memory
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:376] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst | StorageBuffer }
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:396] Kompute Tensor allocating and binding memory
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:419] Kompute Tensor allocating memory index: 0, size 16, flags: { DeviceLocal }
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:348] Kompute Tensor creating staging buffer and memory
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:376] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst }
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:396] Kompute Tensor allocating and binding memory
[2021-03-05 13:06:46.964] [debug] [Tensor.cpp:419] Kompute Tensor allocating memory index: 1, size 16, flags: { HostVisible }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:361] Kompute Tensor buffer & memory creation successful
[2021-03-05 13:06:46.965] [debug] [Manager.cpp:401] Kompute Manager tensor creation triggered
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:11] Kompute Tensor constructor data length: 3, and type: 0
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:34] Kompute Tensor rebuilding with size 3
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:326] Kompute Tensor creating buffer
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:335] Kompute Tensor creating primary buffer and memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:376] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst | StorageBuffer }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:396] Kompute Tensor allocating and binding memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:419] Kompute Tensor allocating memory index: 0, size 16, flags: { DeviceLocal }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:348] Kompute Tensor creating staging buffer and memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:376] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:396] Kompute Tensor allocating and binding memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:419] Kompute Tensor allocating memory index: 1, size 16, flags: { HostVisible }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:361] Kompute Tensor buffer & memory creation successful
[2021-03-05 13:06:46.965] [debug] [Manager.cpp:401] Kompute Manager tensor creation triggered
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:11] Kompute Tensor constructor data length: 3, and type: 0
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:34] Kompute Tensor rebuilding with size 3
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:326] Kompute Tensor creating buffer
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:335] Kompute Tensor creating primary buffer and memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:376] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst | StorageBuffer }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:396] Kompute Tensor allocating and binding memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:419] Kompute Tensor allocating memory index: 0, size 16, flags: { DeviceLocal }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:348] Kompute Tensor creating staging buffer and memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:376] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:396] Kompute Tensor allocating and binding memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:419] Kompute Tensor allocating memory index: 1, size 16, flags: { HostVisible }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:361] Kompute Tensor buffer & memory creation successful
[2021-03-05 13:06:46.965] [debug] [Manager.cpp:401] Kompute Manager tensor creation triggered
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:11] Kompute Tensor constructor data length: 3, and type: 0
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:34] Kompute Tensor rebuilding with size 3
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:326] Kompute Tensor creating buffer
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:335] Kompute Tensor creating primary buffer and memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:376] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst | StorageBuffer }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:396] Kompute Tensor allocating and binding memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:419] Kompute Tensor allocating memory index: 0, size 16, flags: { DeviceLocal }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:348] Kompute Tensor creating staging buffer and memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:376] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:396] Kompute Tensor allocating and binding memory
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:419] Kompute Tensor allocating memory index: 1, size 16, flags: { HostVisible }
[2021-03-05 13:06:46.965] [debug] [Tensor.cpp:361] Kompute Tensor buffer & memory creation successful
[2021-03-05 13:06:47.068] [info] [Shader.cpp:68] Kompute Shader Information: 


[2021-03-05 13:06:47.068] [debug] [Shader.cpp:85] Kompute Shader all result messages: 



[2021-03-05 13:06:47.071] [debug] [Manager.cpp:421] Kompute Manager algorithm creation triggered
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:14] Kompute Algorithm Constructor with device
[2021-03-05 13:06:47.071] [info] [Algorithm.cpp:19] Kompute Algorithm initialising with tensor size: 4 and spirv size: 480
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:44] Kompute Algorithm rebuild started
[2021-03-05 13:06:47.071] [info] [Algorithm.cpp:407] Kompute OpAlgoCreate setting dispatch size
[2021-03-05 13:06:47.071] [info] [Algorithm.cpp:421] Kompute OpAlgoCreate set dispatch size X: 3, Y: 1, Z: 1
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:172] Kompute Algorithm createParameters started
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:187] Kompute Algorithm creating descriptor pool
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:208] Kompute Algorithm creating descriptor set layout
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:219] Kompute Algorithm allocating descriptor sets
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:225] Kompute Algorithm updating descriptor sets
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:245] Kompue Algorithm successfully run init
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:251] Kompute Algorithm createShaderModule started
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:258] Kompute Algorithm Creating shader module. ShaderFileSize: 480
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:266] Kompute Algorithm create shader module success
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:272] Kompute Algorithm calling create Pipeline
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:358] Kompute Algorithm Create Pipeline Success
[2021-03-05 13:06:47.071] [debug] [Manager.cpp:436] Kompute Manager sequence() with queueIndex: 0
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:11] Kompute Sequence Constructor with existing device & queue
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:270] Kompute Sequence creating command pool
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:286] Kompute Sequence Command Pool Created
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:292] Kompute Sequence creating command buffer
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:308] Kompute Sequence Command Buffer Created
[2021-03-05 13:06:47.071] [debug] [OpTensorSyncDevice.cpp:9] Kompute OpTensorSyncDevice constructor with params
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:253] Kompute Sequence record function started
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:32] Kompute sequence called BEGIN
[2021-03-05 13:06:47.071] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:257] Kompute Sequence running record on OpBase derived class instance
[2021-03-05 13:06:47.071] [debug] [OpTensorSyncDevice.cpp:29] Kompute OpTensorSyncDevice record called
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:120] Kompute Tensor copying data size 12.
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:120] Kompute Tensor copying data size 12.
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:120] Kompute Tensor copying data size 12.
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:120] Kompute Tensor copying data size 12.
[2021-03-05 13:06:47.071] [debug] [OpAlgoDispatch.cpp:10] Kompute OpAlgoDispatch constructor
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:253] Kompute Sequence record function started
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:32] Kompute sequence called BEGIN
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:35] Kompute Sequence begin called when already recording
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:257] Kompute Sequence running record on OpBase derived class instance
[2021-03-05 13:06:47.071] [debug] [OpAlgoDispatch.cpp:24] Kompute OpAlgoDispatch record called
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:364] Kompute Algorithm binding pipeline
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:369] Kompute Algorithm binding descriptor sets
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:383] Kompute Algorithm binding push constants size: 1
[2021-03-05 13:06:47.071] [debug] [Algorithm.cpp:397] Kompute Algorithm recording dispatch
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:79] Kompute sequence EVAL BEGIN
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:52] Kompute Sequence calling END
[2021-03-05 13:06:47.071] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 13:06:47.071] [debug] [OpTensorSyncDevice.cpp:42] Kompute OpTensorSyncDevice preEval called
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:234] Kompute Tensor local mapping tensor data to host buffer
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:234] Kompute Tensor local mapping tensor data to host buffer
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:234] Kompute Tensor local mapping tensor data to host buffer
[2021-03-05 13:06:47.071] [debug] [Tensor.cpp:234] Kompute Tensor local mapping tensor data to host buffer
[2021-03-05 13:06:47.071] [debug] [OpAlgoDispatch.cpp:49] Kompute OpAlgoDispatch preEval called
[2021-03-05 13:06:47.071] [debug] [Sequence.cpp:115] Kompute sequence submitting command buffer into compute queue
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncDevice.cpp:55] Kompute OpTensorSyncDevice postEval called
[2021-03-05 13:06:47.072] [debug] [OpAlgoDispatch.cpp:55] Kompute OpAlgoDispatch postSubmit called
[2021-03-05 13:06:47.072] [debug] [OpAlgoDispatch.cpp:10] Kompute OpAlgoDispatch constructor
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:253] Kompute Sequence record function started
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:32] Kompute sequence called BEGIN
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:257] Kompute Sequence running record on OpBase derived class instance
[2021-03-05 13:06:47.072] [debug] [OpAlgoDispatch.cpp:24] Kompute OpAlgoDispatch record called
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:364] Kompute Algorithm binding pipeline
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:369] Kompute Algorithm binding descriptor sets
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:383] Kompute Algorithm binding push constants size: 1
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:397] Kompute Algorithm recording dispatch
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:79] Kompute sequence EVAL BEGIN
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:52] Kompute Sequence calling END
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncDevice.cpp:42] Kompute OpTensorSyncDevice preEval called
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:234] Kompute Tensor local mapping tensor data to host buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:234] Kompute Tensor local mapping tensor data to host buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:234] Kompute Tensor local mapping tensor data to host buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:234] Kompute Tensor local mapping tensor data to host buffer
[2021-03-05 13:06:47.072] [debug] [OpAlgoDispatch.cpp:49] Kompute OpAlgoDispatch preEval called
[2021-03-05 13:06:47.072] [debug] [OpAlgoDispatch.cpp:49] Kompute OpAlgoDispatch preEval called
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:115] Kompute sequence submitting command buffer into compute queue
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncDevice.cpp:55] Kompute OpTensorSyncDevice postEval called
[2021-03-05 13:06:47.072] [debug] [OpAlgoDispatch.cpp:55] Kompute OpAlgoDispatch postSubmit called
[2021-03-05 13:06:47.072] [debug] [OpAlgoDispatch.cpp:55] Kompute OpAlgoDispatch postSubmit called
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:24] Kompute Sequence Destructor started
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:193] Kompute Sequence destroy called
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:214] Kompute Sequence Freed CommandBuffer
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:231] Kompute Sequence Destroyed CommandPool
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncDevice.cpp:21] Kompute OpTensorSyncDevice destructor started
[2021-03-05 13:06:47.072] [debug] [OpBase.hpp:28] Kompute OpBase destructor started
[2021-03-05 13:06:47.072] [debug] [OpAlgoDispatch.cpp:18] Kompute OpAlgoDispatch destructor started
[2021-03-05 13:06:47.072] [debug] [OpBase.hpp:28] Kompute OpBase destructor started
[2021-03-05 13:06:47.072] [debug] [OpAlgoDispatch.cpp:18] Kompute OpAlgoDispatch destructor started
[2021-03-05 13:06:47.072] [debug] [OpBase.hpp:28] Kompute OpBase destructor started
[2021-03-05 13:06:47.072] [debug] [Manager.cpp:436] Kompute Manager sequence() with queueIndex: 0
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:11] Kompute Sequence Constructor with existing device & queue
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:270] Kompute Sequence creating command pool
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:286] Kompute Sequence Command Pool Created
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:292] Kompute Sequence creating command buffer
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:308] Kompute Sequence Command Buffer Created
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncLocal.cpp:11] Kompute OpTensorSyncLocal constructor with params
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:72] Kompute Sequence calling clear
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:52] Kompute Sequence calling END
[2021-03-05 13:06:47.072] [warning] [Sequence.cpp:60] Kompute Sequence end called when not recording
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:253] Kompute Sequence record function started
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:32] Kompute sequence called BEGIN
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:44] Kompute Sequence command now started recording
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:257] Kompute Sequence running record on OpBase derived class instance
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncLocal.cpp:29] Kompute OpTensorSyncLocal record called
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:137] Kompute Tensor copying data size 12.
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:137] Kompute Tensor copying data size 12.
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:137] Kompute Tensor copying data size 12.
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:137] Kompute Tensor copying data size 12.
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:175] Kompute Tensor recording buffer memory barrier
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:52] Kompute Sequence calling END
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:63] Kompute Sequence command recording END
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncLocal.cpp:42] Kompute OpTensorSyncLocal preEval called
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:115] Kompute sequence submitting command buffer into compute queue
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncLocal.cpp:48] Kompute OpTensorSyncLocal postEval called
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncLocal.cpp:50] Kompute OpTensorSyncLocal mapping data into tensor local
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:207] Kompute Tensor mapping data from host buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:207] Kompute Tensor mapping data from host buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:207] Kompute Tensor mapping data from host buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:207] Kompute Tensor mapping data from host buffer
4  8  12  10  10  10  [2021-03-05 13:06:47.072] [debug] [Sequence.cpp:24] Kompute Sequence Destructor started
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:193] Kompute Sequence destroy called
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:202] Freeing CommandBuffer
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:214] Kompute Sequence Freed CommandBuffer
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:218] Destroying CommandPool
[2021-03-05 13:06:47.072] [debug] [Sequence.cpp:231] Kompute Sequence Destroyed CommandPool
[2021-03-05 13:06:47.072] [info] [Sequence.cpp:235] Kompute Sequence clearing operations buffer
[2021-03-05 13:06:47.072] [debug] [OpTensorSyncLocal.cpp:23] Kompute OpTensorSyncLocal destructor started
[2021-03-05 13:06:47.072] [debug] [OpBase.hpp:28] Kompute OpBase destructor started
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:32] Kompute Algorithm Destructor started
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:83] Kompute Algorithm Destroying pipeline
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:95] Kompute Algorithm Destroying pipeline cache
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:107] Kompute Algorithm Destroying pipeline layout
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:119] Kompute Algorithm Destroying shader module
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:145] Kompute Algorithm Destroying Descriptor Set Layout
[2021-03-05 13:06:47.072] [debug] [Algorithm.cpp:157] Kompute Algorithm Destroying Descriptor Pool
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:23] Kompute Tensor destructor started. Type: 0
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:436] Kompute Tensor started destroy()
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:449] Kompose Tensor destroying primary buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:463] Kompose Tensor destroying staging buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:477] Kompose Tensor freeing primary memory
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:491] Kompose Tensor freeing staging memory
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:504] Kompute Tensor successful destroy()
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:28] Kompute Tensor destructor success
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:23] Kompute Tensor destructor started. Type: 0
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:436] Kompute Tensor started destroy()
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:449] Kompose Tensor destroying primary buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:463] Kompose Tensor destroying staging buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:477] Kompose Tensor freeing primary memory
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:491] Kompose Tensor freeing staging memory
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:504] Kompute Tensor successful destroy()
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:28] Kompute Tensor destructor success
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:23] Kompute Tensor destructor started. Type: 0
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:436] Kompute Tensor started destroy()
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:449] Kompose Tensor destroying primary buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:463] Kompose Tensor destroying staging buffer
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:477] Kompose Tensor freeing primary memory
[2021-03-05 13:06:47.072] [debug] [Tensor.cpp:491] Kompose Tensor freeing staging memory
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:504] Kompute Tensor successful destroy()
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:28] Kompute Tensor destructor success
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:23] Kompute Tensor destructor started. Type: 0
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:436] Kompute Tensor started destroy()
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:449] Kompose Tensor destroying primary buffer
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:463] Kompose Tensor destroying staging buffer
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:477] Kompose Tensor freeing primary memory
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:491] Kompose Tensor freeing staging memory
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:504] Kompute Tensor successful destroy()
[2021-03-05 13:06:47.073] [debug] [Tensor.cpp:28] Kompute Tensor destructor success
[2021-03-05 13:06:47.073] [debug] [Manager.cpp:58] Kompute Manager Destructor started
[2021-03-05 13:06:47.073] [debug] [Manager.cpp:66] Kompute Manager destroy() started
[2021-03-05 13:06:47.073] [debug] [Manager.cpp:75] Kompute Manager explicitly running destructor for managed sequences
[2021-03-05 13:06:47.073] [debug] [Manager.cpp:86] Kompute Manager explicitly freeing algorithms
[2021-03-05 13:06:47.073] [debug] [Manager.cpp:97] Kompute Manager explicitly freeing tensors
[2021-03-05 13:06:47.073] [info] [Manager.cpp:107] Destroying device
[2021-03-05 13:06:47.073] [debug] [Manager.cpp:111] Kompute Manager Destroyed Device
[2021-03-05 13:06:47.075] [debug] [Manager.cpp:134] Kompute Manager Destroyed Instance

It appears it's not being provided: [2021-03-05 13:06:46.962] [debug] [Manager.cpp:350] Kompute Manager desired extension layers {}

@axsaucedo
Copy link
Member Author

axsaucedo commented Mar 5, 2021

Interesting... ok yeah I can see it there under VK_EXT_shader_atomic_float but in order for it to be added you need to specify it in the manager initialisation, so you would have to initialise your manager as kp::Manager mgr(0, {}, { "VK_EXT_shader_atomic_float" }); - this should then add it to the enabled extensions.

@unexploredtest
Copy link
Contributor

Oh I wasn't trying to add any extensions, I was trying to add a debug layer, the one you mentioned:
export KOMPUTE_ENV_DEBUG_LAYERS="VK_LAYER_LUNARG_api_dump"

@unexploredtest
Copy link
Contributor

unexploredtest commented Mar 5, 2021

Sorry never mind I just forgot to set DEBUG. It works fine now thanks:
debug.txt

@axsaucedo
Copy link
Member Author

Great - ok I'll just add documentation and will merge

@unexploredtest
Copy link
Contributor

Sounds good! Let me know if you needed a hand

@axsaucedo axsaucedo merged commit 515c3b6 into master Mar 6, 2021
0.7.0 automation moved this from In progress to Done Mar 6, 2021
0.6.1 automation moved this from In progress to Done Mar 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working c++
Projects
0.6.1
Done
0.7.0
Done
Development

Successfully merging this pull request may close these issues.

Move shader defaultResource inside kp::Shader class push_constant not working in my case?
2 participants