Update compileSource function in examples/docs to correct one

[kommander]

I am not a windows guy... but what the hell?
Followed the example here: https://github.com/KomputeProject/kompute/tree/master/examples/array_multiplication

Frustrating... The example does not mention vulkan headers and the other deps should be optional but are not. wth?

cmake -Bbuild/ -DCMAKE_BUILD_TYPE=Debug -DKOMPUTE_OPT_INSTALL=0 -DKOMPUTE_OPT_REPO_SUBMODULE_BUILD=1 -DKOMPUTE_OPT_ENABLE_SPDLOG=1
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.17763.0 to target Windows 10.0.19043.
-- The C compiler identification is MSVC 19.29.30138.0
-- The CXX compiler identification is MSVC 19.29.30138.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Vulkan: C:/VulkanSDK/1.2.198.1/Lib/vulkan-1.lib
CMake Error at C:/Users/rootkid/workspace/kompute/src/CMakeLists.txt:73 (add_subdirectory):
  The source directory

    C:/Users/rootkid/workspace/kompute/external/Vulkan-Headers

  does not contain a CMakeLists.txt file.


CMake Error at C:/Users/rootkid/workspace/kompute/src/CMakeLists.txt:74 (get_target_property):
  get_target_property() called with non-existent target "Vulkan-Headers".


CMake Error at C:/Users/rootkid/workspace/kompute/src/CMakeLists.txt:85 (add_subdirectory):
  The source directory

    C:/Users/rootkid/workspace/kompute/external/fmt

  does not contain a CMakeLists.txt file.


CMake Error at C:/Users/rootkid/workspace/kompute/src/CMakeLists.txt:101 (add_subdirectory):
  The source directory

    C:/Users/rootkid/workspace/kompute/external/spdlog

  does not contain a CMakeLists.txt file.


-- Configuring incomplete, errors occurred!
See also "C:/Users/rootkid/workspace/kompute/examples/array_multiplication/build/CMakeFiles/CMakeOutput.log".

[axsaucedo]

Did you pull the submodule repos?

[kommander]

FML... nope. Checking....

[kommander]

Ok, running BUILD_ALL and says

Severity	Code	Description	Project	File	Line	Suppression State
Error	C2653	'kp_test_utils': is not a class or namespace name	kompute_array_mult	C:\Users\rootkid\workspace\kompute\examples\array_multiplication\src\Main.cpp	54	

[axsaucedo]

Hmm ok you just found out an actual bug on the code. You have to change:

kp_test_utils::compileSource(shader)

to

compileSource(shader)

On line 54 of that example

https://github.com/KomputeProject/kompute/blob/master/examples/array_multiplication/src/Main.cpp#L52-L55

[axsaucedo]

If you confirm that works for you I can update with the fix

[kommander]

Well, it builds. Still getting `Unable to start program (...) Access is denied." Don't know why though.

[axsaucedo]

That seems a bit strange, never seen that before - are you running it different to how you're buidling? Sounds like permissions issue, or perhaps running the different binary? Are you using visual studio or running via command line?

[axsaucedo]

Fixed via #262

[kommander]

Using Visual Studio 19, other projects worked fine. No matter which vcxproj file I open, it always opens a set of all projects. building fine, but then the permission. Let me see if I can fix this.

[kommander]

I can run the debug build manually... not from VS though and it aborts with an error dialog, CLI output is this:

[2022-01-18 19:37:38.553] [debug] [Manager.cpp:144] Kompute Manager creating instance
[2022-01-18 19:37:38.554] [debug] [Manager.cpp:172] Kompute Manager adding debug validation layers
[2022-01-18 19:37:38.581] [debug] [Manager.cpp:203] Available layers: {"VK_LAYER_EOS_Overlay", "VK_LAYER_KHRONOS_synchronization2", "VK_LAYER_KHRONOS_validation", "VK_LAYER_LUNARG_api_dump", "VK_LAYER_LUNARG_device_simulation", "VK_LAYER_LUNARG_gfxreconstruct", "VK_LAYER_LUNARG_monitor", "VK_LAYER_LUNARG_screenshot", "VK_LAYER_NV_optimus"}
[2022-01-18 19:37:38.581] [debug] [Manager.cpp:214] Kompute Manager Initializing instance with valid layers: {"VK_LAYER_KHRONOS_validation"}
[2022-01-18 19:37:38.831] [debug] [Manager.cpp:228] Kompute Manager Instance Created
[2022-01-18 19:37:38.832] [debug] [Manager.cpp:232] Kompute Manager adding debug callbacks
[2022-01-18 19:37:38.833] [debug] [Manager.cpp:280] Kompute Manager creating Device
[2022-01-18 19:37:38.837] [info] [Manager.cpp:324] Using physical device index 0 found NVIDIA GeForce RTX 3080 Laptop GPU
[2022-01-18 19:37:38.837] [debug] [Manager.cpp:375] Kompute Manager desired extension layers {}
[2022-01-18 19:37:38.838] [debug] [Manager.cpp:385] Kompute Manager available extensions {"VK_EXT_4444_formats", "VK_EXT_blend_operation_advanced", "VK_EXT_buffer_device_address", "VK_EXT_calibrated_timestamps", "VK_EXT_conditional_rendering", "VK_EXT_conservative_rasterization", "VK_EXT_custom_border_color", "VK_EXT_depth_clip_enable", "VK_EXT_depth_range_unrestricted", "VK_EXT_descriptor_indexing", "VK_EXT_discard_rectangles", "VK_EXT_extended_dynamic_state", "VK_EXT_external_memory_host", "VK_EXT_fragment_shader_interlock", "VK_EXT_full_screen_exclusive", "VK_EXT_hdr_metadata", "VK_EXT_host_query_reset", "VK_EXT_image_robustness", "VK_EXT_index_type_uint8", "VK_EXT_inline_uniform_block", "VK_EXT_line_rasterization", "VK_EXT_memory_budget", "VK_EXT_memory_priority", "VK_EXT_pci_bus_info", "VK_EXT_pipeline_creation_cache_control", "VK_EXT_pipeline_creation_feedback", "VK_EXT_post_depth_coverage", "VK_EXT_private_data", "VK_EXT_robustness2", "VK_EXT_sample_locations", "VK_EXT_sampler_filter_minmax", "VK_EXT_scalar_block_layout", "VK_EXT_separate_stencil_usage", "VK_EXT_shader_atomic_float", "VK_EXT_shader_demote_to_helper_invocation", "VK_EXT_shader_image_atomic_int64", "VK_EXT_shader_subgroup_ballot", "VK_EXT_shader_subgroup_vote", "VK_EXT_shader_viewport_index_layer", "VK_EXT_subgroup_size_control", "VK_EXT_texel_buffer_alignment", "VK_EXT_tooling_info", "VK_EXT_transform_feedback", "VK_EXT_vertex_attribute_divisor", "VK_EXT_ycbcr_image_arrays", "VK_KHR_16bit_storage", "VK_KHR_8bit_storage", "VK_KHR_acceleration_structure", "VK_KHR_bind_memory2", "VK_KHR_buffer_device_address", "VK_KHR_copy_commands2", "VK_KHR_create_renderpass2", "VK_KHR_dedicated_allocation", "VK_KHR_deferred_host_operations", "VK_KHR_depth_stencil_resolve", "VK_KHR_descriptor_update_template", "VK_KHR_device_group", "VK_KHR_draw_indirect_count", "VK_KHR_driver_properties", "VK_KHR_external_fence", "VK_KHR_external_fence_win32", "VK_KHR_external_memory", "VK_KHR_external_memory_win32", "VK_KHR_external_semaphore", "VK_KHR_external_semaphore_win32", "VK_KHR_fragment_shading_rate", "VK_KHR_get_memory_requirements2", "VK_KHR_image_format_list", "VK_KHR_imageless_framebuffer", "VK_KHR_maintenance1", "VK_KHR_maintenance2", "VK_KHR_maintenance3", "VK_KHR_multiview", "VK_KHR_pipeline_executable_properties", "VK_KHR_pipeline_library", "VK_KHR_push_descriptor", "VK_KHR_ray_query", "VK_KHR_ray_tracing_pipeline", "VK_KHR_relaxed_block_layout", "VK_KHR_sampler_mirror_clamp_to_edge", "VK_KHR_sampler_ycbcr_conversion", "VK_KHR_separate_depth_stencil_layouts", "VK_KHR_shader_atomic_int64", "VK_KHR_shader_clock", "VK_KHR_shader_draw_parameters", "VK_KHR_shader_float16_int8", "VK_KHR_shader_float_controls", "VK_KHR_shader_non_semantic_info", "VK_KHR_shader_subgroup_extended_types", "VK_KHR_shader_terminate_invocation", "VK_KHR_spirv_1_4", "VK_KHR_storage_buffer_storage_class", "VK_KHR_swapchain", "VK_KHR_swapchain_mutable_format", "VK_KHR_synchronization2", "VK_KHR_timeline_semaphore", "VK_KHR_uniform_buffer_standard_layout", "VK_KHR_variable_pointers", "VK_KHR_vulkan_memory_model", "VK_KHR_win32_keyed_mutex", "VK_KHR_workgroup_memory_explicit_layout", "VK_KHR_zero_initialize_workgroup_memory", "VK_NVX_binary_import", "VK_NVX_image_view_handle", "VK_NVX_multiview_per_view_attributes", "VK_NV_acquire_winrt_display", "VK_NV_clip_space_w_scaling", "VK_NV_compute_shader_derivatives", "VK_NV_cooperative_matrix", "VK_NV_corner_sampled_image", "VK_NV_coverage_reduction_mode", "VK_NV_cuda_kernel_launch", "VK_NV_dedicated_allocation", "VK_NV_dedicated_allocation_image_aliasing", "VK_NV_device_diagnostic_checkpoints", "VK_NV_device_diagnostics_config", "VK_NV_device_generated_commands", "VK_NV_external_memory", "VK_NV_external_memory_win32", "VK_NV_fill_rectangle", "VK_NV_fragment_coverage_to_color", "VK_NV_fragment_shader_barycentric", "VK_NV_fragment_shading_rate_enums", "VK_NV_framebuffer_mixed_samples", "VK_NV_geometry_shader_passthrough", "VK_NV_mesh_shader", "VK_NV_ray_tracing", "VK_NV_representative_fragment_test", "VK_NV_sample_mask_override_coverage", "VK_NV_scissor_exclusive", "VK_NV_shader_image_footprint", "VK_NV_shader_sm_builtins", "VK_NV_shader_subgroup_partitioned", "VK_NV_shading_rate_image", "VK_NV_viewport_array2", "VK_NV_viewport_swizzle", "VK_NV_win32_keyed_mutex"}
[2022-01-18 19:37:38.928] [debug] [Manager.cpp:408] Kompute Manager device created
[2022-01-18 19:37:38.929] [debug] [Manager.cpp:422] Kompute Manager compute queue obtained
[2022-01-18 19:37:38.929] [debug] [kompute/Kompute.hpp:2191] Kompute Manager tensor creation triggered
[2022-01-18 19:37:38.929] [debug] [Tensor.cpp:17] Kompute Tensor constructor data length: 3, and type: 0
[2022-01-18 19:37:38.930] [debug] [Tensor.cpp:44] Kompute Tensor rebuilding with size 3
[2022-01-18 19:37:38.930] [debug] [Tensor.cpp:363] Kompute Tensor creating buffer
[2022-01-18 19:37:38.930] [debug] [Tensor.cpp:372] Kompute Tensor creating primary buffer and memory
[2022-01-18 19:37:38.930] [debug] [Tensor.cpp:416] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst | StorageBuffer }
[2022-01-18 19:37:38.931] [debug] [Tensor.cpp:433] Kompute Tensor allocating and binding memory
[2022-01-18 19:37:38.931] [debug] [Tensor.cpp:460] Kompute Tensor allocating memory index: 1, size 256, flags: { DeviceLocal }
[2022-01-18 19:37:38.932] [debug] [Tensor.cpp:385] Kompute Tensor creating staging buffer and memory
[2022-01-18 19:37:38.932] [debug] [Tensor.cpp:416] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst }
[2022-01-18 19:37:38.933] [debug] [Tensor.cpp:433] Kompute Tensor allocating and binding memory
[2022-01-18 19:37:38.933] [debug] [Tensor.cpp:460] Kompute Tensor allocating memory index: 2, size 256, flags: { HostVisible | HostCoherent }
[2022-01-18 19:37:38.933] [debug] [Tensor.cpp:398] Kompute Tensor buffer & memory creation successful
[2022-01-18 19:37:38.933] [debug] [Tensor.cpp:114] Kompute Tensor mapping data from host buffer
[2022-01-18 19:37:38.934] [debug] [kompute/Kompute.hpp:1036] Kompute TensorT constructor with data size 3
[2022-01-18 19:37:38.934] [debug] [kompute/Kompute.hpp:2191] Kompute Manager tensor creation triggered
[2022-01-18 19:37:38.935] [debug] [Tensor.cpp:17] Kompute Tensor constructor data length: 3, and type: 0
[2022-01-18 19:37:38.935] [debug] [Tensor.cpp:44] Kompute Tensor rebuilding with size 3
[2022-01-18 19:37:38.935] [debug] [Tensor.cpp:363] Kompute Tensor creating buffer
[2022-01-18 19:37:38.935] [debug] [Tensor.cpp:372] Kompute Tensor creating primary buffer and memory
[2022-01-18 19:37:38.936] [debug] [Tensor.cpp:416] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst | StorageBuffer }
[2022-01-18 19:37:38.936] [debug] [Tensor.cpp:433] Kompute Tensor allocating and binding memory
[2022-01-18 19:37:38.936] [debug] [Tensor.cpp:460] Kompute Tensor allocating memory index: 1, size 256, flags: { DeviceLocal }
[2022-01-18 19:37:38.936] [debug] [Tensor.cpp:385] Kompute Tensor creating staging buffer and memory
[2022-01-18 19:37:38.937] [debug] [Tensor.cpp:416] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst }
[2022-01-18 19:37:38.937] [debug] [Tensor.cpp:433] Kompute Tensor allocating and binding memory
[2022-01-18 19:37:38.937] [debug] [Tensor.cpp:460] Kompute Tensor allocating memory index: 2, size 256, flags: { HostVisible | HostCoherent }
[2022-01-18 19:37:38.937] [debug] [Tensor.cpp:398] Kompute Tensor buffer & memory creation successful
[2022-01-18 19:37:38.938] [debug] [Tensor.cpp:114] Kompute Tensor mapping data from host buffer
[2022-01-18 19:37:38.938] [debug] [kompute/Kompute.hpp:1036] Kompute TensorT constructor with data size 3
[2022-01-18 19:37:38.938] [debug] [kompute/Kompute.hpp:2191] Kompute Manager tensor creation triggered
[2022-01-18 19:37:38.938] [debug] [Tensor.cpp:17] Kompute Tensor constructor data length: 3, and type: 0
[2022-01-18 19:37:38.939] [debug] [Tensor.cpp:44] Kompute Tensor rebuilding with size 3
[2022-01-18 19:37:38.939] [debug] [Tensor.cpp:363] Kompute Tensor creating buffer
[2022-01-18 19:37:38.939] [debug] [Tensor.cpp:372] Kompute Tensor creating primary buffer and memory
[2022-01-18 19:37:38.939] [debug] [Tensor.cpp:416] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst | StorageBuffer }
[2022-01-18 19:37:38.939] [debug] [Tensor.cpp:433] Kompute Tensor allocating and binding memory
[2022-01-18 19:37:38.940] [debug] [Tensor.cpp:460] Kompute Tensor allocating memory index: 1, size 256, flags: { DeviceLocal }
[2022-01-18 19:37:38.940] [debug] [Tensor.cpp:385] Kompute Tensor creating staging buffer and memory
[2022-01-18 19:37:38.940] [debug] [Tensor.cpp:416] Kompute Tensor creating buffer with memory size: 12, and usage flags: { TransferSrc | TransferDst }
[2022-01-18 19:37:38.940] [debug] [Tensor.cpp:433] Kompute Tensor allocating and binding memory
[2022-01-18 19:37:38.943] [debug] [Tensor.cpp:460] Kompute Tensor allocating memory index: 2, size 256, flags: { HostVisible | HostCoherent }
[2022-01-18 19:37:38.943] [debug] [Tensor.cpp:398] Kompute Tensor buffer & memory creation successful
[2022-01-18 19:37:38.943] [debug] [Tensor.cpp:114] Kompute Tensor mapping data from host buffer
[2022-01-18 19:37:38.943] [debug] [kompute/Kompute.hpp:1036] Kompute TensorT constructor with data size 3
<< was unexpected at this time.

[axsaucedo]

What is the << was unexpected at this time.? Is that the error you are seeing? Everything looks good until then but seems to cut

[kommander]

Thats all I see... plus the error dialog:

[kommander]

Got it running from VS...

EDIT:

EDIT2:
Seems also like an access error... Let me checkout the repo fresh again.

[axsaucedo]

Hmm I wonder if this is failing due to support on windows or due to permissions, let me set up the windows machine to try it as I normally run it on linux or on wsl2

[axsaucedo]

Ok I can reproduce the permission error from the visual studio editor

As well as the error when running it from the console

I will have a deeper look at why this is happening

[kommander]

Cloned fresh, submodules init/update, cmake, VS build... all fine. Running it - same issue. I cannot find the file tmp_kp_shader.comp.spv anywhere in the repo, which it seems to be trying to load

read access violation aka file not found?

[kommander]

I am at easy that it is not only my stupidity :) Thanks for looking into this!

[axsaucedo]

Hmm quite strange as that is a tmp file that gets created to put the SPIR-V compiled contents of the string so that it can bse used by the algorithm - if it indeed is that I will try to see if updating permissions is the only thing required to fix or whether there's actually something deeper

[axsaucedo]

Ok @kommander it seems the issue was just lack from my side on updating the correct one which is this one:

kompute/test/utils/kompute_test/Shader.hpp

Lines 19 to 32 in ec27834

	std::vector<uint32_t>
	compileSource(
	const std::string& source)
	{
	std::ofstream fileOut("tmp_kp_shader.comp");
	fileOut << source;
	fileOut.close();
	if (system(std::string("glslangValidator -V tmp_kp_shader.comp -o tmp_kp_shader.comp.spv").c_str()))
	throw std::runtime_error("Error running glslangValidator command");
	std::ifstream fileStream("tmp_kp_shader.comp.spv", std::ios::binary);
	std::vector<char> buffer;
	buffer.insert(buffer.begin(), std::istreambuf_iterator<char>(fileStream), {});
	return {(uint32_t*)buffer.data(), (uint32_t*)(buffer.data() + buffer.size())};
	}

I have now updated in the PR https://github.com/KomputeProject/kompute/pull/262/files

Can you try adding the same change and confirming it works for you? I can confirm that it works from my side:

I have to look at adding these examples to the CI as well

[kommander]

Works 🏆

[axsaucedo]

I've had a deeper scan in the code and found a couple of other parts in the docs and godot example where this needed updating as well - it shoudl all be good, thanks for the heads up, let me know once you can confirm.

[axsaucedo]

Ah great - nice one, thanks 👍

[kommander]

Thank you! Closing this then...

EDIT: Should it stay open?

[fzStella]

Hello,
Could you tell me how you solved this problem finally? I hope you can check my issue and tell me which step I did wrong.