vulkan: fix clang-cl debug build

[Adriankhl]

As discussed here #7130

With clang-cl 18.1.5, MSVC 19.40.33808, vulkan SDK 1.3.283.0

An MSVC runtime error shows up: "Expression: can't dereference invalidated vector iterator" when I run the debug build of embedding with vulkan backend. I think I have also seen it somewhere when I run main/llava but I didn't manage to reproduce it.

The problem happens here

llama.cpp/ggml-vulkan.cpp

Lines 625 to 646 in e23b974

	static void ggml_vk_submit(vk_context * ctx, vk::Fence fence) {
	#ifdef GGML_VULKAN_DEBUG
	std::cerr << "ggml_vk_submit(" << ctx->seqs.size() << ", " << fence << ")" << std::endl;
	#endif
	if (ctx->seqs.empty()) {
	return;
	}
	std::vector<std::vector<uint64_t>> tl_wait_vals;
	std::vector<std::vector<uint64_t>> tl_signal_vals;
	std::vector<std::vector<vk::Semaphore>> tl_wait_semaphores;
	std::vector<std::vector<vk::Semaphore>> tl_signal_semaphores;
	std::vector<vk::TimelineSemaphoreSubmitInfo> tl_submit_infos;
	std::vector<vk::SubmitInfo> submit_infos;
	int idx = -1;
	std::vector<std::vector<vk::PipelineStageFlags>> stage_flags;
	size_t reserve = 0;
	for (const auto& sequence : ctx->seqs) {
	reserve += sequence.size();
	}

ctx->seqs.size() shows 1 but MSVC thinks it is of size 0 when I step through the code using lldb. I believe it comes from this issue. The stackoverflow answer suggested adding a /Zc:nrvo- flag, and it doesn't work for clang-cl so I added another _ITERATOR_DEBUG_LEVEL=0 definition for clang-cl.

Seems like the problem in MSVC is gone , clang-cl still needs _ITERATOR_DEBUG_LEVEL=0 to work

[cebtenzzre]

Why only do this when Vulkan is enabled? This seems like it should apply to all C++ code compiled by MSVC or clang-cl.

[Adriankhl]

I don't see this problem in other backends (i.e. cpu), and this is a workaround that causes a lost of debug information, so I don't want to fix something thats not break

[cebtenzzre]

I am surprised that /Zc:nrvo- even has the intended effect because in all of the linked StackOverflow examples, the problematic vector (or object) is being returned (hence, Named Return Value Optimization applies) and the debugging is incorrect between the declaration and the time it is returned (because the storage is actually elsewhere).

The Kompute backend definitely returns vectors and is affected by this, but I don't see where the Vulkan backend does.

[slaren]

I agree that disabling NRVO doesn't make much sense here. @Adriankhl how can I reproduce this issue? I ran a test with a debug build with cl 19.29.30154 and I didn't get any errors.

[Adriankhl]

@slaren I did some MSVC updates in the past few days and the error is now gone, and I failed to pinpoint the problematic version. Now I can only reproduce the issue by clang-cl:

cmake .. -GNinja -DCMAKE_C_COMPILER=clang-cl -DCMAKE_CXX_COMPILER=clang-cl -DCMAKE_EXPORT_COMPILE_COMMANDS=1 -DLLAMA_NATIVE=OFF -DLLAMA_VULKAN=ON -DCMAKE_BUILD_TYPE=Debug

ninja -j6

.\bin\embedding.exe -m ..\..\..\models\mxbai-embed-large-v1-f16.gguf -p "Antibiotics are a type of medication used to treat bacterial infections. They work by either killing the bacteria or preventing them from reproducing, allowing the body's immune system to fight off the infection. Antibiotics are usually taken orally in the form of pills, capsules, or liquid solutions, or sometimes administered intravenously. They are not effective against viral infections, and using them inappropriately can lead to antibiotic resistance.`nI love cat"

So I have updated my PR to only deal with clang-cl

[slaren]

Thanks, I was able to reproduce this with clang-cl.