segfault when using glm-4-9b-chat ( fix in description )

[tv1wnd]

Name and Version

version: 4623 (21c84b5)
built with MSVC 19.29.30153.0 for x64

Operating systems

Windows, Linux

GGML backends

CUDA, CPU

Hardware

any

Models

https://huggingface.co/THUDM/glm-4-9b-chat

Problem description & steps to reproduce

segfault due to model.layers[il].wq being a nullptr in build_chatglm()

quick fix: ( behaviour consistent with llama_model.cpp line 3086 )

diff --git a/src/llama.cpp b/src/llama.cpp
index 5760017e..b91f4873 100644
--- a/src/llama.cpp
+++ b/src/llama.cpp
@@ -7215,7 +7215,7 @@ struct llm_build_context {
                 struct ggml_tensor * Qcur = nullptr;
                 struct ggml_tensor * Kcur = nullptr;
                 struct ggml_tensor * Vcur = nullptr;
-                if (model.type == LLM_TYPE_1_5B || model.type == LLM_TYPE_4B || model.type == LLM_TYPE_9B) {
+                if (model.layers[il].wqkv == nullptr) {
                     Qcur = llm_build_lora_mm(lctx, ctx0, model.layers[il].wq, cur);
                     if (model.layers[il].bq) {
                         Qcur = ggml_add(ctx0, Qcur, model.layers[il].bq);

First Bad Commit

0cec062

Relevant log output

see description

[MoonRide303]

Same for me - I cannot launch glm-4-9b-chat-GGUF and glm-4-9b-chat-1m-GGUF. Tested on Q6_K quants, using llama.cpp b4641.

Both models worked fine with b4608.