[Bug] Gemma 4 GGUF models (31B & 26B) fail to load on macOS with "Failed to load model" error

[ranajahanzaib]

Which version of LM Studio?
LM Studio 0.4.8+1

Which operating system?
macOS (26.4) - M1 Max (64GB RAM)

What is the bug?

Models were downloaded in-app via: https://lmstudio.ai/models/google/gemma-4-31b

When attempting to load the newly released Google Gemma 4 GGUF models (gemma-4-31B and gemma-4-26B-A4B) using the default settings, LM Studio instantly aborts the process and throws a generic "Failed to load model" UI error.

Given that Gemma 4 just dropped, it is highly likely that the current llama.cpp backend in 0.4.8+1 does not yet recognize the new Gemma 4 architecture or its specific MoE routing/tensor layers. The models easily fit within my 64GB of Unified Memory, so it is not an Out-Of-Memory (OOM) error.

Screenshots

• 
• 
• 
• 

Logs

2026-04-02 23:38:35 [DEBUG]
 LlamaV4::load called with model path: /Users/rj/.lmstudio/models/lmstudio-community/gemma-4-31B-it-GGUF/gemma-4-31B-it-Q4_K_M.gguf
LlamaV4::load config: n_parallel=4 n_ctx=32768 kv_unified=true
2026-04-02 23:38:35 [DEBUG]
 ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: using embedded metal library
2026-04-02 23:38:35 [DEBUG]
 ggml_metal_library_init: loaded in 0.007 sec
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name:   MTL0
ggml_metal_device_init: GPU family: MTLGPUFamilyApple7  (1007)
ggml_metal_device_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_device_init: GPU family: MTLGPUFamilyMetal4  (5002)
ggml_metal_device_init: simdgroup reduction   = true
ggml_metal_device_init: simdgroup matrix mul. = true
ggml_metal_device_init: has unified memory    = true
ggml_metal_device_init: has bfloat            = true
ggml_metal_device_init: has tensor            = false
ggml_metal_device_init: use residency sets    = true
ggml_metal_device_init: use shared buffers    = true
ggml_metal_device_init: recommendedMaxWorkingSetSize  = 55662.79 MB
2026-04-02 23:38:35 [DEBUG]
 srv    load_model: loading model '/Users/rj/.lmstudio/models/lmstudio-community/gemma-4-31B-it-GGUF/gemma-4-31B-it-Q4_K_M.gguf'
2026-04-02 23:38:35 [DEBUG]
 llama_model_load_from_file_impl: using device MTL0 (Apple M1 Max) (unknown id) - 53083 MiB free
2026-04-02 23:38:35 [DEBUG]
 llama_model_loader: loaded meta data with 43 key-value pairs and 833 tensors from /Users/rj/.lmstudio/models/lmstudio-community/gemma-4-31B-it-GGUF/gemma-4-31B-it-Q4_K_M.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = gemma4
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                     general.sampling.top_k i32              = 64
llama_model_loader: - kv   3:                     general.sampling.top_p f32              = 0.950000
llama_model_loader: - kv   4:                      general.sampling.temp f32              = 1.000000
llama_model_loader: - kv   5:                               general.name str              = Gemma 4 31B
llama_model_loader: - kv   6:                           general.finetune str              = it
llama_model_loader: - kv   7:                         general.size_label str              = 31B
llama_model_loader: - kv   8:                         gemma4.block_count u32              = 60
llama_model_loader: - kv   9:                      gemma4.context_length u32              = 262144
llama_model_loader: - kv  10:                    gemma4.embedding_length u32              = 5376
llama_model_loader: - kv  11:                 gemma4.feed_forward_length u32              = 21504
llama_model_loader: - kv  12:                gemma4.attention.head_count u32              = 32
llama_model_loader: - kv  13:             gemma4.attention.head_count_kv arr[i32,60]      = [16, 16, 16, 16, 16, 4, 16, 16, 16, 1...
llama_model_loader: - kv  14:                      gemma4.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  15:                  gemma4.rope.freq_base_swa f32              = 10000.000000
llama_model_loader: - kv  16:    gemma4.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  17:                gemma4.attention.key_length u32              = 512
llama_model_loader: - kv  18:              gemma4.attention.value_length u32              = 512
llama_model_loader: - kv  19:             gemma4.final_logit_softcapping f32              = 30.000000
llama_model_loader: - kv  20:            gemma4.attention.sliding_window u32              = 1024
llama_model_loader: - kv  21:          gemma4.attention.shared_kv_layers u32              = 0
llama_model_loader: - kv  22:    gemma4.embedding_length_per_layer_input u32              = 0
llama_model_loader: - kv  23:    gemma4.attention.sliding_window_pattern arr[bool,60]     = [true, true, true, true, true, false,...
llama_model_loader: - kv  24:            gemma4.attention.key_length_swa u32              = 256
llama_model_loader: - kv  25:          gemma4.attention.value_length_swa u32              = 256
llama_model_loader: - kv  26:                gemma4.rope.dimension_count u32              = 512
llama_model_loader: - kv  27:            gemma4.rope.dimension_count_swa u32              = 256
llama_model_loader: - kv  28:                       tokenizer.ggml.model str              = gemma4
2026-04-02 23:38:35 [DEBUG]
 llama_model_loader: - kv  29:                      tokenizer.ggml.tokens arr[str,262144]  = ["<pad>", "<eos>", "<bos>", "<unk>", ...
2026-04-02 23:38:35 [DEBUG]
 llama_model_loader: - kv  30:                      tokenizer.ggml.scores arr[f32,262144]  = [-1000.000000, -1000.000000, -1000.00...
2026-04-02 23:38:35 [DEBUG]
 llama_model_loader: - kv  31:                  tokenizer.ggml.token_type arr[i32,262144]  = [3, 3, 3, 3, 3, 1, 1, 1, 1, 1, 1, 1, ...
2026-04-02 23:38:35 [DEBUG]
 llama_model_loader: - kv  32:                      tokenizer.ggml.merges arr[str,514906]  = ["\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n \n", ...
llama_model_loader: - kv  33:                tokenizer.ggml.bos_token_id u32              = 2
llama_model_loader: - kv  34:                tokenizer.ggml.eos_token_id u32              = 1
llama_model_loader: - kv  35:            tokenizer.ggml.unknown_token_id u32              = 3
llama_model_loader: - kv  36:            tokenizer.ggml.padding_token_id u32              = 0
llama_model_loader: - kv  37:               tokenizer.ggml.mask_token_id u32              = 4
llama_model_loader: - kv  38:                    tokenizer.chat_template str              = {%- macro format_parameters(propertie...
llama_model_loader: - kv  39:            tokenizer.ggml.add_space_prefix bool             = false
llama_model_loader: - kv  40:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  41:               general.quantization_version u32              = 2
llama_model_loader: - kv  42:                          general.file_type u32              = 15
llama_model_loader: - type  f32:  422 tensors
llama_model_loader: - type q4_K:  355 tensors
llama_model_loader: - type q6_K:   56 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_K - Medium
print_info: file size   = 17.39 GiB (4.87 BPW)
2026-04-02 23:38:35 [DEBUG]
 llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4'
llama_model_load_from_file_impl: failed to load model
common_init_from_params: failed to load model '/Users/rj/.lmstudio/models/lmstudio-community/gemma-4-31B-it-GGUF/gemma-4-31B-it-Q4_K_M.gguf'
srv    load_model: failed to load model, '/Users/rj/.lmstudio/models/lmstudio-community/gemma-4-31B-it-GGUF/gemma-4-31B-it-Q4_K_M.gguf': error loading model: error loading model architecture: unknown model architecture: 'gemma4'
2026-04-02 23:38:35 [DEBUG]
 [LLMProcess] Failed to load model _0x12629c [Error]: Failed to load model.
    at _0x4935cb.loadModel (/Applications/LM Studio.app/Contents/Resources/app/.webpack/lib/llmworker.js:1:609772)
    at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
    at async _0x4935cb.handleMessage (/Applications/LM Studio.app/Contents/Resources/app/.webpack/lib/llmworker.js:1:601972) {
  cause: 'Failed to load model',
  suggestion: undefined,
  errorData: undefined,
  data: undefined,
  displayData: undefined,
  title: 'Failed to load model.'
}

To Reproduce
Steps to reproduce the behavior:

1. Download either the gemma-4-31B or gemma-4-26B-A4B GGUF via the in-app search tab.
2. Go to the 'Chat' interface (the speech bubble icon on the left).
3. Click on the top drop-down menu to load a model and select the newly downloaded Gemma 4 model.
4. Keep the preset hardware settings on default.
5. See error: Failed to load model.

[ranajahanzaib]

Following up with a quick log snippet:
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4'

It seems the current backend is just missing the gemma4 definitions. Would syncing with the latest upstream llama.cpp resolve this, or is there more involved with the new MoE routing? I'd love to learn more about how LM Studio handles these Day 0 architecture changes. Happy to help test a beta build if needed.

[JacobMillner]

Yea updating llama.cpp should do the trick.

[cryptobench]

Had same problem, noticed that inside the runtime settings it mentioned I had the latest version of llama.cpp, but yet it was using version 2.8.0 as the default for GGUF.

Select 2.10.0 as default and it works.

[eliyastein]

Had same problem, noticed that inside the runtime settings it mentioned I had the latest version of llama.cpp, but yet it was using version 2.8.0 as the default for GGUF.

Select 2.10.0 as default and it works.

I was having the same issue, but after doing the runtime update gemma-4-26b-a4b loads perfectly.

[will-lms]

Thank you for sharing. As @cryptobench and @eliyastein shared, Gemma 4 support was first included in the 2.10.0 version of the llama.cpp runtime. A further improvement to the 31B model was just released in the 2.10.1 version. Please make sure you are using that version or higher.

[argentini]

What about Gemma 4 MLX models?

[kanishk619]

Wheneve the entire context length is selected, entire OS freezes and have to do a force reboot. Works fine approx till 100000+ context length. This is with lmstudio-community/gemma-4-26b-a4b-it and I have already updated runtime and lmstudio.

[ranajahanzaib]

Fixed

✅ Updating the llama.cpp runtime completely fixed the loading error for me. Huge thanks to @JacobMillner, @cryptobench, and @eliyastein.

✅ Bumping llama.cpp to version 2.10.1 also resolved a secondary issue I had where the Gemma 4 31B model was returning empty or whitespace responses. Thanks @will-lms.

@argentini, I'm holding out for the MLX versions as well. Given that MLX avoids memory copying by taking full advantage of the unified architecture, the optimizations should be significant. I'm curious how llama.cpp compares in its memory handling, but from what I've seen regarding specifically M1 benchmarks, MLX has a slower TTFT but pulls ahead drastically in tokens-per-second on longer generations.

@kanishk619, you are spot on. I noticed that pushing the context window higher with these specific models eats up a massive amount of RAM. Depending on your workflow, you might want to scale it back... I've dropped mine down to 8k for now, even though I comfortably run 32k on other sub-35B models.

[zyc530]

lm studio 核心 2.10.1 加载gemma-4-26B-A4B-it-Q4_K_M.gguf依然死机

[nicolaric]

same issue as @argentini

[CHesketh76]

Same issue, I have the latest llamacpp v2.10.1 but any use of the GPU causes the model to fail loading and returns a code that it failed to write context.

[ranajahanzaib]

Same issue, I have the latest llamacpp v2.10.1 but any use of the GPU causes the model to fail loading and returns a code that it failed to write context.

Just to double-check, are you using the GGUF model or the MLX one?

For me, the GGUF versions are working fine after bumping llama.cpp to v2.10.1. Btw, llama.cpp v2.11.0 literally just came out, so it might be worth trying that update to see if it fixes your GPU/context writing error.

If you're trying to use the MLX version, it's definitely still broken. LM Studio updated the MLX runtime to v1.5.0, but it now explicitly says: ValueError: Gemma 4 support is not ready yet, stay tuned!.

I opened a new issue (#1741) to track the MLX side of things.