support for gemma-4-unified needed

[ratio91]

Currently (using 0.4.1) I get:

omlx.engine_pool - WARNING - VLM loading failed for gemma-4-12B-mxfp8, falling back to LLM: Model type gemma4_unified not supported. Error: No module named 'mlx_vlm.speculative.drafters.gemma4_unified'

omlx.server - ERROR - POST /v1/chat/completions → 500 (unhandled): VLM load failed: Model type gemma4_unified not supported. Error: No module named 'mlx_vlm.speculative.drafters.gemma4_unified'; LLM fallback also failed: Model type gemma4_unified not supported.

[iGeeky]

Hi,

I encountered the exact same error while using the model. I wanted to confirm that this is not an isolated case.

My Environment:

• omlx version: 0.4.1 (1426)
• Model: mlx-community/gemma-4-12B-it-4bit

Error Log:

VLM load failed: Model type gemma4_unified not supported. Error: No module named 'mlx_vlm.speculative.drafters.gemma4_unified'; LLM fallback also failed: Model type gemma4_unified not supported.

It seems like the gemma4_unified model type is not currently supported in this version of mlx_vlm. Could you please clarify if there is a known workaround or if this is a pending fix?

[dapx]

It's fixed in the last commit, we just need a new release to download.

[RonaldAtZalion]

Looking forward to this - excited to try 12B.

[lekhnath]

waiting...

[jundot]

@ratio91 @iGeeky @dapx @rbe4zalion @lekhnath I published 0.4.2.dev1 as a prerelease with the updated mlx-vlm pin for Gemma4 Unified (12B) support.

Release: https://github.com/jundot/omlx/releases/tag/v0.4.2.dev1

Please try one of the new DMGs and let me know if gemma4_unified still fails to load.

[rgroli]

I loads the model now, but when trying to use it in a basic chat, it outputs:

Error: {"error":{"message":"Chat template error: Cannot use chat template functions because tokenizer.chat_template is not set and no template argument was passed! For information about writing templates and setting the tokenizer.chat_template attribute, please see the documentation at https://huggingface.co/docs/transformers/main/en/chat_templating","type":"invalid_request_error","param":null,"code":null}}

Not sure if this is related

[jundot]

@rgroli That error usually means the loaded checkpoint does not provide a chat template. I would use an instruction-tuned Gemma 4 model with it in the model name, for example mlx-community/gemma-4-12B-it-4bit or a local model named like gemma-4-12B-it-4bit.

The non-it base checkpoints are not chat-ready unless you provide a chat template manually.

[derpwiz]

Any thoughts on why the gemma-4-12B-it-assistant-* refuses to load and says gemma4_unified not supported?

[jundot]

Any thoughts on why the gemma-4-12B-it-assistant-* refuses to load and says gemma4_unified not supported?

Will update soon, sorry about that.

[jundot]

Just released v0.4.2.dev2 with Gemma4 unified MTP and VLM image recognition support. @derpwiz
Release notes: https://github.com/jundot/omlx/releases/tag/v0.4.2.dev2

[bodonisystems]

I'm getting This error trying to load gemma-4-12B-it-mlx-4bit 0.4.2.dev2

omlx.server - ERROR - [-] - POST /v1/chat/completions → 500 (unhandled): VLM load failed: Could not import module 'Gemma4UnifiedProcessor'. Are this object's requirements defined correctly?; LLM fallback also failed: Model type gemma4_unified not supported.

[samuelperezh]

@jundot any plan to add the oQ4 in your huggingface? I've been liking your quantizations

[geovansb]

It's working with v0.4.2.dev2
MacOS 26.5 - Apple M4 Pro

2026-06-04 18:53:24,242 - omlx.engine.vlm - INFO - [-] - VLM tool calling enabled: parser=gemma4
2026-06-04 18:53:24,245 - omlx.engine.vlm - INFO - [-] - VLMBatchedEngine loaded: ~/.omlx/models/mlx-community/gemma-4-12B-it-4bit
2026-06-04 18:53:24,281 - omlx.engine_pool - INFO - [-] - Loaded model: gemma-4-12B-it-4bit (actual: 10.88GB, estimated: 10.74GB, total: 10.74GB)
2026-06-04 18:53:24,282 - omlx.admin.routes - INFO - [-] - Manually loaded model: gemma-4-12B-it-4bit

Thanks

[xeoncross]

I updated to v0.4.1 (1426) and I'm still getting the following error on M1 Max:

VLM load failed: Model type gemma4_unified not supported. Error: No module named 'mlx_vlm.speculative.drafters.gemma4_unified'; LLM fallback also failed: Model type gemma4_unified not supported.

When I try to load either gemma-4-12B-it-4bit or gemma-4-12B-it-8bit. Qwen 3.6 models work.

edit

Sorry, I didn't read the comments above close enough. I swapped to the dev branch and v0.4.2.dev2 works. However, gemma-4-12B-it-8bit (12GB) is quite a bit slower than larger models like Qwen3.6-35B-A3B-4bit (19GB) and seems even slower than gemma-4-31b-8bit (32GB).

[RonaldAtZalion]

@xeoncross gemma 4 12B is dense - it will be slower than MoE models with <12B active params.