Are there bugs with E4B/E2B or with Gemma 4 at the moment?

[TomLucidor]

Tool calling seemed to be an issue with Gemma 4 models (including the smaller ones) for multiple agents and maybe they come from parsers/templates NousResearch/hermes-agent#6626 anomalyco/opencode#20995

Update: Hermes is partially busted and need some patches NousResearch/hermes-agent#6626 (comment)

[TomLucidor]

Self-research: There seems to be fixes like:

• @lucianommartins from google DeepMind https://huggingface.co/google/gemma-4-26B-A4B-it/discussions/47 + https://huggingface.co/google/gemma-4-E4B-it/discussions/36
• @mann1x or ManniX-ITA https://huggingface.co/google/gemma-4-31B-it/discussions/119 + https://huggingface.co/google/gemma-4-E4B-it/discussions/37
• @RyanMullins https://huggingface.co/google/gemma-4-E4B-it/discussions/13 + https://huggingface.co/google/gemma-4-31B-it/discussions/55 + https://huggingface.co/google/gemma-4-26B-A4B-it/discussions/20

Side discovery: even dLLMs have similar quality issues needing to be patched https://huggingface.co/google/diffusiongemma-26B-A4B-it/discussions/23

[TomLucidor]

Self-sidenote: PLE quants are okay https://github.com/FakeRocket543/mlx-gemma4

[lucianommartins]

Hi @TomLucidor - yes we identified some improvements to be done on the chat_templates. I'm proposing few PRs to the official ones... could you please test with them?

https://huggingface.co/google/gemma-4-E2B-it/discussions/35
https://huggingface.co/google/gemma-4-E4B-it/discussions/36
https://huggingface.co/google/gemma-4-12B-it/discussions/35
https://huggingface.co/google/gemma-4-26B-A4B-it/discussions/47
https://huggingface.co/google/gemma-4-31B-it/discussions/118

[TomLucidor]

Will do next week, and also would like @raullenchai to also vouch for this initiative to get Gemma 4 working at all costs. Are there ideal testing protocols that we can try with MLX?