gemma4 GGUF: dense Gemma-4 checkpoints cannot load — parse_gguf_config requires gemma4.expert_count and hardcodes moe_enabled=True

[gerbear1990]

Summary

parse_gguf_config (python/freetoken/models/gemma4/gguf.py) unconditionally requires
gemma4.expert_count and hardcodes moe_enabled=True, so a dense Gemma-4 GGUF cannot
load at all — the gemma4 GGUF path structurally cannot represent a non-MoE checkpoint.

docs/models.md lists dense members of the family (google/gemma-4-12B-it,
nvidia/Gemma-4-31B-IT-NVFP4) under Gemma-4, and GGUF is documented as the one native
non-safetensors path, so dense Gemma-4 GGUFs read as in scope.

Reproduction

ft serve --model gemma-4-31B-it-qat-UD-Q4_K_XL.gguf

(unsloth/gemma-4-31B-it-qat-GGUF, a dense QAT checkpoint.)

Traceback

File "python/freetoken/engine/engine.py", line 1307, in freetoken.engine.engine._adjust_config
File "freetoken/engine/config.py", line 94, in model_config
    return parse_config(self.hf_config)
File "python/freetoken/models/gemma4/gguf.py", line 108, in freetoken.models.gemma4.gguf.parse_gguf_config
File "python/freetoken/models/gemma4/gguf.py", line 57, in freetoken.models.gemma4.gguf.parse_gguf_config.g
KeyError: 'missing GGUF metadata key gemma4.expert_count'

The frontend then reports Backend supervisor: KeyError and shuts the API server down
(Backend worker is gone and cannot be restarted), exit code 15.

Metadata

Reading the GGUF KV table directly, the dense checkpoint simply has no expert keys, while
the MoE sibling does:

key	gemma-4-31B-it-qat (dense)	gemma-4-26B-A4B-it (MoE)
gemma4.block_count	60	30
gemma4.context_length	262144	262144
gemma4.expert_count	absent	128
gemma4.expert_used_count	absent	8
gemma4.expert_feed_forward_length	absent	704

Cause

def g(key: str):
    val = m.get(f"gemma4.{key}")
    if val is None:
        raise KeyError(f"missing GGUF metadata key gemma4.{key}")
    return val
...
    num_experts=int(g("expert_count")),
    num_experts_per_tok=int(g("expert_used_count")),
    moe_intermediate_size=int(g("expert_feed_forward_length")),
    ...
    moe_enabled=True,        # <-- unconditional
    expert_quant="q4_0",     # <-- unconditional

g() is fatal-on-missing by construction, and the three expert fields plus moe_enabled
assume every gemma4 GGUF is MoE.

Suggested fix

Make the expert block conditional on the metadata actually present:

is_moe = m.get("gemma4.expert_count") is not None

and populate num_experts / num_experts_per_tok / moe_intermediate_size / moe_enabled
from that, leaving the dense path to feed_forward_length (already read above).

Relationship to existing issues

• Gemma-4 E2B QAT GGUF fails to load: scalar attention.head_count_kv indexed as a per-layer list #63 is also a Gemma-4 GGUF config-parse bug (scalar attention.head_count_kv), but a
  different key and a different failure; for a dense checkpoint this one is the next blocker
  behind it.
• gemma4 GGUF: embedding quant type hardcoded to Q6_K, so a Q4_0-embedding GGUF fails to load with a bare AssertionError #188 is the same class of unconditional-assumption bug one layer down (embedding quant
  type hardcoded to Q6_K).

Environment

• FreeToken 0.1.2 (Desktop, native Windows venv at AppData\Local\FreeToken\venv, Python 3.12)
• RTX 5090 (sm_120, 32 GB), driver 616.56, CUDA 13
• Windows 11 26200

[Cyber-Marty]

Confirmed and fixed locally. Root cause is exactly as described: parse_gguf_config requires gemma4.expert_count via a strict g() getter and hardcodes moe_enabled=True, while the HF-side gemma4.config.parse_config already handles dense correctly (num_experts = getattr(cfg, "num_experts", None) or 0 → moe_enabled = num_experts > 0).

The GGUF parser now mirrors that behavior: expert fields default to 0 when absent, moe_enabled is derived, and expert_quant/moe_weight_format are gated on it (dense → "none", MoE → "q4_0" as before). Dense GGUFs parse through the full geometry path.

Added tests/models/test_gemma4_gguf_config.py — the fixture writes a minimal GGUF header (magic + v3 + zero counts) so the parser runs against metadata dicts with no model download: MoE keys → unchanged moe path; keys absent or zero → dense path. PR incoming.