Repository navigation
v0.8.0 - native Gemma-4-12B coder (MLX)
KrillLM v0.8.0 adds native MLX support for the Gemma-4-12B coder fine-tune (gemma-4-12B-coder-fable5-composer2.5), converted GGUF-free.
- New model:
krillm run gemma-4-12b-coderruns it on the nativegemma4_unifiedruntime (nvfp4, ~25 tok/s on a 24GB Mac). Published atsrv-sngh/gemma-4-12B-coder-fable5-composer2.5-nvfp4. - GGUF-free conversion: ingested from the upstream NVFP4 safetensors (compressed-tensors), not lossy GGUF k-quants.
tools/convert_gemma4_compressed_nvfp4_to_bf16.pydecompresses NVFP4 to bf16 (pure numpy, self-checking), then the proven mixed-nvfp4 requant (8-bit attno_proj+ projectors). - Reasoning toggle:
enable_thinkingfor the Gemma-4 channel template (KRILL_ENABLE_THINKING) so the coder reasons before answering. Stock Gemma-4 prompting unchanged. - Standalone eval harnesses:
coding_eval_standalone.py(HumanEval pass@1, optional--think) +agentic_eval_standalone.py.
Indicative numbers (our own harness, single greedy pass, not EvalPlus-leaderboard-comparable): HumanEval pass@1 89.6% reasoning-on / 82.9% off (n=164); 8/8 on an 8-scenario tool-use suite.
Install: brew tap srvsngh99/krillm && brew install krillm (or brew upgrade krillm) - Use: krillm run gemma-4-12b-coder
Asset: krillm-0.8.0-arm64-apple-macos.tar.gz (Apple Silicon). Changelog: [0.8.0].