Repository navigation
Using Custom models.ini in Llama.app #111
|
I am using llama.cpp (b10282-a035a8887) compiled from source on my Mac. I have a custom models.ini file defined below. [unsloth/Qwen3.6-27B-MTP-GGUF:UD_Q6_K_XL]
model = Models/GGUF/Qwen3.6-27B-MTP-GGUF/Qwen3.6-27B-UD-Q6_K_XL.gguf
mmproj = Models/GGUF/Qwen3.6-27B-MTP-GGUF/mmproj-F16.gguf
; CORE TUNING FOR M4 PRO
n-gpu-layers = all ; Forces full Metal GPU offload (critical for speed)
threads = 8 ; Matches your 8 Performance cores (avoids E-core contention)
threads-batch = 8 ; Aligns with P-cores for prompt processing
flash-attn = on ; Reduces KV cache recomputation overhead on Metal
kv-unified = true
; MEMORY BANDWIDTH OPTIMIZATION (273 GB/s limit)
cache-type-k = q4_1 ; Compresses K-cache: ~60% less bandwidth than f16, near-lossless
cache-type-v = q4_1 ; Compresses V-cache: same benefit
kv-offload = true ; Keeps KV cache on GPU memory (default, but explicit)
; MTP SPECULATIVE DECODING TUNING
spec-type = draft-mtp
spec-draft-n-max = 2 ; Reduce to 1 for long context
spec-draft-p-split = 0.16 ; Higher threshold trusts MTP drafts more aggressively on Metal
spec-draft-p-min = 0.08 ; Prevents fallback to slow greedy decoding too early
temp = 0.7
top-p = 0.8
top-k = 20
presence-penalty = 1.5
min-p = 0.00
chat-template-kwargs = {"preserve_thinking":true}
[bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF:Q6_K]
model = /Models/GGUF/models--bartowski--Kwaipilot_KAT-Coder-V2.5-Dev-GGUF/snapshots/d8f684f08d2950ea9d2db6a35ef7dada0707858b/Kwaipilot_KAT-Coder-V2.5-Dev-Q6_K.gguf
ctx-size = 65536
ubatch-size = 2048
I wanted to try out llama.app on my system and downloaded the binary for mac os. The ini file for llama.app is located at [bartowski/Kwaipilot_KAT-Coder-V2.5-Dev-GGUF:Q6_K]
model = Models/GGUF/models--bartowski--Kwaipilot_KAT-Coder-V2.5-Dev-GGUF/snapshots/d8f684f08d2950ea9d2db6a35ef7dada0707858b/Kwaipilot_KAT-Coder-V2.5-Dev-Q6_K.gguf
ctx-size = 262144
ubatch-size = 2048I tried replacing it with my custom file but it keeps reverting to the original file during runtime. I even tried deleting the app, the Llama folder in Application Support and the .llama-app folder in home and then reinstalling the dmg but the same issue persists. Is there a way for me to use my models.ini file with llama.app instead of the default one? |
Replies: 3 comments 3 replies
|
I have seen discussions regarding adding cli arguments. However, my issue seems to be another angle to this through adding custom arguments with the ini file. |
|
There's no way to do this today -- the app regenerates You're not the first to hit this. Someone else wanted to point a model at a draft model from another repo and ran into the same wall. We want to give power users a proper way to configure this. |
|
Update: 0.41.0 adds this. Put your section in Your config should carry over as-is. You can drop the |
Update: 0.41.0 adds this.
Put your section in
~/.config/llama/models.user.iniand the app merges it into themodels.iniit generates, your keys winning. That file is yours -- the app only ever reads it, so nothing overwrites it.Your config should carry over as-is. You can drop the
modelandmmprojlines, the app fills those in.