Repository navigation
0.7.0
Changes since the last Release
Added
-
--moe-cache-mib Nongen,chatandservegives the MoE experts kept on CPU a GPU cache of N MiB (upstream #29887, experimental). It needs a GPU backend.servealso gains--cpu-moeand--n-cpu-moe. The Python bindings expose the new fields. Upstream's--spec-draft-n-ubatchhas no counterpart, since chimera has no speculative decoding. -
serve --slot-save-pathreadsLLAMA_ARG_SLOT_SAVE_PATHwhen the flag is absent, asllama-serverdoes. -
make testcovers the new upstream models when their GGUFs are inmodels/. d1-omni-600M and d1-3B on/v1/systemone; embeddinggemma-2 with its mmproj on--embeddings-mmproj, which previously neededCHIMERA_TEST_EMBED_VLM.
Changed
-
Update llama.cpp to b11669 (v0.7.0, from b11429). The chat API now returns the rendered prompt as template and user-input parts, and parses output from text plus tokens. Slot files may carry context-checkpoint and draft-state blocks; files saved by 0.6.0 still restore. At temperature 0 the sampler now selects greedily (#29797). Seeded temp-0 output can differ from 0.6.0.
-
servelistens on port 9931 by default, followingllama-server(upstream #30159). Pass--port 8080for the old port. -
chatsplits reasoning out of the reply by default. It usesdeepseek,llama-server's default, when neither--reasoningnor--reasoning-formatis given.<think>text streams in the think colour, is stored in thereasoningcolumn under--persist, and stays out of the history sent back to the template.--reasoning nonerestores the 0.6.0 output.chatnow drives upstream'scommon_chat_sessionper turn.
Fixed
-
chatandgen --mmprojno longer parse special tokens typed by the user. The rendered prompt was tokenized as one string withparse_special=true, so<|eot_id|>in a message became the end-of-turn token. When a user-input part contains special-token text, each part is now tokenized separately and only template parts parse specials. Other prompts tokenize as one string, as before: splitting at every part boundary changes tokens there on SPM vocabularies withadd_space_prefix.serveis unchanged: upstream's server still tokenizes the joined prompt. -
--reasoning-formathad no effect on howchatparsed output.common_chat_parser_params(const common_chat_params &)does not copyparser, so the content-only fallback parser ran, andreasoning_contentwas never set. -
servepadstensor_buft_overridesas upstream's arg parser does. The list was empty, sollama_params_fitaborted with "did not provide buffer to set tensor_buft_overrides" and could not reduce--gpu-layers -1when the model did not fit. A non-empty list also needs the null terminator, which--cpu-moewould otherwise trip incommon_model_params_to_llama. -
The
gen --mmproj --imagetest skipped whenmodels/held only theQ5_K_Mquant of gemma-4-E4B. It requiredQ4_K_M; it now accepts either.