Skip to content

v1.7.0 - Configurable LLM settings

Choose a tag to compare

@freyWylfred freyWylfred released this 13 Sep 23:19
· 2 commits to master since this release

LLM behavior is now configurable via llm-settings.json placed next to the executable (created with defaults on first run, loaded at startup). Supports n_gpu_layers, main_gpu, split_mode, use_mlock, use_mmap, n_ctx, n_batch, n_ubatch, n_threads, n_threads_batch, flash_attention, seed, rope_freq_base/scale, max_tokens, temperature, top_k, top_p, min_p, repeat_penalty, repeat_last_n, frequency/presence_penalty, anti_prompts and system_prompt. See README for details. Requires .NET 10 Desktop Runtime (win-x64).