Repository navigation
Model Settings
New in 0.7.0; thinking, GPU split, draft tuning and the memory estimate added in 0.8.0. You can choose the model and tune it without leaving the Coding Agent.
Click the model name in the top bar to open a menu of the models in RulZ LLM Suite. Models that are already loaded ("ready") are listed first and start right away. A model that isn't loaded is loaded by RulZ on the first request.
You can't switch while a task is running, so a task never changes models halfway through. The same choice is also in Settings → AI model.
Model settings… at the bottom of the menu opens Settings. Under the model picker, click Model settings to expand it. It has two parts.
These settings change how the model writes its answers. They're saved per model in the Coding Agent, and an empty box uses the recommended value shown in grey.
| Setting | What it does |
|---|---|
| Creativity (temperature) | Lower gives more predictable code, higher gives more varied answers. Range 0 to 2. |
| Top P | Only considers the likeliest words that together add up to this share. Range 0 to 1. |
| Top K | Only considers this many of the likeliest words. 0 means no limit. |
| Repeat penalty | Values above 1 discourage the model from repeating itself. |
| Longest answer (tokens) | The upper limit for one reply or tool call. Default 16384. The agent also keeps it within what's left of the context. |
For models that can reason before answering, such as Qwen3 or Gemma 4. Coder models usually can't, so these settings do nothing for them.
| Setting | What it does |
|---|---|
| Think before answering | Model's default, On or Off. |
| Reasoning effort | Minimal to Max, for models whose template takes one. Template's default leaves it to the model. |
| Thinking budget (tokens) | The most it may think before it must answer. Empty means no limit, and 0 skips thinking. |
| When the budget runs out, add | An optional nudge such as "Time to answer." Shown once a budget is set. |
Off wins over the effort and the budget. These are sent the same way RulZ LLM Suite's chat sends them.
Recommended values: Qwen3-Coder models use Qwen's published settings (temperature 0.7, top P 0.8, top K 20, repeat penalty 1.05). Other models use RulZ LLM Suite's defaults. Back to recommended clears your values for that model. Changes apply from the agent's next step. (Summaries of older conversation keep a fixed low temperature.)
These settings decide how RulZ LLM Suite loads the model. They're saved in RulZ LLM Suite as that model's load defaults, the same ones its Models page uses, so both apps always agree.
| Setting | What it does |
|---|---|
| Context length | How much the agent can keep in mind at once. Empty means as much as fits in memory. The hint shows the model's maximum. Bigger projects need more. |
| GPU layers | How much of the model goes on the graphics card: Auto, All, or a number. |
| Flash attention | Faster and uses less memory on most GPUs. Auto, On or Off. |
| Memory cache type (K / V) |
q8_0 roughly halves the cache's memory with little quality loss. A quantized V cache needs flash attention. |
| Speed up with a draft model | Uses the model's draft sidecar (MTP) for speculative decoding, if it has one. The hint says so when the model has none. |
Shown when you have two or more graphics cards.
- All, automatically lets llama.cpp spread the model over every GPU.
- One GPU only puts it all on the GPU you pick.
- Custom split shows a slider per GPU. The sliders always add up to 100%, and moving one rescales the others. Next to each slider, RulZ's estimate shows how much that GPU will hold against its free memory, in red if it won't fit. A new custom split starts in proportion to each card's memory.
Shown when the model has a draft head and Speed up with a draft model is on.
| Setting | What it does |
|---|---|
| Tokens to draft per step | More can be faster if most drafted tokens are accepted. RulZ LLM Suite defaults to 2. |
| Fewest tokens worth drafting | Skips drafting below this. |
| Least draft confidence | Stops drafting when the draft is less sure than this (0 to 1). |
| Draft sampling | Auto, Greedy or Probabilistic. |
| Run the draft on | Auto, GPU or CPU. |
| Draft cache type (K / V) | The draft's own KV cache types. |
Under the settings, a badge shows RulZ's estimate: Fits in VRAM, Partly on CPU (slower), Likely out of memory or VRAM unknown, with how much goes on the GPU and in RAM. It updates shortly after each change ("with these changes") without saving anything.
Changes wait until you press Save (or Save and restart model when the model is loaded), because saving restarts a loaded model and it's unavailable for a moment. This can't be done while a task is running. Press Stop or wait first. Cancel throws away unsaved changes.
The line next to the button shows whether the model is loaded and with how much context. "Using RulZ LLM Suite defaults" means this model has no saved settings of its own yet.
Requirement: load settings need RulZ LLM Suite 0.3.5 or later, and the GPU split, draft tuning and memory estimate need 0.3.7 or later. With 0.3.5 or 0.3.6 those controls are hidden, and only the settings you change are sent. With an older version, this section says to update it. You can still change these settings on RulZ LLM Suite's Models page. (Technical details: load-config endpoint.)
The Android app can pick the model (its Settings page), and every task it starts uses the model settings saved here. Changing the model settings themselves is only possible on the computer for now.
Using the app
- Getting Started
- Projects and Tasks
- Approvals and Safety
- Model Settings
- Run and Share
- Notes and Instructions
- Extra Tools (MCP)
- Phone Access
Reference
Contributing