Coding Agent Model Settings

Model Settings

New in 0.7.0; thinking, GPU split, draft tuning and the memory estimate added in 0.8.0. You can choose the model and tune it without leaving the Coding Agent.

Switching models

Click the model name in the top bar to open a menu of the models in RulZ LLM Suite. Models that are already loaded ("ready") are listed first and start right away. A model that isn't loaded is loaded by RulZ on the first request.

You can't switch while a task is running, so a task never changes models halfway through. The same choice is also in Settings → AI model.

Model settings… at the bottom of the menu opens Settings. Under the model picker, click Model settings to expand it. It has two parts.

How it writes

These settings change how the model writes its answers. They're saved per model in the Coding Agent, and an empty box uses the recommended value shown in grey.

Setting	What it does
Creativity (temperature)	Lower gives more predictable code, higher gives more varied answers. Range 0 to 2.
Top P	Only considers the likeliest words that together add up to this share. Range 0 to 1.
Top K	Only considers this many of the likeliest words. 0 means no limit.
Repeat penalty	Values above 1 discourage the model from repeating itself.
Longest answer (tokens)	The upper limit for one reply or tool call. Default 16384. The agent also keeps it within what's left of the context.

Thinking

For models that can reason before answering, such as Qwen3 or Gemma 4. Coder models usually can't, so these settings do nothing for them.

Setting	What it does
Think before answering	Model's default, On or Off.
Reasoning effort	Minimal to Max, for models whose template takes one. Template's default leaves it to the model.
Thinking budget (tokens)	The most it may think before it must answer. Empty means no limit, and 0 skips thinking entirely, so no thoughts show (a warning next to the field says so since 0.10.0).
When the budget runs out, add	An optional nudge such as "Time to answer." Shown once a budget is set.

Off wins over the effort and the budget. These are sent the same way RulZ LLM Suite's chat sends them. To see what the model thought, open the Thinking box above its reply (Projects and Tasks).

Whether that box stays open after the answer starts is not a per-model setting: it's the Keep thinking open after the answer switch near the top of Settings (since 0.11.0), and it applies to every model.

Recommended values: Qwen3-Coder models use Qwen's published settings (temperature 0.7, top P 0.8, top K 20, repeat penalty 1.05). Other models use RulZ LLM Suite's defaults. Back to recommended clears your values for that model. Changes apply from the agent's next step. (Summaries of older conversation keep a fixed low temperature.)

How it loads

These settings decide how RulZ LLM Suite loads the model. They're saved in RulZ LLM Suite as that model's load defaults, the same ones its Models page uses, so both apps always agree.

Setting	What it does
Context length	How much the agent can keep in mind at once. Empty means as much as fits in memory. The hint shows the model's maximum. Bigger projects need more.
GPU layers	How much of the model goes on the graphics card: Auto, All, or a number.
Flash attention	Faster and uses less memory on most GPUs. Auto, On or Off.
Memory cache type (K / V)	q8_0 roughly halves the cache's memory with little quality loss. A quantized V cache needs flash attention.
Speed up with a draft model	Uses the model's draft sidecar (MTP) for speculative decoding, if it has one. The hint says so when the model has none.

GPU split

Shown when you have two or more graphics cards.

• All, automatically lets llama.cpp spread the model over every GPU.
• One GPU only puts it all on the GPU you pick.
• Custom split shows a slider per GPU. The sliders always add up to 100%, and moving one rescales the others. Next to each slider, RulZ's estimate shows how much that GPU will hold against its free memory, in red if it won't fit. A new custom split starts in proportion to each card's memory.

Draft model (MTP)

Shown when Speed up with a draft model is on and the model has a draft, or there are draft files to pick from. The switch's hint says what Automatic uses.

Draft model (RulZ LLM Suite 0.3.9 or later) picks which draft RulZ uses:

• Automatic: what RulZ found with the model. That's its built-in MTP layers if it has them, or else a draft file in the same folder.
• Built-in MTP layers: many Qwen 3.5 and later "MTP" models carry these inside the model file, so no separate file is needed.
• A draft file from anywhere in RulZ's models folder, shown with its folder. This lets other quantizations of the same model share one mtp-*.gguf.

The draft must come from the same base model, or it won't load or will only slow things down. Run the draft on is hidden for built-in layers, because they run on the model's own weights.

Setting	What it does
Tokens to draft per step	More can be faster if most drafted tokens are accepted. RulZ LLM Suite defaults to 2.
Fewest tokens worth drafting	Skips drafting below this.
Least draft confidence	Stops drafting when the draft is less sure than this (0 to 1).
Draft sampling	Auto, Greedy or Probabilistic.
Run the draft on	Auto, GPU or CPU.
Draft cache type (K / V)	The draft's own KV cache types.

Memory estimate

Under the settings, a badge shows RulZ's estimate: Fits in VRAM, Partly on CPU (slower), Likely out of memory or VRAM unknown, with how much goes on the GPU and in RAM. It updates shortly after each change ("with these changes") without saving anything.

Changes wait until you press Save (or Save and restart model when the model is loaded), because saving restarts a loaded model and it's unavailable for a moment. This can't be done while a task is running. Press Stop or wait first. Cancel throws away unsaved changes.

The line next to the button shows whether the model is loaded and with how much context. "Using RulZ LLM Suite defaults" means this model has no saved settings of its own yet.

Requirement: load settings need RulZ LLM Suite 0.3.5 or later, and the GPU split, draft tuning and memory estimate need 0.3.7 or later. With 0.3.5 or 0.3.6 those controls are hidden, and only the settings you change are sent. With an older version, this section says to update it. You can still change these settings on RulZ LLM Suite's Models page. (Technical details: load-config endpoint.)

From the phone

Since RulZ Coding Agent for Android 0.3.0, the phone has the same Model settings (Settings → AI model → Model settings), and tapping the model under the project name opens a quick switch. Changes made there are saved on the computer, so both stay in sync. Older phone versions can only pick the model; their tasks still use the settings saved here.