My local model setup for REA on Linux (one 12 GB GPU, five models compared)

[CassieFox23]

TL;DR: Here is how I run REA with free local models on one 12 GB GPU. A script runs REA's CLI, and a local model reads the output. I gave five models the same REA output and the same questions. The two 27B models read it best, and neither made anything up. The small fast ones got numbers wrong.

Setup

Part	What I use
GPU	Radeon RX 6700 XT, 12 GB
CPU and RAM	Ryzen 9 5900X, 32 GB
OS	Nobara 44 (Fedora-based Linux)
Model server	llama.cpp, behind llama-swap, which loads one model at a time
REA	6.3.0 through the CLI, with Ghidra 12.1.4 and JDK 21. Not MCP: the full tool list is far bigger than these models' context.

How the test works

1. The target is a small C program of my own. It takes one word, hashes it, and prints "Vault open." or "Wrong word.". The compiler turns the secret word into a length and a hash, so the word is not in the file.

2. REA runs three calls once, and every model reads the same output:

   Call	Flags
   rea inspect vault	--provider ghidra --filter-output normalized_result --token-limit 1500
   rea decompile vault main	--provider ghidra --filter-output raw_result --token-limit 1500
   rea search vault . --mode regex --kind strings	--provider ghidra --filter-output raw_result --token-limit 1500

3. Three questions, asked three times per model: What does this program do? What input makes it succeed? What is the secret it checks for? Each answer also names which REA output it used.

4. Scoring. The best possible answer says the word has 8 characters and its hash must equal 0xBB380CD4. It also says the word can't be read from these outputs. I counted that as "partly right". A wrong answer, a made-up word, and any wrong hex value were counted separately. I checked every number with a script.

Results

Model	Best possible (of 9)	Asks with a wrong number	Made up	Speed
Qwen3.8-27B	9	0 of 3	none	7.9 tokens/s
Qwen3.8-27B, 3-bit with MTP	9	0 of 3	none	11.3 tokens/s
KAT-Coder-V2.5-Dev	9	2 of 3	none	26.7 tokens/s
Qwen3.5-9B	9	2 of 3	none	50.0 tokens/s
Qwen3.5-4B	5	3 of 3	1	79.6 tokens/s

Some "best possible" answers still had a wrong number in them, and the middle column counts those asks.

• Best reader: both 27B models. They gave the best possible answer every time, with no wrong numbers. The 3-bit one is faster.
• Fast but careless: the 4B and 9B are quick, but they mangle hex numbers.
• Middle: KAT-Coder is fast enough to feel live, at about 6 s per warm answer. It wrote the hash wrong twice.

REA itself took about 8 s per call on a quiet box. Full speed, memory and power tables are on #1998.

What the answers looked like

Model	Question 2, "what input makes it succeed?"
Qwen3.8-27B	It says the input "must be exactly 8 characters long". Then it says "the exact string cannot be determined" from the code. It gives the target as 0xBB380CD4.
Qwen3.5-4B	"The program succeeds if the input word is 2155." The word is 8 characters, so 2155 can't be right.

What doesn't work yet

Letting a local model drive REA. In my tries, it picked good rea commands, but it didn't stop cleanly, and twice it made up tool output. So for now my main agent plans, and the local model does the reading. Notes on that are in #1958.

Not run

• More asks per model, with a fixed seed. Three asks is a small sample.
• REA 6.4.0's output in the prompt. Its decompile output now comes wrapped in YAML, and I fed the models 6.3.0's.
• Other models and bigger GPUs.

Model files and settings

Model	File I used	Context	Notes
KAT-Coder-V2.5-Dev (Kwaipilot)	IQ4_XS GGUF by bartowski	32,768	MoE, 22 expert layers on the CPU
Qwen3.5-4B	Q5_K_M GGUF by Unsloth	16,384	all on the GPU
Qwen3.5-9B	Q5_K_M GGUF by Unsloth	16,384	all on the GPU
Qwen3.8-27B	UD-IQ4_XS GGUF by Unsloth	32,768	38 layers on the GPU, MTP draft
Qwen3.8-27B, fast	GSQ-RCO IQ3_XXS GGUF by ISTA-DASLab	32,768	52 layers on the GPU, MTP draft

Thinking is off for all five, and the KV cache is q8_0 where set. The temperature was 0.7, the default in my setup.

Licences

Part	Licence	Where I checked it
KAT-Coder-V2.5-Dev	Apache-2.0	the GGUF header and its Hugging Face card
Qwen3.5-4B	Apache-2.0	the GGUF header and its Hugging Face card
Qwen3.5-9B	Apache-2.0	the GGUF header and its Hugging Face card
Qwen3.8-27B, both files	Apache-2.0	the GGUF headers and its Hugging Face card
llama.cpp	MIT	the repo's licence file
llama-swap	MIT	the repo's licence file
Ghidra	Apache-2.0	the repo's licence file
REA	MIT	the repo's licence file

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊

[CassieFox23]

TL;DR: I retried letting KAT-Coder drive REA itself, on a quiet box with a fixed loop. It now finishes every time, but it guesses. It made up the secret word in all 3 runs. The setup in this post, where a script runs REA and the model only reads, stayed honest every time.

What changed in the loop

• An ANSWER: anywhere in a reply now ends the run.
• A blocked command gets a clear reason and one example to copy.
• One simple quoted string is allowed.
• Text that looks like made-up tool output is dropped.
• The model gets 10 steps, with a warning at step 8.

Results

Setup	Finished	Honest answer	Made up
Script runs REA, model reads (3 asks)	3 of 3	3 of 3	nothing
Model picks rea commands (3 runs)	3 of 3	0 of 3	a word in 3 of 3, a wrong hash in 1

• Before these fixes, the loop finished 0 of 2 runs within 8 steps.
• Two runs turned the program's banner into the word "FoxVault". That word fails.
• The model still wrote fake tool output in 2 of 3 runs, even after a warning. The loop threw it away.
• On the quiet box, KAT-Coder wrote 26.4 tokens/s. The busy box earlier gave 4.4.

So local models read REA well, but they don't drive it well yet. When they drive, they guess. I'm keeping the script-runs-REA setup.

Not run

• A stronger "unknown is a fine answer" line in the prompt.
• Other models, or more runs.

🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊