Sarge: llama-server with a rule check compiled in, so a coding agent can't ignore your repo's rules

[1picassoai]

My coding agent read the rules of my codebase, agreed with them, and shipped code that broke them anyway. So I built a harness around a local model that checks its own output, and the interesting part for this repo is how it's wired.

The check is a Rust static library linked into llama-server. Not a sidecar, not an HTTP call to the server from outside — it's compiled in, so the judging happens in-process. One binary, one model in memory, no IPC. The patch is small: it touches tools/server/CMakeLists.txt and server-context.cpp, and links a static lib via -DHANDSHAKE_LIB. If the lib isn't there, it builds a stock llama-server.

What it does with that. A small model (Qwen3-4B-Instruct-2507, Q4_K_M) writes code. Every file it writes is judged against a rule book that lives in the repo — by the model itself, not a regex — and the run refuses while a rule is broken. When it gets the same thing wrong twice, a tutor writes the corrected rule into the book, so next time the rule is already there before the model starts.

Measured on a 4070 laptop card (8 GB), Q4_K_M: writing 72 tok/s on the GPU, 11.4 tok/s on 4 CPU cores with -ngl 0. A check on one 2,250-token file is about 70 seconds on 4 CPU cores and a few seconds on the card — the check makes one pass over the file with the whole book, then two short confirmations per suspected line, so it's the number of calls that costs, not the length of any one.

Server flags that came out of real failures, in case they're useful to anyone: -c 32768 with -ctk q8_0 -ctv q8_0 so 32k of context fits beside the model on 8 GB; -np 1 because the default four slots of 4096 each cancelled an agent turn mid-reply and it silently rewrote the same bytes; --repeat-penalty 1.1 --repeat-last-n 256 after a turn ran to the token cap repeating one paragraph for 97 seconds.

Honest about what it isn't. Seven tasks on one repo, seven of seven correct; the same model without the harness, none. The check itself caught six of seven known faults with one false flag — it's a local model judging a line, so it's only as good as that model. Rule books ship for Node and C#.

macOS (Apple silicon) shipped today, built with GGML_METAL=ON. CI builds and installs it on an M1 runner, but those runners have no usable GPU, so Metal is compiled and never executed — I don't own a Mac and I'm not publishing a Metal number I haven't measured. If anyone with an M-series machine runs it, I'd genuinely like to know what it does.

Pinned to 89fe242. Install is one command on either platform, every download sha256-verified, MIT.

https://github.com/1picassoai/Sarge