v0.0.33

v0.0.33 — Claude Code runs on a model Chaos serves

Point claude at a Chaos node and a model on your own machine drives the agent.
It reads files, writes them, and runs commands through Claude Code's own tools.

$ claude-chaos "read notes.txt and tell me what city it names"
The city named in notes.txt is Lyon.        (called Read, quoted line 1)

$ claude-chaos "create hello.py that prints 1 to 5"
hello.py created.                            (called Write)

Verified end to end against a local Qwen3-4B, with real files on disk.

Read the speed section before you plan a day around it.

How to use it

Two steps. Start a node, then run the wrapper — both now ship with Chaos:

chaos-serve <model.gguf> --port 8231 --context 16384
claude-chaos "your prompt"

On Windows there is a button instead: USE WITH CLAUDE CODE on the CHAOS
page. It checks Claude Code is installed and offers the npm command if not,
checks a model is loaded, asks which project folder, and opens a terminal with
everything set.

docs/CLAUDE-CODE.md ships in every archive and is the whole path from nothing
to a working turn.

Pick the model on whether it calls tools

This is not the same as picking the best model at code, and getting it wrong
looks like the agent refusing to work:

model	calls tools?
Qwen3-4B (2.3 GB)	yes, measured twice — read a file and wrote one
Qwen2.5-Coder-7B-Instruct (4.4 GB)	no. Printed the code and said "you can save this as hello.py"; on a second try suggested a shell command. Never called the tool

A model that will not emit a tool call is unusable here however good its code
is: it connects, converses, and changes nothing. Chaos does not paper over it —
a malformed or absent call stays text, because inventing a tool_use block
would make the agent run something the model never asked for.

--tools decides whether it works at all

Claude Code's default tool set does not fit in any model this size. Measured
with a real tokenizer:

tools	definitions	tokens before you type anything
default	28	40,255
the six the wrapper uses	6	11,706
none	0	9,155 — Claude Code's own system prompt

Against a 32,768-token context, the default set leaves no room for a
conversation. The wrapper restricts it for that reason.

Speed, measured

Qwen3-4B on an i7-13650HX with 15.7 GiB:

reading a file    turn 1  386.0s  tool_use    turn 2  52.9s
writing a file    turn 1  352.4s  tool_use    turn 2  67.4s

Turn 1 is the expensive one — about six minutes of reading the prompt. After
it the node keeps the KV cache and later turns pay only for what changed, which
took turn 2 from 135.6 s to 52.9 s. The reuse is verified not to change the
answer: the same turn warm and cold produces identical output.

So: keep one node running (restarting throws the cache away), and expect
minutes per turn on a CPU machine. A four-turn task is a coffee break. What
changes that is hardware — a machine that fits the model in VRAM prefills in
seconds.

Also fixed

• chaos-serve refused any prompt over 2,048 tokens on the dense path, and
  -c could only lower it. This made every agent client impossible, not just
  Claude Code — an editor sending one file for context exceeds it. The prefill is
  chunked now and the ceiling is 16,384.
• A reasoning model's <think> working was returned as its answer. The first
  live request spent its whole budget reasoning and returned the reasoning.
• The server could wedge on a client that abandoned a request — the port
  stayed open with connections in CLOSE_WAIT and new ones timing out. Bounded
  now.
• Two instrument defects: nothing checked that a control the app declares is
  actually laid out, and the run-through script's control lists were written by
  hand, so a new button could be on screen and unmentioned. Both closed — and the
  new sweep found six controls that had never been exercised.

What is not built

• Token-by-token streaming. The answer is buffered and sent as one set of
  events: a tool call cannot be recognised until it has been seen, and text
  already sent cannot be recalled. At about 1 tok/s this is not what you notice.
• Prefix reuse on the V4-Flash path. Its cache cannot be truncated to an
  arbitrary position, so that path prefills from scratch. At 0.728 tok/s it is
  not the model to drive an agent with anyway.
• Prompt caching across restarts. The cache lives as long as the node.
• Android still cannot install over the previous release — no signing key.
  Uninstall first. bash scripts/make-release-keystore.sh fixes it permanently
  for every release after the secrets are set.

Verified for this release: 1032 tests passing (0 failed, 50 ignored), clippy
-D warnings and fmt clean, 34 surface checks against the built binaries, and
the full window run-through with nothing blocking longer than 48 ms.