CAGE 0.1.2

What's New in CAGE 0.1.2

0.1.2 has one theme: tool calls stop being a guessing game. The format a
model uses to call a tool is now taken from that model's own chat template, and
the model is constrained at sampling time so it cannot produce anything else.

It is a breaking release for backend packs: a pack built before 0.1.2 is
refused. See below.

---

Before you upgrade

What changed	What you have to do
Tool-call parsing moved into the backend pack	Re-download the backend pack for your GPU. A 0.1.1 pack will not load.
Nothing else	The server configuration, database schema and extension settings are unchanged from 0.1.1.

The extension and server are compatible across this bump; the pack is the only
mandatory download.

---

Tool calls now come from the model's own template

Every model family writes tool calls differently — Qwen tags them, GLM tags them
differently, Kimi uses its own control tokens, Llama-3 emits bare JSON. Until now
CAGE told the model its own format in the system prompt and then tried to
recognise whatever came back with five competing regular expressions.

That was the wrong shape, and it produced exactly the failure you would expect:
the model's chat template already declared a format, our prompt declared a
different one, and the model drifted between them. When it drifted, one of the
five parsers usually matched something — but not always the right thing.

0.1.2 removes the guesswork in three steps:

• The prompt is generated from the chat template. No hand-written
  instructions about tool syntax, no invented example call ids.
• The reply is parsed by the parser that template ships with, not by pattern
  matching over prose.
• The model is constrained while sampling. A GBNF grammar built from the
  template means a malformed tool call is not merely detected, it is
  unrepresentable — the tokens that would produce one are never candidates.

The five hand-rolled parsers, the hold-back heuristic that guessed how many
trailing bytes might be a partial tool marker, and the model-specific special
casing are all deleted — about 1200 lines.

Measured, not assumed: 14 of 14 local models are recognised by their
template; 11 of 14 can additionally be grammar-constrained (the other three fall
back to template-driven parsing without the grammar). Constrained decoding costs
almost nothing — 33 tok/s against ~36 unconstrained — because sampling stays on
the fast path until the unconstrained pick is rejected.

---

Backend packs must match the server

The code that reads a chat template lives in the backend pack, next to the
inference code it belongs with. A pack from 0.1.1 does not contain it, so the
server refuses to load one:

missing chat-template symbols […]. This backend pack predates the
chat-template shim (0.1.1 or earlier). Download the backend pack that
matches this server.

This walks back a claim made in 0.1.1's notes — that the server and the pack
version independently and upgrading one would not mean re-downloading the other.
That holds for pure kernel updates. It does not hold here, because 0.1.2 moves a
capability across that boundary. Loading an old pack would mean falling back to
the guessing behaviour this release exists to remove, and a silent regression in
tool calling is far harder to diagnose than a refused load.

All eleven packs are rebuilt for 0.1.2.

---

Long generations are no longer cut off

The generation timeout was a deadline for the whole turn: five minutes from the
first token, no matter what. Any legitimately long reply hit it and was reported
as a stall.

It is now a stall timeout — the clock resets on every token, and fires only
when nothing has been produced for the timeout window. A 2894-token file that
used to be truncated at the 300-second wall now completes in 88 seconds and is
written out whole.

The same mistake was found on the way in: reading a long prompt had its own
three-minute deadline, so a large context was reported as a stall even while every
batch was making progress. That is now a stall timeout too, and the message says how
far it got.

---

Retrieval no longer takes a GPU you did not give it

If you enabled RAG, the embedding model ignored the GPUs you picked for it and
loaded onto the first one it found. On a machine with a display card and compute
cards, that is usually the display card — and nothing said so.

Every other model in CAGE honours the GPU selection on its AI config. The embedder
was the one that did not, on both the in-process and vLLM paths. It does now, and it
says which devices it took.

---

Retrieved code is labelled as excerpts

Context retrieved for a prompt is a set of fragments chosen by similarity — not whole
files, not in order. That was never stated in the prompt, and models drew conclusions
from it that the fragments do not support: in one direction assuming a project was
already finished, in the other reporting a snippet that ends mid-statement as broken
code. The second one is worth naming because it is convincing: a reviewing step
produced a list of compiler errors whose line numbers were exactly the chunk
boundaries, for a program that compiled and ran correctly.

The block now says what it is. If you are building a logic graph, also note the
matching rule: a node that judges a piece of text rather than the codebase should
have RAG turned off on that node. Retrieval helps the steps that act on code; it
misleads the ones that grade an answer.

---

Fixes worth naming

• A backend pack that is too old fails at load, with the reason and the fix,
  instead of loading and then behaving incorrectly.
• Tool arguments that contain a JSON object are no longer mis-split by a
  parser looking for the next closing brace.
• Streaming no longer holds back trailing text waiting to see whether it was
  the start of a tool marker.

---

Known gaps

Unchanged from 0.1.1 except where noted:

• Three of fourteen models expose a chat template but no usable grammar; they
  run with template-driven parsing only.
• The server binary still reports no version of its own; /health does not
  return one.
• The AMD detection path has not been exercised on real AMD hardware.
• Multi-node exists in the control plane, but each client connects to a single
  server — there is no scheduling across machines.
• Server-side session history is recorded but not replayed into a conversation
  resumed after a restart.
• mesh-trellis still needs an upstream interactive setup step.
• Pascal and older NVIDIA cards are not supported.

CAGE 0.1.1

What's New in CAGE 0.1.1

0.1.0 was the first evaluation build. 0.1.1 is the first release where the
product is shaped the way it will keep being shipped: a small server, GPU
support downloaded separately for your hardware, and files that land in your
workspace the moment you approve them.

It is also a breaking release in both directions. The project was renamed,
the workflow layer was removed, and GPU inference now requires a second
download. Update the server and the editor client together.

---

Before you upgrade

What changed	What you have to do
Everything is called cage, not aiwrapper	Edit settings.json — the old aiwrapper.* keys are silently ignored. Uninstall the old side-loaded VSIX; the extension id is new.
The MySQL database and user were renamed	Run server/sql/migrate-aiwrapper-to-cage.sql before starting the new server.
GPU inference moved out of the server binary	Download the backend pack for your GPU as well as the server package. See below.
The workflow layer is gone	Nothing to migrate — but an old client cannot talk to a new server, or the reverse.
Build environment variables are CAGE_*	Update any shell profile that exports AIW_*.

Existing build directories cache the old option names. Reconfigure from scratch
rather than incrementally.

---

GPU support is a separate download now

The server used to be a 1.7 GB binary, almost all of which was compiled GPU
kernels for architectures your machine does not have. It is now 49 MB with no
CUDA dependency at all — verified: zero CUDA shared libraries, zero
unresolved llama_ symbols. The GPU code lives in a backend pack that the
server opens at runtime.

What this buys you:

• You download kernels for your card, not for everyone's. One pack per GPU
  family instead of one binary carrying all of them.
• Non-NVIDIA hardware is reachable. There is a Vulkan pack, which runs on
  AMD, Intel and NVIDIA alike, and a CPU-only pack that needs no GPU at all.
• New architectures do not require a new server. A pack for a newer card is
  a separate, much smaller release.
• Updating the server does not mean re-downloading a gigabyte of kernels.
  The two halves version independently.

Packs are named cage-backend-<hardware>-<version>-<platform>.tar.gz. Unpack
one and put libcage-llama-*.so in the server's bin/.

Pack	Covers
cuda-sm70_sm75	Volta, Turing — V100, T4, RTX 20xx, GTX 16xx
cuda-sm80_sm86	Ampere — A100, A40, RTX 30xx
cuda-sm89_sm90	Ada, Hopper — L40S, RTX 40xx, H100
cuda-sm120_sm121	Blackwell — RTX 50xx, GB10 (DGX Spark)
hip-*	AMD via ROCm, one pack per ISA generation (RDNA2/3/4, CDNA)
vulkan	Any Vulkan 1.2 GPU — AMD, Intel, or NVIDIA
cpu	No GPU

Exactly how far each piece has been tested is listed in
verification status.

The AMD packs are new and unproven. They compile, load, resolve every symbol
and identify themselves correctly — but all of that was verified on a machine
with no AMD hardware. Nobody has yet confirmed they compute the right answers or
how fast they are. If you have an AMD card, that is exactly the report we need.
They also need four ordinary distribution packages the pack cannot ship for you:
libdrm2 libdrm-amdgpu1 libnuma1 libelf1.

You can install more than one. The server probes every pack it finds, per
device, and picks per AI config — so a machine with both an NVIDIA and an AMD
card can drive each with the backend that suits it.

If you download the wrong pack, you are told so directly: cage-backend-probe
reports that the pack carries no device code for your card, instead of the
server failing later when a model loads. A machine with no pack still starts
normally and still runs vLLM and remote endpoints — only in-process GGUF
inference is unavailable, and the error says which pack to fetch.

---

Files land in your workspace as they are written

0.1.0 accumulated write_file and apply_patch results in a staging area and
asked you to flush them in a batch, before any tool that would see the
workspace. In practice that failed three ways: the model was told its file did
not exist yet and rewrote it over and over trying to make it stick, sometimes
resorting to shell heredocs to get around it; a turn that never ran a terminal
command left every write suspended, so nothing happened at all; and when the
prompt did come, it asked you to review a pile of unrelated files at once.

Staging is gone. write_file and apply_patch now ask before each
individual file, write straight to your real workspace once approved, and tell
the model the file is on disk — or that you declined, in as many words.

Approvals are serialised: one question at a time, the rest queued in arrival
order, with later tool calls held until you answer. An apply_patch touching
five files asks five times, each with the diff for that file, syntax-coloured.

The working directory is always your workspace root. The server does not decide
where files go.

---

GPU detection is no longer NVIDIA-only

Hardware detection shelled out to nvidia-smi and asked CUDA for PCI
addresses, so on anything else the GPU list came back empty and automatic
placement had nothing to work from.

There are now four detection backends chosen at runtime: nvidia-smi, AMD's
KFD interface (including XGMI links, with no rocm-smi dependency), Apple's
sysctl, and a Vulkan fallback that works on any vendor.
CAGE_GPU_PROBE=nvidia|amd|apple|vulkan|none forces one.

Honest limits: the AMD path has not been run against real AMD hardware. And
Vulkan reporting no interconnect is correct rather than missing — split modes
that need one are not available on Vulkan, so the choice there is made from
free VRAM instead of topology.

---

vLLM is optional: safetensors convert themselves

backend="vllm" used to mean "unavailable unless you got a vLLM environment
working", which is not a given — it has to match your GPU generation, CUDA
toolkit and torch build simultaneously.

Now the server probes vLLM on first use and, if it is absent or broken, falls
back to converting the weights to GGUF and running them on the llama backend.
Quantization still follows your AI config; 4bit converts to f16 and quantizes
to Q4_K_M in-process.

The converted GGUF is never written to disk. It goes to tmpfs and is
unlinked as soon as the model has loaded — no cache to manage, nothing to
invalidate. Budget eviction means reconverting, and the tmpfs directory needs
the room in RAM while it runs, which is checked before starting.

Verified end to end on Qwen3-0.6B safetensors with vLLM unset and with a
deliberately broken interpreter path: convert 10 s, quantize 6 s, 456 MiB
Q4_K_M, loaded and generating; the second request reused the loaded model in
1.3 s with no reconversion.

---

The project remembers what it changed

Project memory had exactly one writer — a tool the model was asked to call
"when you have just done or learned something non-obvious and durable". Across
three test runs it never once decided to. The memory directory was never
created, and every turn injected zero memory files.

History is a log of how a file was changed and why. It should not depend on the
model remembering to write it. The server now records it directly: at the end
of a turn, one entry per file that was actually written, taken from the
tool arguments rather than from the model's own summary — summaries claim to
have created files that were never touched, and following them would invent
history for files that do not exist. The reason comes from the node's summary
output; nodes that produce no summary (a build-verification pass, for example)
write nothing, because they have no reason to give.

Both the local and remote tool loops write it, so a graph running against a
cloud endpoint keeps the same history.

---

AI configs and modality configs are one list

Image, speech, audio, music and 3D configs used to be a separate list with
their own "+ New" and their own editor. For you they were the same thing as an
AI config — pick a model, decide how it loads — differing only in the property
sheet. They are now one AI CONFIGS list, with generative entries labelled by
type, and the property sheet switching to match the model you pick. Choosing a
different kind of model converts the config in place, keeping its name so the
nodes bound to it stay bound.

Multimodal workers are also on the same VRAM ledger as everything else now,
instead of being invisible to it, and they pick GPUs by device rather than by a
configuration whitelist.

---

Register models that are already on disk

cage-model-dl --scan <dir> walks a directory and registers what it finds —
format, quantizations, vision projector and licence — using the same scanner
the server runs at startup. Previously the downloader only went one way (repo
id → download → config), so weights that arrived by any other route had to be
typed into config.xml by hand.

Licence is resolved from the GGUF metadata, then the model card, then the git
remote, then the upstream card. Two traps it handles: a repacker's embedded
model name is whatever they felt like — one GGUF calls itself Downloads, and
searching for that finds a real, unrelated repository whose licence would then
be attached to your model — so name searches are validated against the
directory on disk, never against metadata. And models distilled from restricted
weights advertise the upstream company's licence; those are promoted to the
licence that actually governs the weights, or cleared with a reason where it
cannot be established. Nothing in this path invents a licence it does not have.

Available as cage-model-dl --scan (with --dry-run and --offline) and as an
import panel in the control plane's Models tab.

---

A message names the graph it runs

The workflow layer w...

AIWrapper 0.1.0 — evaluation build

Evaluation build. Proprietary — see LICENSE; licensed for evaluation
and testing only.

Downloads

File	What
aiwrapper-server-0.1.0-linux-x64.tar.gz	Inference server, node agent, control plane, model downloader, Python workers, installer
aiwrapper-0.1.0.vsix	VSCode extension (also in dist/ in the repository)

sha256  9370305c0fcd6b175780439dba0241f9dfa16c770c8a603f3bb7402a5a959c91  aiwrapper-server-0.1.0-linux-x64.tar.gz
sha256  a68d1935604f3993a8bb9364abbb696da5684e611033ae04486222098855bf0e  aiwrapper-0.1.0.vsix

Install

Two paths, both tested — see doc/server-install.md.

Docker, if you have no reason to prefer otherwise:

cd docker
cp ../aiwrapper-server-0.1.0-linux-x64.tar.gz .
cp .env.example .env      # models directory, state directory, passwords
docker compose up -d --build

Native:

tar -xzf aiwrapper-server-0.1.0-linux-x64.tar.gz
cd aiwrapper-server-0.1.0-linux-x64
./install.sh
./setup-workers.sh        # the Python worker environments

Then install the extension and point it at the server:
doc/extension-install.md.

Requirements

• Linux x86-64, glibc 2.38+ (Ubuntu 24.04 or newer)
• NVIDIA driver + CUDA 12 for GPU inference; CPU-only works and is slow
• Optional: MySQL for accounts and sessions

Documentation

• Server guide — the three processes, models, backends, modalities, retrieval, accounts
• Client guide — both windows, every node type and configuration parameter
• Python workers — what each worker family needs, and what was verified