Qwenputer model: Hugging Face · ModelScope
Filesystem model: Hugging Face · ModelScope
NPU model: Hugging Face · ModelScope
QwenPU, the vector unit: GitHub · Hugging Face

Qwenputer is a new kind of experimental model: Qwen3.5-4B with a 32-bit RISC-V CPU compiled and inserted directly into its weights. It isn't a plugin, a tool or a sandbox. The CPU is 11 transformer layers in front of Qwen's own, and none of their weights are learned: a compiler computes every one.

It uses the model's own Gated DeltaNet state as RAM. Qwen3.5's DeltaNet layers carry a running state that the delta rule can overwrite one row at a time, exactly, so each row works as a memory cell. The CPU keeps its code, data, registers and program counter there.

The CPU runs C and MicroPython code inside the model's forward passes, and passes the official riscv-tests suites.

Greedy decoding, 40 problems per set. Each row compares base Qwen3.5-4B and Qwenputer on the same problems. Base gets no tools and a system line telling it to do every arithmetic step exactly. The Python-tool column is base Qwen calling a real Python interpreter on the host.

We replayed about 230,000 of the computer's output positions through a reference model of the machine and found zero mismatches. Qwen's text behaviour matches base: KL 6e-4, against a noise floor of 5e-4.

git clone https://github.com/divagr18/qwenputer && cd qwenputer/experimental/riscv && uv sync --extra hf
hf download divagr1925/Qwen3.5-4B-Qwenputer --local-dir qwenputer-rv
uv run python -m chipformer.rv.chat qwenputer-rv --firmware tests/data/rv/calc-bridge.elf \
    --ask "Use calc: what is 123456789 * 987654321?"

Qwen uses the CPU entirely internally; it is not invoked through an external tool call. Instead, it piggybacks on the tool-call format Qwen3.5 was trained on: Qwen writes a <tool_call> block for calc exactly as it would for any tool, but nothing outside the model ever runs it. The computer layers read the call, run it on the RISC-V CPU, and the model itself writes the <tool_response>, digit by digit. No runtime, sandbox or tool executor is involved.

The model loads as a stock Qwen3_5ForCausalLM. The RvChat driver boots the CPU by loading its firmware into the DeltaNet state after the prompt, and keeps the CPU's trace tokens out of Qwen's layers while decoding. Every token still comes from the weights. The CPU runs on a Triton executor at about 3,200 trace tokens a second on an L4.

The CPU is RV32IM with Zba and a bignum coprocessor. Its code lives in experimental/riscv/, and experimental/riscv/docs/rv32.md has the design and every measurement.

The computer's RAM is the state of Qwen3.5's own Gated DeltaNet layers. Qwen3.5 mixes full attention with these layers, and each DeltaNet head keeps a state matrix that it updates at every token with the gated delta rule (panel a). In the computer's heads the gate is held open (α = 1, so nothing fades) and the write strength is β = 1. Each key is a one-hot vector picking a single row, so every update replaces exactly that row with the value and leaves the rest untouched. That's a RAM write. A RAM read is the same head's query: a one-hot query returns exactly that row (panel b).

The rest is layout (panel c). The low bits of an address choose the head and the remaining bits choose the row, and each row stores one byte as bits. The machine's memory is spread over a few dozen heads, and its registers, program counter and call stack each get heads of their own. Each key head drives two value heads, which doubles the capacity. The gate is useful too: α = 0 for one token wipes a whole head (that's how the CHIP-8 screen is cleared), and α = ½ makes a head halve every step, which the circuit uses as a counter.

Writes arrive through the token stream, so the trace is the computer's execution log. An ADDR token names the row, the VAL token after it carries the value, and STEP ends the instruction. Nothing here needed a new layer type: the computer's memory is the same Gated DeltaNet recurrence Qwen3.5 already uses for language.

The CPU can run neural networks too. The model is up as divagr1925/Qwenputer-npu (also on ModelScope), and its code lives on the npu branch.

It's the same RISC-V model with three more instructions and a weight memory:

• VDOT multiplies 8 int8 activations by the next 8 int4 weights and adds the sum to a register. WB sets where the weights start.
• SMAQA adds four signed byte products to a register, as in the RISC-V P extension.
• The weight memory holds 261,120 int4 weights in DeltaNet heads, loaded from the program.

Each is one instruction and one trace token, computed exactly by the compiled layers. With them, Karpathy's stories260K, a 260K-parameter Llama, runs as a C program on the CPU inside Qwen and writes its stories. Its weights are data in the CPU's memory, not compiled into the checkpoint, so any model that fits runs the same way.

A new fused executor runs the CPU at about 5,500 trace tokens a second on an L4, so a story token takes about 41 seconds. The model card has the quickstart and the checks.

Next: QwenPU. It gives the CPU a vector unit of its own, with its own memories in DeltaNet heads. The same stories260K story takes 53,000 trace tokens per story token instead of 225,000, about 16 seconds each on an L4. It's a work in progress: exact, but its checkpoint is 26.7 GB until the hidden size is trimmed.

The next logical step was to put a filesystem inside the weights. This is experimental and still in progress: the model is up as divagr1925/Qwen3.5-4B-Qwenputer-Filesystem (also on ModelScope), and its code lives on the rv-store branch for now.

It's the same RISC-V model with five more tools next to calc: put, append, get, list and delete. When Qwen calls one, the compiled layers hash the file name to a slot in a directory kept in DeltaNet state, so a lookup takes two or three trace tokens instead of a scan.

The file's text sits in other DeltaNet heads, eight tokens to a row, and the layers print it back into the <tool_response> token for token. It holds up to 512 files, about 53,000 tokens of text. Save those heads at the end of a conversation, load them at the start of the next one, and the files are still there.

The model card has the quickstart and the full results.

divagr1925/Qwen3.5-4B-Qwenputer-CHIP8 is the first Qwenputer. It compiles a CHIP-8 machine into Qwen3.5-4B instead: a CPU, a big-number maths library and its memory, written as transformer layers and appended to the stock model. Its code is at the top level of this repo (chipformer/q35/).

When the CHIP-8 model writes 48213 × 9917 =, the answer isn't a guess. The computer in its weights runs the multiplication, and the model writes 478128321 itself, digit by digit, exactly. It has never printed a wrong answer, from 3-digit sums to 20-digit products, and it still loads like any other Qwen checkpoint.

Qwen writes        "48213 × 9917 ="
computer layers    read the digits, run the BIOS, emit trace tokens
the model prints   " 478128321"
Qwen continues     as if it had written the number itself

git clone https://github.com/divagr18/qwenputer && cd qwenputer
uv sync --extra hf
hf download divagr1925/Qwen3.5-4B-Qwenputer-CHIP8 --local-dir qwenputer-4b

uv run python -m chipformer.q35.demo qwenputer-4b                   # Qwen uses its tool-call syntax
uv run python -m chipformer.q35.demo qwenputer-4b --inline --nudge  # plain inline arithmetic

import json, torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
from chipformer.q35.demo import TOOLS
from chipformer.q35.lib import bcd
from chipformer.ref.emu import Chip8
from chipformer.trace import boot_tokens

torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False  # keep matmuls exact

path = "qwenputer-4b"
tok = AutoTokenizer.from_pretrained(path)
model = AutoModelForImageTextToText.from_pretrained(path, dtype=torch.bfloat16, device_map="cuda")
off = json.load(open(f"{path}/chipformer.json"))["chip_token_offset"]
bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]

msgs = [{"role": "user", "content": "What is 48213 * 9917?"}]
ids = bios + tok.apply_chat_template(msgs, tools=TOOLS, add_generation_prompt=True,
                                     enable_thinking=False)["input_ids"]
out = model.generate(torch.tensor([ids], device="cuda"), max_new_tokens=4096, do_sample=False,
                     cache_implementation="static")
print(tok.decode(out[0, len(ids):], skip_special_tokens=True))

If you use Qwenputer in your work, please cite it. GitHub's "Cite this repository" button reads CITATION.cff; in BibTeX:

@software{agrawal2026qwenputer,
  author  = {Agrawal, Divyansh},
  title   = {{Qwenputer}: a computer compiled into the weights of a pretrained language model},
  year    = {2026},
  version = {0.1.0},
  url     = {https://github.com/divagr18/qwenputer},
  license = {MIT}
}

The code is MIT. The model weights are Apache-2.0, as a modified version of Qwen/Qwen3.5-4B: the config is widened and computer layers are added, but Qwen's own weights are unchanged.