Pooled v2.0.0

Run a 125B model from your SSD, in a browser tab, and pool devices to make it faster

Pooled 2.0 runs Qwen3.8 Flash, a 125B mixture-of-experts model that uses about 6B of its parameters per token (177B with its n-gram table). It is experimental.

Flash is far bigger than most GPUs' memory, so Pooled runs it from disk:

• Each device keeps its share of the model's experts on its SSD: in the browser's storage for a tab (after its user allows it), in the model files for pooled join.
• Each device caches the experts it used recently on its GPU. A small cache serves most lookups, because routing repeats: on recorded traces, about 17% of a layer's experts serve about 77% of lookups.
• The rest are read from disk as tokens need them.

Disk is the slow part. Every device that joins holds part of the model in its memory, so less comes from disk, and it gets faster.

Setup (October 2026)	Decode speed
A Mac Studio alone, in a Chrome tab	about 8 tok/s
The Mac Studio plus an RTX 5070 PC (pooled join)	about 10-12 tok/s, depending on the network between them

Greedy output matches llama.cpp's.

Demo video: flash-demo.mp4 (recorded October 13, 2026, 2:02, voice and subtitles): Qwen3.8 Flash in a Chrome tab on a Mac Studio, then an RTX 5070 PC joins the room and it gets 1.4x faster. It is also on the docs' Pick a model page.

Long context by default

Rooms now open at a long context without any ?ctx= in the address, and the picker has a Context row for each big model:

• Qwen3.8 Flash: up to 64K tokens (#396), checked against llama.cpp at 2K, 8K, 16K, 32K and 64K. The picker offers 8K, 32K and 64K; 64K is the default. 128K is in progress.
• Qwen3.6 35B MoE: up to 128K (was 32K by default), and Qwen3.8 27B: up to 64K (was 16K) (#421). A room goes past the old default only when it stays at full speed (no experts moved to RAM or disk, no extra devices holding layers); otherwise it stays at 32K / 16K, with longer one click away.
• A room short of memory opens at the longest length it holds, down to 8K, instead of not starting. ?ctx= and --ctx still ask for an exact length.

What else is new

Flash

• A normal row in the room's picker, with what it needs on GPUs, each device's share of the disk copy and an Allow button for disk storage (#370, #383, #385, #387, #389). One loading screen from Start to the first answer (#388-#392, #395).
• Prompts over disk layers are read in batches (#400-#402, #405). On a GB10 with a tab lending 22 GB: a 2K prompt in 51 s alone and 33 s with a room node; 8K in 134 s alone (was about 75 min) and 92 s with a room node.
• Short questions on a disk-tier tab reach the first token in about 4.5 s (#407).
• Prompt reading ~8x on the GB10 (#377), batched across a room's pipeline (#381).
• Adding a second device to a running Flash room takes about 36 s (was 98 s, #397). Devices that join are added on their own (#394).
• pooled join "<link>" needs no flags: it picks what to lend and says why (#393).
• The speed panel shows live tok/s, each device, the time per hop and the expert cache's hit rate (#353, #358).

Other models

• Qwen3 1.7B opens at 16K context (8K when the room is short).
• The tokenizer uses Qwen's own pre-tokenizer split, checked against llama.cpp.

Experts in RAM and on SSD (terminal)

• --ram N: a discrete-GPU computer keeps MoE experts in system RAM with a GPU cache. One RTX 5070 runs the 35B MoE alone at ~38-45 tok/s.
• --disk: when GPU and RAM still fall short, some experts are read from the model files on the SSD.
• Converted weights and prompt checkpoints are cached on disk, so a second load is fast.

Speed

• MoE prompt processing ~2x (GB10: 402 -> 861 tok/s at 2K); MoE plain decode in Chrome 55 -> 64 tok/s at 1K.
• DP4a prefill on by default on NVIDIA. Never on Apple.
• Windows room nodes start in ~30 s instead of 200-290 s.

Rooms and security

• 6-character room codes, and Allow / Deny for new devices (on by default).
• Channel-bound join auth: the invite key and passes are proved, never sent, and a mesh key covers every link.
• Every device checks who runs the model against the creator's roster.
• Rooms never use more memory than a device pledged. Pledges count at most 64 GB per device; for Flash only, a computer lending from the terminal counts up to 128 GB.
• A default TURN relay, used by the CLI too, for networks where direct links fail.

Terminal and API (new since 1.0)

• An OpenAI- and Anthropic-compatible API for any room: pooled serve. It serves the OpenAI Chat Completions and Responses APIs and the Anthropic Messages API on 127.0.0.1, so coding agents and apps use the room's model like a cloud API: Codex CLI, Claude Code, opencode, Continue, Open WebUI, LiteLLM, the openai and anthropic SDKs and curl. Streaming, tool calling (parallel calls, every tool_choice), JSON mode and JSON schemas all work, and every tool call is grammar-constrained to its schema, so it always parses. The Serve API button in a room gives the command; POOLED_TOKEN makes it require a key. Setup for each tool: cli/README.md.
• The pooled command line (npx @pooled/cli, Node 22+): pooled join lends a computer's GPU to a room on Windows, macOS and Linux, pooled host opens a room from the terminal, pooled chat talks to a room's model, and pooled pull / list / rm manage downloaded models.
• @pooled/openclaw 0.3.2: OpenClaw's gateway as a room device. It doesn't have Flash, --disk or the TURN relay until a new plugin release.
• A docs site at pooled.run/docs.

Full list: CHANGELOG.md.

Compatibility and upgrading

• The room protocol is unchanged (4). 1.x and 2.0 tabs can share a room, with two limits: a 2.0 creator refuses a Start pressed on a v1.0.0 tab (reload it to press Start), and a v1.0.0 tab turned away with "this tab runs an older Pooled" only needs a reload.
• To lend a computer to Flash from the terminal, use @pooled/cli 2.0.0 (npx @pooled/cli join <ROOM CODE>): cli/README.md. An older pooled join can still join a 2.0 room and run the other models; a Flash start on it fails with "couldn't load its layers (unknown model qwen3.8-flash)", and a 0.4.1 node dealt a Flash offload refuses it (the host deals again without it). pooled serve / pooled chat 0.4.x keep working.
• pooled serve can't use a v1.0.0 host (it has no API) and says so.
• Flash checkpoints saved on disk by an older build count as missing (#379): the first question on them reads the prompt again.
• Behaviour changes: the layer split defaults to "For speed"; MoE DP4a prefill is on by default on NVIDIA (?dp4a=0); --ram is on by default when a room is short; iPhone pledges are capped at 1 GB; scripts that join a gated room need the invite link or --allow-all.

Known limits

• Flash is experimental. It is slower than the other models, most of all on one computer, where much of it comes from disk. How it loads and what it needs may change between releases.
• A one-time download of about 99 GB. The first Flash load in a browser copies the tab's share of the experts (all of them for one computer alone) and, on the device holding layer 1, the 28.8 GB n-gram table into the browser's storage. The tab asks first. Each browser origin and profile keeps its own copy.
• 128K context for Flash is in progress.
• The tab's disk-tier split uses costs measured on the demo's Mac and PC, not calibrated per device.
• WiFi stalls make two-device speed vary; a wired link is steadier.
• In a room whose model host isn't the creator, pooled serve stays not-ready (503).
• Chrome or Edge on a laptop or desktop is the tested host. Firefox and Linux Chromium need WebGPU switched on.

Research Pooled builds on

The disk tier and expert cache build on LLM in a flash (running from flash storage) and on MoE offloading with a per-layer expert cache; the room builds on Petals, WebLLM and EdgeShard. Papers and how each maps to Pooled: README: Research Pooled builds on.

Everything is open source (MIT): the kernels, the protocol, the planner, the benchmarks (docs/bench-log.md) and the roadmap.