tierlm 0.6.0

gpt-oss support, a workspace that lets the model work on your project files, faster edits with prompt lookup, and one-click setup for opencode, Open WebUI, Continue and Cline.

Downloads

• tierlm-0.6.0-windows.zip (about 550 MB): CUDA + Vulkan + CPU. Get this for an NVIDIA RTX card.
• tierlm-0.6.0-windows-lite.zip (about 20 MB): Vulkan + CPU. For Intel Arc, AMD Radeon, or no GPU.

Windows may warn you the first time. tierlm isn't code-signed yet, so Windows can't show a publisher name:

• "Windows protected your PC": click More info, then Run anyway. It only asks once.
• "Part of this app has been blocked" (only on PCs with Smart App Control turned on): tierlm loads only the files it uses, which avoids the case we found. If you still see it, please open an issue with the file name from the message.

You can check your download against the .sha256 file next to it. Code signing is on the way.

To start, unzip, double-click start-tierlm.bat and pick a model. To keep your settings from an earlier version, copy tierlm-settings.json and the tierlm-usage folder into the new folder.

New: gpt-oss

OpenAI's gpt-oss models (gpt-oss-20b, gpt-oss-120b) and models built on them, such as HyperNova-60B, now run in tierlm:

• The model: attention sinks, the sliding-window layers, biases throughout, the clamped SwiGLU experts and MXFP4 weights, checked against a reference written from Hugging Face's gpt-oss code (matching to about 0.0002%), including when the model is split between VRAM and RAM.
• Long context done exactly: gpt-oss stretches its positions with YaRN. Most GGUF engines round part of that calculation; tierlm computes it exactly as the original model does.
• OpenAI's chat format ("harmony"): prompts are built exactly as OpenAI's own chat template builds them (checked byte for byte, tool definitions included). The model's reasoning shows as the usual collapsible "Thinking", and its tool calls work through the API, so coding agents work with gpt-oss too.
• Its tokenizer: OpenAI's o200k word splitting, checked against the original pattern on 3,000 test strings.

Not yet for gpt-oss: busiest-experts-in-VRAM (gpt-oss puts whole expert layers in VRAM or RAM for now), and a shorter memory for its sliding-window layers.

New: the workspace

In the chat's Settings, Workspace → Choose… picks a project folder. The model can then list, read and search its files, and suggest changes: each one shows in the chat as a diff with Apply change and Skip, and nothing is written until you click Apply (if the file changed meanwhile, the change isn't made). Each step shows as a small card, and the model carries on until it has answered.

The model can't run commands or touch anything outside the folder, and the workspace only works from the tierlm page on the computer running tierlm, not from other devices or other websites.

New: prompt lookup

When a reply repeats text from the conversation (rewriting a file with a change, quoting code), tierlm guesses the repeat and the model checks it in one pass: the same model output, faster. It works with every model; with Qwen3.5 / 3.6's draft layer, the two guessers work together. Measured: rewriting code with a rename, 2.6× faster on a model without a draft layer; about 5% faster for Qwen3.6-35B-A3B on an RTX 3070 laptop, whose draft layer already guesses most of a rewrite. Prompt lookup in the chat settings: Automatic (the default), off, or a longer or shorter guess.

New: set up an app in one click

In the chat's side menu, API → Set up an app: pick opencode, Open WebUI, Continue, Cline or a Python script, and where it runs, and tierlm writes the settings for the model you've loaded (its name, context length, reply limit, tool calling and the right address) ready to copy. For opencode there's also a PowerShell command that saves the file.

Better

• Maths in replies: formulas the model writes in LaTeX, like \(17 \times 20 = 340\) or a fraction on its own line, now show as real maths instead of backslash code. Works offline (the maths renderer, KaTeX, ships with tierlm); code and dollar amounts are left as written.
• Picking each word is faster: the step that chooses the next token from the model's predictions now looks only at the likely candidates instead of the whole vocabulary: the same choices, about 8× quicker. It matters most for models with big vocabularies; gpt-oss (201,088 tokens) spent about a third of each token's time there on a fast GPU.
• Prompt lookup checks that it's paying off: it times its own guesses against ordinary tokens while a reply is written and pauses whenever guessing is slower, so Automatic is never slower than Off. On an RTX 5090, gpt-oss-20b went from 135 to 270 tok/s on a short maths question. Find the fastest settings now also asks a short worked question, so its result matches what you see in a chat.
• Console: Expand shows the engine's log full size in place of the chat.
• Speed figure: smaller and steadier at the top right.
• Tool calling: tool definitions now keep the order your app sent them in, as the models' own templates show them (checked against Qwen3.6's template).

tierlm 0.5.2

Coding agents can now use tierlm: the API does tool calling, so opencode, Cline and Continue's agent mode can read and edit your project with a model running on your own PC.

Downloads

• tierlm-0.5.2-windows.zip (about 550 MB): CUDA + Vulkan + CPU. Get this for an NVIDIA RTX card.
• tierlm-0.5.2-windows-lite.zip (about 20 MB): Vulkan + CPU. For Intel Arc, AMD Radeon, or no GPU.

Windows may warn you the first time. tierlm isn't code-signed yet, so Windows can't show a publisher name:

• "Windows protected your PC": click More info, then Run anyway. It only asks once.
• "Part of this app has been blocked" (only on PCs with Smart App Control turned on): tierlm loads only the files it uses, which avoids the case we found. If you still see it, please open an issue with the file name from the message.

You can check your download against the .sha256 file next to it. Code signing is on the way.

To start, unzip, double-click start-tierlm.bat and pick a model. To keep your settings from an earlier version, copy tierlm-settings.json and the tierlm-usage folder into the new folder.

New: tool calling in the API

Coding agents work by asking the model to use tools: read a file, edit a file, search, run a command. The agent runs the tool and sends back the result. tierlm's OpenAI-compatible API (/v1/chat/completions) now supports this:

• The model's own format. tierlm describes the tools to the model the way it was trained (Qwen3.5 / 3.6 and Qwen3-Coder use <function=...> blocks; Qwen3, Qwen2.5 and others use JSON) and turns its replies into standard OpenAI tool_calls, streamed or not. Several calls in one reply come back together.
• Thinking kept apart. With tools, a thinking model's reasoning is sent as reasoning_content, so agents show it as thinking instead of as part of the answer.
• The cache keeps working. As the conversation grows, tierlm reads only the new part each turn. Tool calls sent back by the agent are written exactly as the model wrote them, so they still match the cache.
• tierlm never runs a tool itself: the agent does, with its own permission prompts.

Tested end to end with opencode against Qwen3.6-35B-A3B, on an RTX 3070 laptop and across the network to an RTX 5090. The setup is in Connect other apps; details in API.md.

Load the model with at least 32K context. opencode's instructions and tool list are about 11,000 tokens. The first message of a session reads all of it, so it takes longer (about two minutes on the RTX 3070 laptop, including the session-naming request below); later messages reuse it from the cache.

Docs

• opencode setup, and a note on the Open WebUI desktop app: its Connections list is for other Open WebUI servers, not for tierlm.

Known

• opencode also sends a short request to name each new session. tierlm keeps one conversation in its cache, so on the first message the long instructions are read twice. Keeping both is planned.

tierlm 0.5.1

Chats are faster, and "Find the fastest settings" now shows the speed you actually get in a chat.

Downloads

• tierlm-0.5.1-windows.zip (about 550 MB): CUDA + Vulkan + CPU. Get this for an NVIDIA RTX card.
• tierlm-0.5.1-windows-lite.zip (about 20 MB): Vulkan + CPU. For Intel Arc, AMD Radeon, or no GPU.

Windows may warn you the first time. tierlm isn't code-signed yet, so Windows can't show a publisher name:

• "Windows protected your PC": click More info, then Run anyway. It only asks once.
• "Part of this app has been blocked" (only on PCs with Smart App Control turned on): tierlm now loads only the files it uses, which avoids the case we found. If you still see it, please open an issue with the file name from the message.

You can check your download against the .sha256 file next to it. Code signing is on the way.

To start, unzip, double-click start-tierlm.bat and pick a model. To keep your settings from 0.5.0, copy tierlm-settings.json and the tierlm-usage folder into the new folder.

Faster chats

In a chat, tierlm records which experts each layer picks (for the live expert map and for learning hot experts). It used to read them back from the GPU one layer at a time, 40 separate reads per token for Qwen3.6, each one making the CPU wait for the GPU. Now they're collected on the GPU and read once per token. Same experts, much less waiting:

Qwen3.6-35B-A3B, real chat	0.5.0	0.5.1
RTX 5090 + 64 GB RAM, Q8_0 (37 GB)	79 tok/s	93.7 tok/s
RTX 3070 laptop + 32 GB RAM, UD-Q3_K_XL (17 GB), speculative decoding	20–22 tok/s	27 tok/s

Real-world speed tests

Find the fastest settings and Benchmark now run exactly what a chat runs: the same reply code, your sampling settings (temperature, top-k, top-p, min-p, repeat penalty), answering two real questions, with a fixed seed so every setting is compared fairly. The speed they show is the speed you'll see in chats. Before, they generated a fixed text the quickest way, which overstated chat speed by a quarter to a third. On the RTX 5090, the Benchmark now says 95.9 tok/s and a real chat ran at 93.7.

Also

• Formatted replies in the chat: headings, lists, tables, code blocks with a Copy button, and a model's thinking folded away.
• Windows: tierlm loads only the CPU engine your processor uses, instead of trying all eight at start-up.
• The tuning summary says "busiest experts in VRAM" when that's what won.
• New results: RTX 5080 + Ryzen 7 9800X3D, Qwen3.6-35B-A3B Q6_K (27 GB): 65.2 tok/s with experts in RAM.

Checked

• The new expert recording gives exactly the same picks as before on every mixture-of-experts test model, for prompts and single tokens.
• Every model type still matches the reference, including split between stand-in GPUs and RAM.
• The tuner and Benchmark were run end to end through the server, and on the RTX 3070 laptop and RTX 5090.

tierlm 0.5.0

tierlm 0.5.0 generates text about 50% faster than 0.4.0 on NVIDIA cards (21.7 → 33.3 tok/s on an RTX 3070 laptop), can keep a model’s busiest experts in VRAM, and documents every change it makes to ggml.

Downloads

• tierlm-0.5.0-windows.zip (about 550 MB): CUDA + Vulkan + CPU. Get this for an NVIDIA RTX card.
• tierlm-0.5.0-windows-lite.zip (about 20 MB): Vulkan + CPU. For Intel Arc, AMD Radeon, or no GPU.

To start, unzip, double-click start-tierlm.bat and pick a model. Your normal graphics driver is all it needs. New to tierlm? Read the guide (PDF).

New

• CUDA graphs: much faster on NVIDIA. Each token's GPU work is now recorded once and replayed with a single launch, instead of thousands of small launches. On a laptop RTX 3070, Qwen3.6-35B-A3B with experts in RAM went from 21.7 to 33.3 tok/s. Answers are exactly the same; only how the work is launched changed.

• Hot experts. tierlm learns which experts each mixture-of-experts model uses as you chat, and can keep those individual experts in VRAM instead of whole expert layers.

  • Whether that's faster depends on the PC. On the RTX 3070 laptop the plain split still won, so it's off by default.
  • Find the fastest settings measures both and turns hot experts on only where they win.
  • Model info shows how many experts are in VRAM and what share of the expert work they did.

• Open about ggml. tierlm changes ggml in two small places (50 lines):

  • the CPU skips experts that are computed elsewhere
  • an optional timing switch

  GGML_CHANGES.md explains every changed line, includes a patch file for other projects, and covers what it took to make CUDA graphs work. The changes are published as a fork: EHTi-Chad/ggml, branch tierlm.

• Timing switch. set TIERLM_PROFILE=1 before starting tierlm, and the black window shows where each token's time goes: preparing the step, and for each device the copying and the work.

Fixed

• An illegal-memory-access crash on NVIDIA cards with hot experts.

Measured

RTX 3070 laptop (8 GB), Qwen3.6-35B-A3B MTP UD-Q3_K_XL (17 GB MoE), 16K context, measured by Find the fastest settings. It uses short replies at temperature 0, so a normal chat comes out somewhat lower.

Setting	Without CUDA graphs	0.5.0
Whole layers	13.2 tok/s	13.0 tok/s
Experts in RAM	21.7 tok/s	33.3 tok/s
Busiest experts in VRAM	21.0 tok/s	29.7 tok/s
Experts in RAM + speculative decoding, 1 guess	27.5 tok/s	35.3 tok/s
Experts in RAM + speculative decoding, 2 guesses	28.3 tok/s	35.1 tok/s

RTX 5080 and RTX 5090 results will be added.

Checked

• Every model type matches the reference to about 1e-6, including 300-token prompts and speculative decoding. That holds whether the model is split between one or two stand-in GPUs and RAM by layers, by experts, or with hot experts, at many VRAM sizes.
• CUDA graphs and the hot-expert split were run on the RTX 3070 laptop.

tierlm 0.4.0

tierlm now picks the fastest settings for your PC by itself, uses more than one graphics card, and has a new chat layout.

Downloads

• tierlm-0.4.0-windows.zip (about 550 MB): CUDA + Vulkan + CPU. Get this for an NVIDIA RTX card.
• tierlm-0.4.0-windows-lite.zip (about 20 MB): Vulkan + CPU. For Intel Arc, AMD Radeon, or no GPU.

To start, unzip, double-click start-tierlm.bat and pick a model. Your normal graphics driver is all it needs. New to tierlm? Read the guide (PDF).

New

• Smarter split for MoE models. When a mixture-of-experts model doesn't fit in VRAM, attention, the router and the context now stay on the GPU, and only the experts go to RAM. Dense models still move whole layers. This is the new Automatic setting.

• Speculative decoding defaults by model type. The default is 2 guesses per check for MoE models (each guess pulls in its own experts from RAM) and 3 for dense models. There's also a new 1-guess option.

• Find the fastest settings. One button on the setup screen loads the model and measures what affects speed on your PC: the split mode, CPU threads, and speculative decoding at 1–4 guesses. It shows each result as it goes, then keeps the fastest and saves it for that model.

• Several GPUs. Two or more cards of the same kind act as VRAM tiers: the model fills the first card, then the next, then RAM. The setup screen shows a VRAM bar for each card.

• New chat layout. A side menu that folds away, with:

  • Settings: sampling, speculative decoding on or off without reloading, and a system prompt
  • API
  • Console: shows what the engine is doing
  • Model info
  • Chats: saved chats and system-prompt presets
  • Benchmark: one click

  The right side is now a thinner live view, and there's an Auto / Light / Dark theme switch.

• The expert map explains itself. It has:

  • a ? with a plain-words explanation
  • a sorted view (each layer's busiest experts first)
  • a strip showing which layers' experts are in VRAM or RAM
  • details when you hover
  • a full-size view

  It now also works with speculative decoding on.

• Smaller fixes:

  • a loading progress bar
  • a steadier speed chart with speculative decoding
  • a request that arrives just as a reply finishes now waits a moment instead of failing

Measured

RTX 3070 laptop (8 GB), Qwen3.6-35B-A3B MTP UD-Q3_K_XL (17 GB MoE), 16K context, 8-bit context memory:

Split mode	Speculative decoding	Generation
Whole layers	off	9.9 tok/s
Whole layers	3 guesses	11.3 tok/s
Experts in RAM	off	17.0 tok/s
Experts in RAM	3 guesses	18.7 tok/s
Experts in RAM (now automatic)	2 guesses (now the default)	22.3 tok/s

RTX 5090 results will be added.

Checked

• Every model type, split over two stand-in GPUs and RAM, matches the reference, including with speculative decoding.
• The expert map gets exactly the same picks with speculative decoding on and off.
• Auto-tune, the benchmark and the new layout work in a browser, in both themes and at phone width.

tierlm 0.3.0

Faster replies for models that don't fit on the GPU, and one download for every PC.

Downloads

• tierlm-0.3.0-windows.zip (about 550 MB): CUDA + Vulkan + CPU. Get this for an NVIDIA RTX card.
• tierlm-0.3.0-windows-lite.zip (about 55 MB): Vulkan + CPU. For Intel Arc, AMD Radeon, or no GPU.

Unzip, double-click start-tierlm.bat, pick a model. Only your normal graphics driver is needed. New to tierlm? Read the guide (PDF).

New

• Speculative decoding. Qwen3.5 / 3.6 / 3.8 and Qwen3-Next files often include a small built-in draft layer (MTP). Turn on Speculative decoding in the settings: the draft layer guesses the next few words and the model checks them all in one pass. The answers are the same; it's faster when the model is partly in RAM. Unsloth's separate mtp-*.gguf files are found and used automatically.
• Every CPU. The zip carries a CPU engine for every x86 generation (x86-64, SSE4.2, AVX, AVX2, AVX-VNNI, AVX-512). tierlm picks the best one your processor supports when it starts. The hardware card shows which one is in use.
• Faster CPU work. Q4_0, Q4_K and IQ4_NL weights in RAM are rearranged for the CPU on load. Prompts read about 3.5× faster on the CPU, with the same results.
• Context memory precision. 16-bit, 8-bit (half the memory, practically the same quality), 4-bit or 32-bit. A smaller context leaves VRAM for more of the model.
• Qwen3-Next and Qwen3-Coder-Next (80B-A3B), in both GGUF layouts.
• Smaller download. The full zip is about 550 MB instead of 1.1 GB.

Measured

RTX 3070 laptop (8 GB), Qwen3.6-27B MTP Q4_K_M (dense, 17 GB), 23 of 64 layers in VRAM, 16K context, 8-bit context memory:

Speculative decoding	Generation
Off	3.0 tok/s
2 guesses per check	3.9 tok/s
3 guesses per check (default)	5.3 tok/s
4 guesses per check	4.1 tok/s

RTX 5090 results will be added.

Checked

At temperature 0, speculative decoding gives exactly the same replies as normal generation on every model type. This holds with deliberately wrong guesses, over several turns, and when the model is split between GPU and RAM (140 of 140 checks). Undoing rejected guesses is exact.

tierlm 0.2.0

Run models bigger than your GPU. tierlm fills GPU memory first and puts the rest of the model in system RAM, automatically.

• 14.7 GB Qwen3-30B-A3B on an 8 GB laptop RTX 3070: about 27 tokens/second
• 46 GB Qwen3.6-35B-A3B (Q8_0) on an RTX 5090 (32 GB) with 64 GB RAM: about 36 tokens/second

Which download?

• tierlm-0.2.0-windows.zip (about 1.1 GB): everything, including NVIDIA's CUDA engine, the fastest option on GeForce RTX 20/30/40/50 cards. Get this one if you have an NVIDIA RTX card.
• tierlm-0.2.0-windows-lite.zip (about 55 MB): the same app without CUDA, for Intel Arc, AMD Radeon, or no graphics card. Also runs on NVIDIA through Vulkan, just slower.

Both pick the fastest engine they have by themselves. Nothing else to install: your normal graphics driver is enough. Extract the zip, double-click start-tierlm.bat, and pick a model. tierlm can use your LM Studio models folder directly.

You need: Windows 10/11 64-bit, an AVX2 CPU, and at least 32 GB of RAM to run models bigger than your GPU (64 GB or more recommended).

New in 0.2.0

• Models bigger than VRAM are split between VRAM and RAM automatically, with memory bars showing exactly where the model, its context and other programs sit
• Qwen3.5 / 3.6 / 3.8, dense and MoE (hybrid Gated DeltaNet), plus Llama 2/3, Mistral, Mixtral, Qwen2/2.5, Qwen3, Qwen3 MoE and OLMoE
• Multi-part GGUFs (-00001-of-0000N.gguf) load as one model
• CUDA engine for NVIDIA RTX cards
• OpenAI-compatible API (/v1/chat/completions, /v1/models): use tierlm from Open WebUI, Continue, LibreChat or the OpenAI SDK, with an "Allow other devices" switch and an optional API key
• Setup screen: detected hardware, models folder picker, context up to each model's maximum, per-model settings
• No limit on reply length by default (thinking models need room)

Not yet

Tool calling (for Cursor / Cline agent modes), the SSD tier, Qwen3-Next / Qwen3-Coder-Next, Gemma, Phi, DeepSeek, gpt-oss.

Full guide: docs/tierlm-0.2.0-guide.pdf in the repository. Problems: docs/FAQ.md.