v.0.5.6-alpha

Install command, run in PowerShell. From an admin PowerShell it installs for the whole machine instead.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

On Linux, with sudo sh at the end to install for the whole machine:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Changed

• An install is your own unless you ask for the machine. Both installers now set llmash up for the person running them, in ~/.local on Linux and %LOCALAPPDATA%\llmash on Windows, with no sudo. Run as root (sudo sh) or from an admin PowerShell, they install for the machine, into /usr/local/lib/llmash or %ProgramData%\llmash. An existing install is upgraded where it is.
• A machine install can be run by every user and changed only by administrators. On Linux its service runs as an unprivileged llmash account instead of root, cannot change the program, and keeps its models in /var/lib/llmash/models unless a models folder is already set. On Windows, other users no longer get the right to add files beside the program, which %ProgramData% gave them.
• The key for llmash link, in link.json, is readable by its owner alone: mode 0600 on Linux, and only that user, SYSTEM and Administrators on Windows. It was readable by every account on the machine. An install made by an older version is narrowed by llmash update, and by llmash itself the next time it starts. The CLI also no longer mints that key from an unseeded std::rand on Linux.
• The Linux installer keeps an existing local.json instead of rewriting it on every update, so settings made there survive llmash update.
• Claude Code's auto-mode check gets its verdict. It asks for no thinking and wants the verdict first; a local model thought by default, spent the check's whole budget reasoning and never gave one, so every tool call waited on a check that failed. A Messages request that does not ask for thinking is now answered without it, as on Anthropic's API.

Runtime

Unchanged, build 770. llmash update keeps the one you have.

Checked before release:

• The client program and its tests on Windows and Linux.
• Both installers, as root or admin and as a normal user: a machine install's service account adds its own files but cannot replace the program, other users run it but cannot write into it or read the key, and a user install is readable by that user alone.

0.5.6+b512c07

Built from b512c07. llmash update installs this; the runtime comes from the latest release.

v.0.5.5-alpha

Install command, run in a user level powershell.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

On Linux:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Changed

• Qwen3.8-Flash-Next runs on llmash's runtime and drafts with an MTP head when one sits beside it as <model>.mtp.gguf. Its decode no longer falls off as a chat grows: on a dev build with this fix it decoded 122, 118, 107 and 90 tok/s with MTP at 2k, 64k, 135k and 256k tokens of chat, where it had gone from 119 to 38.
• A model split into shards finds its projector. The registry looked for <first shard>.mmproj.gguf, so Qwen3.8-Flash-Next was listed without vision and never reloaded with its projector on an image turn.
• llmash launch claude runs Claude Code against llmash. It ran Claude-on-Local.ps1, which no installer shipped, and stopped at "launcher missing". llmash now forwards Anthropic's Messages API (/v1/messages) to the runtime, folds the system messages Claude Code puts inside the conversation (Qwen's template refused them), and tells Claude Code the context the model runs at. With no model picked it takes the first one that runs at the 64k Claude Code needs.
• The OpenAI chat and completion routes also answer without /v1, as /models already did. A client given the server's URL without /v1 listed the models and then got a 404 for every chat.
• /api/chat sends the reasoning of earlier assistant turns back to the server. It was dropped, so a template that keeps reasoning rendered a history the model never wrote, and the server's cache stopped matching at that turn.
• A q8_0 KV cache decodes at about the speed of f16: flash attention reads it in place instead of converting the whole cache every step.
• The picker offers the NVFP4 builds of a model on sm_120 cards and MXFP4 on the others; it was finding neither.
• llmash ablate loads the model the way the server does, so a model bigger than RAM no longer fills it.

Runtime

The runtime changed, so llmash update replaces the one you have (its stamp carries the new commit).

• Qwen4 QSA attention (Qwen3.8-Flash-Next) attends only to the cells its indexer selects, a q8_0 cache included, and a draft round's rollback cuts the k-pool layout at the edit instead of rebuilding it from the whole cache. At 256k tokens that rebuild was most of a round.
• The upstream Qwen4 and k-pool work since the last runtime: the MTP head, cheaper mask construction, half the indexer score memory, no graph reallocation mid-run, and a fix for a scatter race on shared sequences.

Checked before release:

• The client program and its tests on Windows and Linux, and the Linux runtime from a bare Ubuntu 22.04: every binary loads from another folder with no library path set.

Fixed since release (same version; the programs and both runtimes were replaced in place, and llmash update fetches them on a machine that already has 0.5.5)

• An image inside a tool result loads the model's projector. That is where Claude Code puts a screenshot it took or a picture it opened, and llmash only looked for images at the top level of a message, so the model ran without vision and the runtime answered "image input is not supported".
• An image for a model with no projector is refused at once with a 400 that says where the projector goes. The turn reached a runtime started without one, which answered 500, and Claude Code retried it ten times.
• A request whose client hung up is cut at the runtime. Claude Code times out a long answer and retries, and the first request stayed alive in llmash: the runtime went on generating it, up to the token limit, for nobody, and the retry waited behind it on the one slot, minutes at a time. llmash now drops the runtime's request the moment the client's connection closes, which cancels it.
• A reply that runs to its token limit is saved beside the model's log with the request that made it (logs/<model>-runaway-<time>.json), the way a crash is, so a tool call that never ends can be looked at.
• A streaming reply opens at once and is pinged every five seconds while the model loads and reads the prompt. Claude Code gave up on a stream that carried nothing for a minute (a reload plus an 85 second read of a long conversation), retried without streaming, and in that mode dropped every reply over a few minutes, so a long local answer never arrived. llmash launch claude also gives Claude Code request timeouts sized for a local model, unless you set your own. A reply the client drops is recorded beside the model's log like one that runs to its limit.
• The runtime is build 770:
  • No drafts while the prompt holds a picture. The drafter skips a picture's rows, and the draft rounds after one wrecked the model itself: Qwen3.8-Flash-Next at 97k tokens with two screenshots in the conversation decayed into noise within a few hundred tokens of every reply, ran each one to Claude Code's 32k limit, and kept 5% of its drafts. The same request with the drafts off wrote 2,500 sound tokens at 71 tok/s; without the pictures the drafts were fine. Text-only prompts draft as before.
  • A drafter that keeps nothing is rested. A drafter out of step with the target (an image in the prompt leaves it positions it never saw) still cost a draft every round: one Qwen3.8-Flash-Next request kept 459 of 94,614 drafted tokens and decoded at a third of its plain speed for 32k tokens. After sixteen rounds that keep fewer than four drafted tokens the drafter rests for 32 rounds, is probed for sixteen, and rests twice as long each time the probe finds it still out of step. With drafts forced to zero acceptance, Qwen3.5-4B went from 150 to 255 tok/s; a drafter that works is not touched.
  • A request with tools keeps its sampling on the GPU up to the pick, which the CPU makes from the top candidates. Its tool-call grammar turned GPU sampling off, so every drafted token was sampled on the CPU over the whole vocabulary, and the grammar was copied every draft round. With 16 tool definitions, Qwen3.8-35B-A3B with MTP went from about 205 to about 380 tok/s, and from about 195 to about 340 inside a tool call, on an RTX PRO 6000; without tools it decodes about 500.
  • A draft round rolls back the tokens it did not accept, and that rollback walked every cell of the KV cache, used or not, so a model given a 1M context paid for the whole million on every round however short the chat was. It now stops at the last used cell.

v.0.5.4-alpha

Install command, run in a user level powershell.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

On Linux:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Changed

• A drafter pulldraft fetches for a model in an Ollama store your user can't write to (on Linux the store usually belongs to the ollama account) is used again. pulldraft saved it to the loose-GGUF folder, but the server only looked beside the model's blob, so the model kept drafting by n-gram lookup. The server, pulldraft and run now look in both places, for an MTP head, DSpark, DFlash, EAGLE-3 or a draft model alike. Reported and pinned down by @ibilbrey.
• llmash serve reads LLMASH_MODELS, as the help, the installers and llmash models already said it did.

Checked before release:

• The client program and its tests on Windows and Linux.

The runtime is still build 146, so an update from 0.5.3 does not download it again.

v.0.5.3-alpha

Install command, run in a user level powershell.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

On Linux:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Abliteration

llmash abliterate <model> uncensors a model in place. It does not touch the weights: it writes a small control vector beside them that the model picks up on its own the next time it runs. ablate is the same command, and pull --ablate does it right after a download.

• It reads the model's own refusal direction, layer by layer, from the gap between how it answers harmful and harmless prompts, then searches sixteen projection vectors for the gentlest one that still stops the refusals. A refusal is decided in the opening words, so a first-token gate counts them cheaply, and the search is held to a forward-KL budget measured against the base model on harmless prompts. A model that will not come apart cleanly writes nothing and says so, rather than shipping a damaged vector.
• The result is <model>.ablation.gguf, a control vector with mode = "project": zero bytes changed in the model, and the removal happens at inference as cur - w (r·cur) r per layer. Launch applies it on its own and the registry hides it like any other sidecar; LLMASH_NO_ABLATE=1 runs the model untouched, and deleting the file puts the refusals back.
• The harmful/harmless split it reads is pulled once and cached under ablate-data. The run shows friendly progress the whole way and finishes well under two minutes on a single modern card; a mixture-of-experts model is cheaper still, since it only moves its active parameters.

Runtime

The runtime changed, so llmash update replaces the one you have (its stamp carries the new commit).

• It carries llama-ablate and the projection-mode control vector. The projection runs per layer on that layer's own device, so it is correct on a model split across several GPUs, by the same mechanism the ordinary additive control vector already relies on.
• A fused matvec-add no longer trips the CUDA mmvq assert at high concurrency: it counts the columns from the added tensor's rows and reads a strided source as columns only when that source is contiguous. This is a general correctness fix and is not specific to abliteration.

Checked before release:

• Refusals on 100 harmful prompts, judged by a model reading each whole answer, base → abliterated: Qwen3.8-27B 94 → 2, Qwen3.5-4B 98 → 22, Gemma 4 E4B 98 → 3, Gemma 4 26B-A4B 99 → 2. Gemma needs the per-layer direction; a single global direction leaves one massive-activation channel that keeps it refusing.
• StrongREJECT, the human-validated fine-tuned grader rather than a teacher model judging itself, on qwen3.5-4b: 0.05 base → 0.53 abliterated.
• First-token KL against the base on harmless prompts stays low where the method holds (Qwen3.8-27B 0.11, Gemma 4 E4B 0.06); the search rejects and writes nothing when it would not.
• llmash abliterate qwen3.5-4b --yes end to end: the split pulled and cached, the vector written beside the model, picked up on the next run with the refusals gone, and the model running untouched under LLMASH_NO_ABLATE=1.
• The Windows and Linux runtimes built from the same commit; the client program and its tests on both.

The runtime carries abliteration and the mmvq fix; everything else is unchanged from 0.5.2.

Fixed since release (same version; the program and installer were replaced in place)

• llmash ablate says why it failed instead of "ablation did not complete": a runtime that cannot load the model's architecture, too little free GPU memory, or the tool's own last lines.
• On a machine that runs upstream llama.cpp (an older NVIDIA driver, a non-NVIDIA GPU, or a llama-server found on PATH) llmash ablate says that abliteration needs llmash's runtime and what would get it, instead of "run llmash update".
• The installer no longer lets a llama-server that is only on PATH stand in for llmash's runtime on a card that runs it.
• The server applies an ablation vector only on a runtime that can project it; another build would add it to the stream and damage every answer.
• A model kept in an Ollama store gets its vector in llmash's own folder, not inside Ollama's blobs; the prompt-set download retries when the hub rate-limits it.
• A second llmash serve refuses a port the running one holds instead of sharing it.
• llmash help lists ablate; help run and help pull list the flags they were missing.

v.0.5.2-alpha

Install command, run in a user level powershell.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

On Linux:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Classifiers

A classifier answers questions about a text from one forward pass and generates nothing. The hub tags them text-classification: decision models such as Laya, Jev-Style and decider, and rerankers with a scoring head. llmash runs them as their own kind of model.

• pull records the tag beside the model and takes whatever the repository, or the model it was converted from, publishes for reading the answers: Laya's decision head comes down as a sidecar next to the encoder (<model>.classifier.gguf), a Jev-Style v3 verdict readout or a decider answer-letter layout is written to <model>.classifier.json with its calibration, and a scoring head inside the GGUF needs nothing more. A model pulled before this release gets the same the first time it is shown or run: where it came from is found from its header or by the file on the hub, once, and kept in sources.json beside it.
• The runtime is started with what the kind needs (an encoder serves its states, a scorer its rank), and no drafter is offered, so the wait after pulling one is gone.
• run on a classifier takes the questions first, one per line, then judges every line of text against them and prints each answer with its probabilities: Which team should handle this? [billing, technical, sales], score: How urgent is it? [not at all, slightly, very], Is the customer angry?. A choice lists its options in brackets and name: description describes one; score: lists ordered levels, lowest first; a question with no options is answered yes or no. /questions, /clear, /json, /?.
• The same lines work in any chat window pointed at llmash, after the text or in the system prompt: /api/chat, /api/generate and /v1/chat/completions answer a classifier with the answers as text plus an answers object, and say how to ask when no question was given.
• POST /api/classify (also /v1/systemone, the shape decision models share) takes {"model", "state", "questions": {id: {"type": choice|score|noul, "instructions", "criteria"}}} and answers {"answers": {id: {"type", "choice"|"score"|"noul", "probabilities", "confidence"}}, "usage"}. state may be an object. A chat model asked here is read at the answer letter of the common decision prompt.
• show lists such a model with the capability classification; <model>.classifier.json beside any GGUF says how it is read, and can be written by hand.

Pulling

• Every GGUF repository on the hub pulls. A build whose file name carries -imatrix- is a build, not the importance file beside it. A build named without a width (-APEX-Balanced, gemma-2b.gguf) is listed by what sets it apart, taken by that name, and the middle one by size stands in for the medium build. Ternary widths (PQ2_0, PTQ1_0) are read. A full build that carries its own MTP head is a build.
• A gated repository (Gemma and Llama as their makers publish them) is read with the token hf auth login stores, or HF_TOKEN. The token goes to the hub only, never to the CDN a download redirects to, and a refused token is dropped so public repositories still read.
• The drafter search names a model correctly: a converter that wrote the whole name as the basename gives Qwen3.5-4B, not Qwen3.5-4B-4B; an expert-count label (64x2.6B) is not part of a name; Meta-Llama is looked for as Llama; and a file whose header cannot name its model (Agents-A1's says A1) is named by the repository it was pulled from, or the model the hub's card says that repository quantized. So a model's own heads are found under its own name.
• A drafter named for its kind and then its own words (DFlash-UltraChat, EAGLE3-converted, SpecForge-EAGLE3) is a drafter for the model, not a project of its own, and Instruct inside a candidate's name no longer hides the model's. A head published on its own in a repository named -MTP- is an MTP head, not a plain draft the runtime could not have loaded.
• After a pull, the drafter search gives up after twenty seconds and says that pulldraft looks longer, instead of holding the console; a prompt with no console to answer from takes no for an answer.

Context

• ctx --kv and the server's kv_type are checked before anything launches: a made-up cache type (q16_0) is refused with the list llama.cpp takes, instead of being written to local.json and failing every load after. A runtime that refuses an argument is quoted in its own words rather than its usage tail. The context report says what the bytes in RAM are: the model's token table, which llama.cpp keeps in system RAM as the input layer whatever the card holds.

Console

• Ctrl and Alt with the arrow keys jump by words again. The sequence they send was read a byte at a time, so it moved nothing and typed the rest.
• The line no longer flickers as you type: a redraw is one write, with the cursor hidden meanwhile, instead of pieces that showed the cursor jumping to the start of the line and back on every key.
• LLMASH_KEYLOG=<file> writes every byte the editor reads, for a key that does nothing in some terminal.

Checked before release:

• 170 GGUF repositories through the picker, nothing downloaded: the top ones by downloads, the trending ones, and the awkward namings (APEX builds, a lone gemma-2b.gguf, -imatrix- in the file name, full builds carrying their own MTP head, ternary spellings). Every one lists its builds; seven did not before.
• The drafter search from the headers of fifteen models (Qwen 3.5, 3.6 and 3.8, GLM-4.7-Flash, Gemma 4 E4B and 26B-A4B, Agents-A1, Llama 3.1 and 3.2, Qwen3-8B, DeepSeek-R1-Qwen3-8B, Mistral Small 3.2, gpt-oss-20b): each finds what the hub publishes for it, checked against the weights, and says so when there is nothing.
• Laya (laya:q8_0, the head fetched from the hub) against its reference cases: every probability within 0.01, token counts equal. Jev-Style-0.8B-Decision-v3 and decider-2b (its own layout, temperatures and level-by-level scores) give the expected answers on sentiment, ticket routing and news sections in 0.1 to 2 seconds once loaded; a reranker with a cls head scores through the runtime's rerank route.
• A Laya file pulled by an earlier version, with nothing beside it, run from the console: its repository found by the file's name and size, the head taken, the questions answered.
• The chat, generate, OpenAI and System One routes on a classifier, streaming and not; the editor fed ctrl+arrow, delete, home, end and paste sequences; a bad hub token against a public repository; all client tests on Windows and Linux.

The runtime is unchanged from 0.5.1.

v.0.5.1-alpha

Install command, run in a user level powershell.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

On Linux:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Drafters

The drafter lookup is remade. A model is known by its own header (Qwen3.5 + 4B), not by the pretraining base it names, and never with its size doubled; pulldraft says what it is looking for in one line and lists what fits, and --verbose says why each candidate was passed over. What it finds:

• An EAGLE-3, DFlash, DSpark or plain draft model published for the model.
• Gemma 4's assistant drafter, which Google publishes for each of its models, as the model's MTP sidecar. Gemma 4 E4B generates 1.2 times as fast on conversation and nearly twice as fast on code and reasoning with it.
• The MTP head out of a full build that carries one: the -MTP GGUFs of a model count, and only the head is fetched out of them, by byte range, as a sidecar. A Qwen3.5-4B NVFP4 build without its head gets one back in half a minute, 0.6 GB out of a 2.8 GB file, and generates 605 to 880 tokens a second with it against 390 without. As with a model's own head, the greedy text can differ from the undrafted run where two tokens nearly tie: the verify batch and the single-token decode take different kernels.
• For a fine-tune with nothing under its own name, the drafters of the model it was tuned from (a MiMo distill of Qwen3.5-9B gets Qwen3.5-9B's), said so: they share its vocabulary and draft somewhat less well.

Also:

• pulldraft MODEL hf.co/ORG/REPO takes a drafter you name, :FILE or @QUANT for one of its builds, checked against the model before it downloads. When several fit, you choose.
• One MTP sidecar serves every build of its model: a drafter installed beside the Q4_K_M is used by the Q8_0 and RCO-3 of the same model too.
• run asks about a drafter once per model instead of every time, as a menu with the choices spelled out: look now, don't ask about this model again, or don't ask about any model again. Escape leaves it for next time.

Pulling

• On an all-Blackwell machine the medium build is NVFP4 even when the repository pulled from has none: an NVFP4 GGUF of the same exact model from another repository is offered and pulled from there. RTX 50 cards count.
• A pull that asked for no width in particular no longer reports "no Q4_K_M, taking NVFP4" when it takes the one build a repository has.

Runtime (build 144, was 143)

• NVFP4 under speculative decoding. A mixture-of-experts model's per-expert scales ride the expert kernel of a verify batch, so gate and up run as one launch again and the router weights ride the down projection; an output projection's scale and the residual after it ride its matvec; and a float gate and up pair of 2 to 8 columns (the BF16 shared experts NVFP4 files keep) runs as one launch with its GLU. Qwen3.6-35B-A3B NVFP4 with MTP generates 7-12% faster, with the same text.
• Measured here: on a dense model NVFP4 is the better 4-bit format on Blackwell. Qwen3.8-27B NVFP4 generates 2-14% faster than the UD-Q4_K_XL build of the same size and reads a 4k-token prompt 31% faster (4,732 against 3,616 tokens a second).

README

• Rewritten around two charts: every build of Gemma 4 E4B, Qwen3.8-27B and Qwen3.6-35B-A3B on llmash, Ollama and vLLM, speed against size on the same card and prompts; and what a 3-bit build costs in accuracy on Qwen3.8-27B and Gemma 4 E4B, with the tokens each build generated. Version-specific wording is gone; a feature says the release it arrived in.

Checked before release:

• The same greedy text as build 143 on Qwen3.6-35B-A3B (NVFP4 and UD-Q4_K_XL, MTP), Qwen3.8-27B NVFP4 (MTP), gemma-4 26B-A4B RCO-3 (MTP sidecar), gemma-4 E4B, GLM-4.7-Flash, Qwen3-1.7B, and the Qwen3.8-27B and Qwen3.6-35B-A3B RCO-3 builds, on Linux, and on Qwen3.6-35B-A3B NVFP4 on Windows.
• The drafter search against the live hub: Qwen3.5-4B NVFP4 gets its MTP head extracted and installed and drafts with it; a MiMo distill of Qwen3.5-9B gets the head out of its own -MTP build; Gemma 4 E4B gets its assistant drafter.
• All client tests on Windows and Linux, and the Windows install run from its own folders.

v.0.5.0-alpha

Install command, run in a user level powershell.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

Over SSH or from a script, save it and pass the options up front (with none, the installer only lists them):

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 -OutFile install.ps1
powershell -ExecutionPolicy RemoteSigned -File .\install.ps1 -NoOllama

On Linux:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Runtime (build 143, was 79)

Against 0.4.35's own runtime, on one RTX PRO 6000 Blackwell under Linux, the same files give the same text faster: Qwen3.6-35B-A3B UD-Q4_K_XL 526 -> 631 tok/s (+20%) and Qwen3.8-27B GSQ-RCO IQ3_S 192 -> 197 tok/s (+3%). The README has the numbers for every format.

• An MTP head drafts through a 131,072-row copy of the LM head requantized to Q4_K at load, where 0.4.35 used 65,536 rows at the head's own type: twice the vocabulary for about the bytes of 65,536 rows at Q8_0, so more drafts land. Verification still reads the full head. LLAMA_DRAFT_HEAD_ROWS and LLAMA_DRAFT_HEAD_TYPE=same bring the old head back.
• Delta-net layers (Qwen3.5, 3.6, 3.8, the Next models) launch fewer kernels per token: the input projection runs the causal convolution, its SiLU and the state snapshots in its own store; the two gating projections run as one launch; the recurrence normalizes q and k as it loads them; and the gated norm is quantized straight into the output projection's input. Qwen3.6-35B-A3B's verify round launches at least a quarter fewer kernels.
• A mixture-of-experts layer applies the router weights inside the expert matvec, and a residual add after an output projection rides that matvec, on any model with the pattern.
• Fixed: an NVFP4 mixture-of-experts model with an MTP head (Qwen3.6-35B-A3B NVFP4) aborted on its first speculative verify batch; 0.4.35 did the same.
• Fixed a race in the fused expert sum: it could read router weights the allocator had already given to its own output, which changed a token now and then between otherwise identical runs.

Every fusion is matched on the graph rather than by model name, and each has a bit in GGML_CUDA_FUSE_OFF.

Pulling

• On a machine where every NVIDIA card is Blackwell (RTX 50, RTX PRO 6000, B200), the medium build pull offers is an NVFP4 build when the repository has one, labelled for what it is: FP8 quality at a 4-bit size, on the FP4 tensor cores. Elsewhere medium stays Q4_K_M. LLMASH_FP4=1 or 0 overrides the detection.
• A repository with no GGUF of its own is answered only by a GGUF conversion of that exact model. It could take an abliterated or otherwise derived build before, when the exact conversions failed to list; now the pull names the near matches as different models and takes none of them.
• The custom build's row no longer claims Q8 quality. It says what it is: 3 bits, one type per tensor.
• llmash list right after a pull shows the finished file rather than the table from while it was downloading.

Everything else

• rm takes any part of a model's name, and remove is the same command: llmash rm qwen removes the one model with qwen in its name, and lists the candidates without removing any when there are several.
• run on a model with no drafter asks whether to look for one on Hugging Face (yes, no, or don't ask again) instead of printing the command to do it.
• llmash update installs only published releases; the edge build of every push is gone, and --stable is accepted but no longer needed.
• The README is rewritten, with charts for speed by weight format, this release against the last, and what a 3-bit build costs in accuracy.

Checked before release:

• The same greedy text as 0.4.35's runtime on Qwen3.6-35B-A3B UD-Q4_K_XL and Qwen3.8-27B GSQ-RCO with MTP on Linux, and on Qwen3.8-27B GSQ-RCO on Windows.
• Each fusion checked the same way against the build before it on Qwen3.8-27B NVFP4 and RCO-3, Qwen3.6-35B-A3B RCO-3, Qwen3-1.7B, gemma-4 26B-A4B and E4B, and GLM-4.7-Flash, with MTP and n-gram drafting.
• Qwen3.6-35B-A3B NVFP4 with its MTP head runs on Linux and Windows.
• The same text on 1, 2 and 3 GPUs under layer split (emulated on one), and eight identical requests in a row giving identical drafts.
• The Linux install on Ubuntu 22.04, Fedora 40 and Arch, serving a model with MTP drafting, and on Ubuntu as an update and a runtime switch.
• This release as published: a fresh install, an update from 0.4.35 and a pull on Ubuntu, and llmash update from 0.4.35 on Windows.
• The runtime source built for Turing (sm_75, PTX only, run through the driver's JIT): all 20,363 test-backend-ops cases match the CPU.
• All client tests on Windows and Linux, and the Windows install run from its own folders.

v.0.4.35-alpha

Install command, run in a user level powershell.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

Over SSH or from a script, save it and pass the options up front (with none, the installer only lists them):

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 -OutFile install.ps1
powershell -ExecutionPolicy RemoteSigned -File .\install.ps1 -NoOllama

On Linux:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Changed

• llmash update works on Linux. It could never download the installer (GitHub serves release files through a redirect, which the Linux download did not follow) and always ended by printing the curl command to run instead. Installs from 0.4.15 and older still print that command once; run it, and later updates work by themselves.

The runtime is still build 79, so an update from 0.4.34 does not download it again. See 0.4.34 for Linux getting llmash's own CUDA runtime.

v.0.4.34-alpha

Install command, run in a user level powershell.

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 | iex

Over SSH or from a script, save it and pass the options up front (with none, the installer only lists them):

irm https://raw.githubusercontent.com/omgitsbase/llmash/main/install.ps1 -OutFile install.ps1
powershell -ExecutionPolicy RemoteSigned -File .\install.ps1 -NoOllama

On Linux:

curl -fsSL https://raw.githubusercontent.com/omgitsbase/llmash/main/install.sh | sh

Already installed: llmash update.

Changed

• Linux gets llmash's own CUDA runtime, the same one Windows uses: MTP drafting with the built-in draft head, one CUDA graph per shape, and the rest. An NVIDIA card from Turing (RTX 20) on with driver 580 or newer takes it; anything else keeps upstream's Vulkan or CPU build. If it cannot run on a machine, the installer says why and falls back to the upstream build.
• llmash on Linux never learned how much GPU memory it had (it never searched the PATH for nvidia-smi), so it started every model without speculative decoding or backend sampling. It finds it now; Qwen3.8-27B GSQ IQ3_S through llmash on Linux runs at 176 tok/s on prose and 247 on code.
• llmash now runs on Ubuntu 22.04 (the last Linux build, 0.4.15, needed glibc 2.36), and Fedora no longer prints a libcurl warning on every command.
• llmash update on Linux installs over the existing copy (it put a second one inside it) and restarts the service so the new version is what runs. A later install or update keeps --no-service, --no-ollama and --runtime.
• The runtime picks CPU code for the processor it runs on, so a CPU without AVX2 (older Pentium and Celeron, or a VM whose CPU model hides it) no longer crashes it. The Windows runtime also carries the Visual C++ runtime, so it starts on a machine that never installed it.
• A verify of 7 drafted tokens (DFlash, or LLMASH_MTP_DRAFT=7) over IQ3_S / IQ3_XXS weights runs on tensor cores: Qwen3.8-27B GSQ IQ3_S +7-10%, RCO-3 +4%. The default of 3 drafts is unchanged.
• Fixed two runtime bugs the full op test suite turned up: a CUDA graph captured for f16 weights could be replayed over bf16 weights of the same shape, and the mixture-of-experts fusion could read past the end of a graph.

The runtime is build 79 (was 76), for Windows and now Linux. The full op test suite passes on builds for sm_75, sm_86, sm_89 and sm_120 (the older ones compiled for an RTX PRO 6000 at run time, so their code paths ran there), layer split gives the same text on 2 and 3 GPUs (emulated on one), and the Linux install was run end to end on Ubuntu 22.04, Fedora 40 and Arch.