Fork 0.1.44.1-nvfp4.2: Claude Code's system prompt rewritten for the local model

Fork 0.1.44.1-nvfp4.2 is one fix on top of 0.1.44.1-nvfp4.1. The engine is the same build (3dc869fb). Only the server's Python changed.

Claude Code's system prompt is rewritten for the local model. Claude Code writes its system prompt for Claude. About half of it is no use to a local model:

• an identity the model is not ("You are a Claude agent");
• Anthropic's security policy;
• the list of Claude model ids and Fast mode;
• the pronoun rule;
• a 1.3K-character memory how-to.

The server now recognises Claude Code's prompt by its billing header, drops those parts and cuts the memory section to its path. In front it adds a short note for a local agent: call only the listed tools, by their exact names, never as plain text, and read a tool's result before acting on it. These target what local models got wrong before: invented tool names and tool-call XML in the reply.

The harness rules, context management, plan mode, output styles and every section the rewrite does not know pass as sent. A future Claude Code keeps working. The rewrite gives the same text on every turn, so the conversation cache keeps hitting.

• Claude Code 2.1.296's prompt goes from 1,361 to 676 tokens. A real cllm request went from 17,375 to 16,657 input tokens.
• An 8-step agent task finished correctly, and every file:line it cited was right.
• "claude_code_prompt": "as_sent" in the model's config keeps the prompt as it comes.
• The server's 930 tests and the tools' 979 pass.

Update with UPDATE.bat, or replace the serve folder from this zip. No model files change.

SHA-256 72fb6aee02a7af4d65d92e35039c0c49b8dbd6cf5350ab1d681e2141cac71366

Fork 0.1.44.1-nvfp4.1: on upstream 0.1.44.1 - dynamic RoPE to 512K, kept dialogs give RAM back, faster several-GPU splits and dialogs, Qwen3.6-35B-A3B

Fork 0.1.44.1-nvfp4.1: on upstream Strata 0.1.44.1. The first release since 0.1.42-nvfp4.2 (0.1.43.1-nvfp4.1 and
0.1.44-nvfp4.1 were prepared and not published), so these notes cover all three. One GPU and one dialog run as
before: with the default settings and the same expert cache the engine gives the same bits as 0.1.43.1-nvfp4.1's and
0.1.44-nvfp4.1's, at the same speed.

Upstream 0.1.44.1 (hotfix):

• CPUs without AVX2 start again (#2324): the NVFP4 CPU rows built their tables before main with AVX/FMA
  instructions, so such a CPU died with an illegal instruction and nothing in the log; now there are no such
  initialisers, and the build's ISA check flags them.
• A crashed engine start is named (how the process ended, and the CPU's AVX flags after an illegal instruction)
  instead of being retried as a GPU runtime start.
• The Responses API (Codex) honours the config's reasoning_effort.
• Setup run again keeps the installed size instead of moving it to the new recommendation (IQ3_S). Here this also
  covers Qwen3.6, whose re-run moved an installed UD-Q4_K_XL to Q8_0 (a third of its speed on a 32 GB card) and
  defaulted to another family; a named --model still wins.

New in this fork since 0.1.44:

• Kept dialogs give RAM back when it runs short. Below --conversation-cache-min-free-mib (checked before every
  request, between batch windows and every 2 s while idle) the engine releases the retained K/V, then the parked
  dialogs oldest first (to the disk tier when one is set), then idle batch slots' checkpoints, until the floor + 1 GiB
  is free. On the 5090 with RAM pushed down to 8.6 GB available: three dialogs (3.3 GB) went in 0.1 s and 11.9 GB was
  available again; to the disk tier in 2.2 s. A released 32K dialog is read again in 3.6 s (0.9 s from the disk); the
  one still parked came back in 0.19 s. Without pressure nothing changes (the same bits).
• The low-RAM mode page-locks its RAM copy in slices, within the adapter's budget, instead of one allocation
  Windows usually refuses at this size. Q8_0 on 128 GB: decode 48.8 -> 55.0 tokens/s, a 32K prompt 9.9 -> 8.1 s, the
  start 8 s sooner, for ~0.7 GiB less available RAM. Under Windows the low-RAM mode now keeps 1500 MiB of VRAM free
  for the desktop by default (53.3 tokens/s and 8.5 s on an idle desktop; --vram-reserve-mib overrides it).
• Contexts past 256K use --rope-scaling dynamic: no scaling while a dialog fits the trained 262,144 positions
  (the same bits as none), yarn past them with the factor the length needs (rounded up to 0.125, at most
  --max-context / 262,144); the cached keys are rotated when the factor changes (tens of ms). Past 262K it gains what
  static yarn gains (-0.011 nats, ~1% perplexity, the needles 13/13 either way) and leaves every shorter dialog as it
  was (static yarn moves them by KL 0.056). Setup and Model settings write it for a context above 256K on one NVIDIA
  GPU with one dialog at a time (parallel 1); with several GPUs, several dialogs at once or an AMD card they write
  static yarn and say why. A choice you make yourself is kept; one the engine cannot run is refused with the reason.
• Every Qwen3.6-35B-A3B size can be installed on any PC: a size bigger than the RAM gets a note, not a question.

Upstream 0.1.44 ships this fork's NVFP4 engine path. Upstream's "NVFP4 packs" are a port of this fork's NVFP4
work: six of this fork's commits (the NVFP4 experts, the MMQ tails, mixed formats, the CPU share's NVFP4 rows, the AVX2
rows, the CPU rows' scales) are upstream's code now, byte for byte, and this fork runs upstream's copy. Upstream's
setup installs the same two Hugging Face uploads (--family nvfp4 --model NVFP4 | NVFP4-NVIDIA); here those names
install this fork's families (qwen-nvfp4-gptq, qwen-nvidia-nvfp4), so upstream's commands work and there is one
install path. Everything this fork adds on top stays: the faster prompt and decode kernels, the GPTQ, Q8_0 and mix
packs, the censorship switch, the low-RAM mode, the arena's own thread.

Upstream 0.1.44, of what runs here:

• Unsloth's files: UD-Q4_K_XL is a regular size now, and UD-Q5_K_XL, UD-Q6_K_XL and Q8_0 are new choices in
  setup. Setup keeps every expert in RAM where they fit with 16 GiB more (the rule upstream took from this fork) and
  writes upstream's settings for them (STRATA_HC_Q8=1, STRATA_MMQ_KQ=1 for Q5/Q6, the tail skip off where every
  expert is in RAM). Images are on by default for them. This fork's UD-Q3_K_XL stays a size.
• Setup's sizes: IQ3_S is the recommended GGUF size from 64 GB (it was IQ3_XXS).
• Qwen3.6-35B-A3B is a new family in setup (--family qwen36, Unsloth's GGUFs with the MTP layer; its own engine
  path in the same binary, text only so far). It runs here exactly as on upstream (the same tokens and logits): on an
  RTX 5090, UD-Q4_K_XL keeps every expert in VRAM and answers at ~410 tokens/s with its MTP drafts (a 33K prompt in
  5.3 s); UD-Q6_K_XL does not fit 32 GB whole and runs at ~130-150 tokens/s with part of its experts on the CPU. Setup
  recommends the largest size your RAM allows (Q8_0 from 64 GB), so on a 32 GB card pick UD-Q4_K_XL for speed.
• A picture in a long conversation no longer loses the cached state (#2228): pictures are matched by their bytes.
• A long prompt gives way to a short request at the next chunk (#2170); a quoted <tool_call>,
  <tool_response> or </tool_response> in a message is text (#2026).
• Two GPUs: the auto split keeps the most room on its fullest card among near-ties (#1440); a card behind a slow
  link costs more only from a 4x gap on (#2007); the VRAM check at start covers every card (#2192).
• Opt-ins: --adapt-async 1 in the arena mode (#1904), STRATA_PIN_BY_NEED=1 on Windows (#1905),
  --prefill-head-alias for 8 GB cards. The commit graph runs in one launch per layer kind (on by default, the same
  bits).

Upstream 0.1.43 and 0.1.43.1: non-finite logits are caught and the prompt read again instead of an "!" reply
(#879, #1892); the PLE probe only warns (#1984, #2040); the Windows stall watchdog no longer ends the engine while it
parks a conversation (#2049, #2065); a request's smaller prompt chunk no longer ends the engine (#2090); BENCH.bat.

Also in this fork since 0.1.42-nvfp4.2 (prepared as 0.1.44-nvfp4.1):

• Several GPUs (layer split, #7). A prompt on a split is read in chunks per stage, so the stages work at the
  same time; a card on a slow PCIe link holds every expert of its layers instead of swapping them; the CPU prompt
  share works on a split; a capped pin is shared between the stages; the arena loads on its own thread with one GPU
  only; a refused hand-off buffer makes the prompt chunk smaller instead of stopping the start; a later card's PCIe
  share starts at 0.25. Measured on rented RTX 4090s (PCIe 4, a Zen 2 host), decode / 32K prompt / 2K prompt in
  tokens/s:
  • 2 cards: 149.9 / 4,056 / 1,239 before, 144.8-150.4 / 6,936 / 1,742 now (one card: 93.8 / 5,472 / 1,250), so two
    cards now read a long prompt faster than one;
  • 4 cards: 154.9 / 3,880 / 3,149 before, 154.1-158.3 / 9,703-9,997 / 2,903-3,303 now;
  • a pair whose second card is on a ~2 GB/s link (16 GB cards): 66.8 / 1,592 / 434 before, 70.1 / 3,867 / 1,204 now,
    no longer slower than its first card alone (61.1 / 3,882 / 1,055).
    The answers' tokens are the same as on one card; one card runs exactly as before. The full tables are under
    "Measured on several GPUs" below.
• Pinned memory on Windows (WDDM). On a split, each GPU page-locks its own layers in its own context, up to
  that adapter's shared-memory budget, instead of one 8 GiB cap for all of them. Where little or nothing can be
  page-locked, the copy path is faster: a full copy ring and 8 copy threads. On this RTX 5090 with nothing pinned
  (STRATA_ARENA_PIN_GIB=none) a 32K prompt reads at 7,402-7,965 tokens/s instead of 6,065-6,434 (87-88% of the
  fully pinned speed instead of 70-71%); with 8 GiB pinned 7,717-8,225 instead of 6,365-6,673. On the rented cards:
  one 4090 with nothing pinned 4,363 / 824 (32K / 2K) instead of 3,153 / 554; two cards with 8 GiB pinned each
  4,053 / 1,216 instead of 2,547 / 835. The per-GPU pin gives the same bits and speed as the shared one on Linux; a
  real Windows PC with several GPUs has not run it yet (#7 asks for that test).
• Several dialogs at once ("parallel", --batch). A window graph that finds no VRAM no longer ends the engine
  (all answers in flight were lost); the slots' K/V grows with their context instead of taking the whole context up
  front (4 slots at 262K: 2.8 GiB instead of 14.6). On one 24 GB RTX 4090 with parallel 4 at 262K, 1 / 2 / 4 / 8
  dialogs at once ran 36.5 / 36.8 / engine ended / 30.9 tokens/s in all before and 72.0 / 73.4 / 94.0 / 96.2 now; a
  16 GB card did not start and now runs 53-65; two 4090s 68.4 / 83.1 / 105.2 / 108.4 before, 86.1 / 108.0 / 137.1 /
  146.3 now. On this RTX 5090: 127 / 141 / 179 / 185. parallel stays 1 by default: one dialog alone is fastest
  (on the 5090 it decodes 8-17% slower beside 3 empty slots), and Model settings lets you choose more.
  Also conversation-cache fixes from upstream pull requests #1503, #1163, #1164 and opt-in --prompt-cache-spread
  (#1959).
• Kept dialogs, in RAM and on the disk. --conversation-cache-mib keeps parked dialogs in RAM (off by default;
  --conversation-cache-mib 8192 keeps the default 4 dialogs, about 2.2 GB each at 100K tokens). New, opt-in:
  --conversation-disk-dir (from xrip's upstream PR #1937) writes what the RAM cache evicts to a folder and reads it
  back; an idle save after 60 s survives the tray's kill, and --conversation-disk-persist brings the dialogs back
  after a restart. Measured on the 5090, a dialog of 8K / 32K / 100K / 200K tokens comes back in...

Fork 0.1.42-nvfp4.2: qwen-nvfp4-mix and qwen-q8 for a 128 GB PC, Unsloth's files in RAM where they fit, UD-Q3_K_XL

Fork 0.1.42-nvfp4.2: two more exact models of the original Qwen for a 128 GB PC, and Unsloth's files faster.

New models in setup (the original, censored Qwen3.8-Flash-Next, our packs, ready-made from Hugging Face):

• --family qwen-nvfp4-mix: our GPTQ NVFP4 with Q8_0 experts in 38 of its 48 layers
  (Maximilian228/Qwen3.8-Flash-Next-NVFP4-Q8-Mix-Strata).
  It picks the Q8_0 model's top token 95.0% of the time (GPTQ: 93.6%). It runs at about half GPTQ's speed: 91-95
  tokens/s on an RTX 5090, a 32K prompt in 4.2 s. Every expert stays in RAM: 106 GiB in use with images and the server.
• --family qwen-q8: every routed expert Q8_0
  (Maximilian228/Qwen3.8-Flash-Next-Q8_0-Strata),
  the reference the other models are measured against. 46-47 tokens/s, a 32K prompt in 10.3 s. Its 119.5 GiB of
  experts do not fit 128 GB beside Windows, so the engine runs it in the low-RAM mode (100 GiB of them locked in RAM).
• Both have the censorship switch, off by default: 5 of 5 held-out harmful requests refused with it off, 0 of 5 with
  it on. Neither looped on the long Claude Code request (5 of 5 runs each).
• Both share qwen-nvfp4-gptq's dense weights, n-gram table, embedding and draft head. Installed after it, only their
  experts are downloaded (111 / 128 GB).
• Both are made for a 128 GB PC. Below that, setup says so and asks (default no): the engine would run them in its
  low-RAM mode, which was not measured with them. qwen-nvfp4-gptq stays the default.

Engine:

• The low-RAM mode also follows the pack's size. From 92 GiB installed, it now starts by itself where the pack's
  experts and 16 GiB more do not fit the RAM, and then it leaves 10 GiB free instead of 6. On a 128 GB PC the Q8_0
  pack runs in it; the mix and every other pack keep all their experts in RAM. Below 92 GiB nothing changes.
• The route tail skip is off by default for all of this fork's packs: NVFP4, Q8_0 and their mix. Before, it was
  off only for packs with NVFP4 experts, so the all-Q8_0 pack would have had it on. The GGUF models keep upstream's
  default; STRATA_ROUTE_TAIL_SKIP=7 still turns it on.

Unsloth's files:

• Every expert stays in RAM where they fit with 16 GiB more: from ~88 GB of RAM for UD-Q4_K_XL, ~72 for UD-IQ4_XS
  and ~68 for UD-Q3_K_XL. A 96 GB PC qualifies; a 64 GB one keeps the RAM-budget mode.
• On a 128 GB PC with an RTX 5090, UD-Q4_K_XL now decodes 169 tokens/s and reads a 32K prompt in 4.9 s, using
  77.3 GiB. The old RAM-budget mode gave 118 tokens/s and 30 s, using 98.5 GiB.
• To get this on an existing install, run setup again: START-HERE.bat --setup. --resident-budget-gib N still
  chooses the budget mode.
• UD-Q3_K_XL is a size of --family unsloth, as a user asked. With every expert in RAM it picks the Q8_0
  model's top token 90.7% of the time, decodes 208 tokens/s, reads a 32K prompt in 3.8 s and uses 57.5 GiB.

Checked (RTX 5090 + Ryzen 9 9950X3D, 128 GB, release build ddd78216, through the server):

• The Q8_0 pack starts in the low-RAM mode by itself and leaves 10 GiB free; the mix and qwen-nvfp4-gptq keep every
  expert in RAM.
• The tail skip is off by default on the Q8_0 pack, the mix and GPTQ, and on with =7. On upstream's IQ2_XS GGUF it
  is on by default.
• The censorship switch on the Q8_0 pack: 5 of 5 refused off, 0 of 5 on.
• Setup's plans for the four models at 128, 96 and 64 GB, and their configs.
• Tests: 115 of 118 engine tests pass; the 3 others need model files this PC does not have, and the PLE reader's
  timing test passes when run alone. The server's 708 pass. The tools' and setup's 788 pass, except one that needs
  yaml.
• Not run end to end: a real UD-Q3_K_XL install (its files were deleted after the measurement; the install path ran on
  a mocked PC), and the two new models on a PC with less than 128 GB.

docs/NVFP4.md, "Q8_0 for a 128 GB PC", has the measurements and every check.

SHA-256 7ea12301c0b7968f41e54745f06ca165567c3d0214b498c679d6153cc665fde4

Fork 0.1.42-nvfp4.1: on upstream 0.1.42 (its measured PCIe share; the route tail skip opt-in for NVFP4)

Fork 0.1.42-nvfp4.1: on upstream Strata 0.1.42.

Upstream 0.1.42, of what runs here:

• The route tail skip: upstream's default for the GGUF models, off by default for the NVFP4 models. With
  STRATA_ROUTE_TAIL_SKIP=7 (upstream's default on CUDA), a missed expert that every token of a verify window routes
  only at rank 7-9 is neither fetched nor computed. On qwen-nvfp4-gptq it moves the answers too much to be a
  default:

  • teacher-forced KL against the skip off is 0.035, as large as the whole GPTQ quantization error against an
    all-Q8_0 reference (0.038). Upstream measured 0.003-0.006 on its models. The top token is the same 93.7% of the
    time;
  • in the normal mode, thinking ends sooner: p(</think>) comes early x1.35.

  So with NVFP4 experts (qwen-nvfp4-gptq, qwen-nvidia-nvfp4) the engine leaves it off, and its start line says
  so. To turn it on for +11.5% decode (193.7 against 174.0 tokens/s over 5 interleaved rounds, faster in all 5):
  "env": {"STRATA_ROUTE_TAIL_SKIP": "7"} in the model's config. The GGUF models (qwen, unsloth and the other
  i-quants) keep upstream's default; STRATA_ROUTE_TAIL_SKIP=0 turns it off there.

• Decode's PCIe share is now upstream's measured one. It is set from the CPU pool's speed in the first decode windows: 0.25 -> 0.35 on an RTX 5090 + 9950X3D. It replaces this fork's per-layer cost model, which ran no faster on that PC. Speed is the same as with the fixed share: 0.983 +- 0.045 of it. STRATA_PCIE_FRAC_DEFAULT=old keeps the fixed share.

• Four fixes this fork sent upstream are upstream's code now, and the fork's own copies are gone:

  • the resident RAM mode's start reads experts.bin unbuffered (#1696);
  • the page-file advice is a fixed 65536 MB, counting only what is there for sure (#1693). This fork's warning below 60000 MB for the NVFP4 models stays and now advises the same 65536 MB;
  • the K/V trim waits for its residency upload (#1690);
  • the prompt buffers are sized for the layout in use (#1283).

• Upstream's coalesced Q8_0 dequant replaces this fork's.

• Upstream's other changes are new here too:

  • the CPU prompt share is not armed when RAM or commit is short, and a failed buffer no longer fails a chat;
  • the thinking budget is capped below max_tokens (#984);
  • a config reasoning_effort (#1641);
  • repeat_penalty / repeat_last_n (#1819);
  • the reasoning loop check also catches one long word (#1753);
  • keep-alives while a request waits, opt-in prevent_sleep, data-parallel replicas, logit bias on CUDA, a Prometheus endpoint;
  • CPU expert rows for CPUs without AVX2.

Two fixes the port needed:

• A small prompt with the CPU share stopped at prefill moe_combine: invalid argument. 0.1.42 allocates the share's rows buffer once, ahead of time, so this fork's NVFP4 row scales were never allocated. Fixed.
• Upstream's new PLE probe switches a "slow" drive to mapped reads. Here it timed its reads while the expert arena loaded from the same SSD, called a Samsung 9100 PRO slow, and a 32K prompt took 23.4 s instead of 3.7. It now runs once the arena is in: 810,000 rows/s, fine.

Checked (RTX 5090 + Ryzen 9 9950X3D, qwen-nvfp4-gptq):

• With fixed settings, a 2K and a 32K prompt give the same logits and 32 tokens as 0.1.41-nvfp4.7's engine. The tail skip, a server default, is not in these runs.
• Through the server, with the tail skip left unset, the answers are byte for byte those with STRATA_ROUTE_TAIL_SKIP=0 (13 of 13 test texts). With =7 it acts (KL 0.034 against it off). Upstream's IQ2_XS GGUF still starts with the skip on.
• With this fork's defaults off, the logits equal upstream 0.1.42's byte for byte on IQ2_XS (32K).
• Tests: 115 of 118 engine tests pass (the 3 others need model files this PC does not have). The server's 708 pass, and the tools' and setup's 769 pass except one that needs yaml.

docs/NVFP4.md, "Upstream 0.1.42", has the port, the measurements and every check.

SHA-256 b0505465404a396f93a63357939870777e8086c2cfe31f5bd4edec797798d939

Fork 0.1.41-nvfp4.7: Model settings has a Disable censorship switch too

Fork 0.1.41-nvfp4.7: the web page's Model settings has a Disable censorship switch too.

Disable censorship by default, from the web page. Model settings (the web page's About tab; GET / POST /config) now lists the run config's "uncensored" right after the sampling settings, like the other switches: default / on / off.

• On turns the refusal projection (layers 8-33) on for every request that does not say otherwise. It is the same default that START-HERE.bat --uncensored on writes.
• A chat or a request still decides for itself: the chat's Disable censorship switch and the API field "uncensored" win over this default.
• It takes effect from the model's next start, like every setting on that page.
• Only where setup loaded the projection: the original Qwen (qwen-nvfp4-gptq, ISTA's qwen, unsloth) and the Windows bundle. For NVIDIA's model (qwen-nvidia-nvfp4), Swift 1.5 and the Coder the entry is greyed out and saving it on is refused with the reason: NVIDIA's license does not allow bypassing the model's safety guardrails. Off and default are always allowed.

The engine is unchanged: 0.1.41-nvfp4.3's (1217e500). Tests: the server's 611 (13 skipped), including the new Model settings ones.

docs/NVFP4.md, "The censorship switch in Model settings", has the details.

SHA-256 569d31d55f7e06ba738a7085231a8f441479261f47effee8daed14bb9d1d834a

Fork 0.1.41-nvfp4.6: setup writes the model's recommended sampling

Fork 0.1.41-nvfp4.6: setup writes the model's recommended sampling, so a client that sends none no longer decodes greedy.

The model's recommended sampling is the default. Setup now writes this block into the config it builds (strata-<model>.json):

"sampling": {"temperature": 1.0, "top_p": 0.95, "top_k": 20}

These are the values from Qwen3.8-Flash-Next's generation_config.json. The server uses them for a request that sends none:

• a request's own values win;
• temperature 0 still means greedy.

Before this release, a client that sent no temperature decoded greedy. Claude Code is one such client. Qwen advises against greedy decoding with thinking: in our loop study, huihui-nvfp4 looped in 9 of 10 greedy runs and in 2 of 5 sampled ones.

• Every family gets it: the original Qwen's models (qwen-nvfp4-gptq, qwen-nvidia-nvfp4, ISTA's qwen, unsloth), Swift 1.5 and the Coder. Their own generation configs say the same; the Coder has none and takes Qwen's.
• Your own values stay. Running setup again keeps the sampling values you changed and adds only the missing ones.
• Older installs get it with UPDATE.bat (Linux ./update.sh): it adds the block to a config that has none. A block you wrote yourself is left as it is.
• The Windows bundle's config has it too. update.cmd brings it into a bundle whose config you did not change.

The engine is unchanged: 0.1.41-nvfp4.3's (1217e500). Tests: setup's 518, the bundle's 21.

docs/NVFP4.md, "The model's recommended sampling", has the details.

SHA-256 129a937e63359f56f7e08a99a8b246c203829795204b8d6c9d74ee9b6311e525

Fork 0.1.41-nvfp4.5: orca-nvfp4 withdrawn, the original Qwen the default, censorship off by a switch

Fork 0.1.41-nvfp4.5: orca-nvfp4 is withdrawn, the original Qwen is the default, and a switch turns its censorship off.

orca-nvfp4 is withdrawn: it is a much weaker agent. OrcaRouter's abliteration edits all 149 matrices that write the model's residual stream. On the same Claude Code task it ran 10-70 steps against the original Qwen's 175, with a clearly worse result. It also ends its thinking sooner: log p(</think>) is +0.119 nats higher over the model's own thinking, and on a long Claude Code request it thought ~1,000-1,150 tokens against the original's ~1,700.

• An install of orca-nvfp4 keeps working and nothing of it is deleted. Setup says so on each run and names the replacement.
• The default NVFP4 model is now qwen-nvfp4-gptq, our GPTQ quant of the original Qwen: Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata.

A "Disable censorship" switch, off by default. For qwen-nvfp4-gptq, setup loads our refusal-direction projection (data/uncensor) and leaves it off.

• How to switch it: the web app's Disable censorship switch (Sampling) or the API field "uncensored": true turns it on for a request, with no new setup run. Setup's question "Disable censorship?" (default no, or --uncensored on|off) sets the default: "uncensored" in strata-<model>.json.
• Off, a request runs the original model bit for bit.
• On, the engine removes the model's refusal direction from its residual stream after layers 8-33. The model's weights stay the original's.

Measured on our GPTQ quant:

	the original	censorship off	OrcaRouter's abliteration
refusals, thinking on (held-out, EN / UK)	almost all	0 of 104, 0 of 30	-
KL to the original	0	0.026	0.047
log p(</think>) shift	0	+0.019 nats	+0.119 nats
thinking on a long Claude Code request (5 runs, loops)	1,699 tokens (0)	1,947 tokens (0)	~1,000-1,150 tokens (0)

• The original Qwen's GGUFs too: the same switch, loaded and off by default, for ISTA's models (--family qwen) and Unsloth's (--family unsloth); it replaces upstream's speed projection there. Measured against each quant's own stock model on Q2_0, IQ2_XS and UD-Q4_K_XL: thinking-on refusals 1, 0 and 0 of 104; KL 0.016, 0.017 and 0.028; log p(</think>) +0.027 to +0.040 on Q2_0 and +0.050 on UD-Q4_K_XL's thinking, against +0.087 to +0.149 and +0.163 for upstream's vector. The same vector works unchanged on all three. IQ2_XS loops in long greedy agentic thinking by itself (3 of 3 replays, with the switch on or off), and setup now says so when it is chosen.
• Upstream's experimental_speed_projection is the same switch under its old name.
• It is never available for qwen-nvidia-nvfp4: NVIDIA's Open Model License does not allow bypassing the model's safety guardrails, so nothing is loaded there and setup refuses --uncensored on. It is not offered for Swift 1.5 or the Coder either, which have other weights.
• The vector is under the Qwen Community License 1.0. Removing refusals removes a safety behaviour; what the model writes with it on is your responsibility.

The Windows bundle downloads the model ready-made. prepare-model.cmd no longer converts orca's ModelOpt checkpoint. It downloads qwen-nvfp4-gptq: the files and revision setup pins, each checked against its SHA-256, plus the image encoder, about 130 GB. Its config loads the censorship switch, off. A bundle from before keeps starting orca from its own config until the new model is downloaded.

The engine is unchanged: 0.1.41-nvfp4.3's (1217e500). Tests: setup's 511, the bundle's 21, the server's 606.

docs/NVFP4.md, "orca-nvfp4 withdrawn, censorship off by a switch", and data/uncensor/README.md have the method and every number.

SHA-256 7eea0f9cc2e548d61fccbde060c7581ba591e90087d3a9c04b34bde471844eae

Fork 0.1.41-nvfp4.4: the original, censored Qwen in NVFP4 - NVIDIA's checkpoint converted, or our GPTQ

Fork 0.1.41-nvfp4.4: setup now also installs the original, censored Qwen3.8-Flash-Next in NVFP4, as NVIDIA's own checkpoint or as our GPTQ quant.

Two new models (orca-nvfp4 stays the default):

• --family qwen-nvidia-nvfp4: NVIDIA's NVFP4 checkpoint. This is nvidia/Qwen3.8-Flash-Next-NVFP4 (ModelOpt), quantized by NVIDIA and only converted to Strata's format, with its experts bit for bit: Maximilian228/Qwen3.8-Flash-Next-NVIDIA-NVFP4-Strata. The conversion is ours, not made or endorsed by NVIDIA. Licensed by NVIDIA Corporation under the NVIDIA Open Model License, with Qwen's Community License 1.0. The model is censored, as Qwen ships it, and NVIDIA's license does not allow bypassing its safety guardrails.
• --family qwen-nvfp4-gptq: our GPTQ quant of Qwen's BF16 model, by the same recipe as orca-nvfp4: Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata, under the Qwen Community License 1.0. It is censored too.
  • The experts' error is 33.3% of plain rounding's, and 44.8% of NVIDIA's: NVIDIA's own is 74.4% of plain rounding's. GPTQ is the lower in all 48 layers.
  • The same size as NVIDIA's pack, and as fast: 179 against 175 tokens/s, because more of the draft head's tokens are accepted.
• Both looped in none of 5 runs on the long Claude Code request that withdrew huihui-nvfp4 in 0.1.41-nvfp4.3.
• The two share the n-gram table, the embedding and the draft head. If you install both, the second links these 53 GB from the first instead of downloading them again.

START-HERE.bat --family qwen-nvfp4-gptq (or qwen-nvidia-nvfp4) installs one; Linux: ./setup.sh --family .... A model you already installed is not touched.

The engine is unchanged: 0.1.41-nvfp4.3's (1217e500). Setup's tests: 497 OK.

docs/NVFP4.md, "The original Qwen in NVFP4", has the details.

SHA-256 ffddc598cff0de516acf9247a43e01a3c19aa9cd456f822143817dd37b830a15

Fork 0.1.41-nvfp4.3: huihui-nvfp4 withdrawn (it loops in long thinking), orca-nvfp4 the default, the page file counted for sure

Fork 0.1.41-nvfp4.3: huihui-nvfp4 is withdrawn, orca-nvfp4 is setup's default, and the page-file check counts only what is there for sure.

huihui-nvfp4 is withdrawn: it often loops in long thinking. We captured a real Claude Code request (a 72K-token prompt, adaptive thinking at effort xhigh) and replayed it against the server:

• huihui-nvfp4 looped in its thinking in 9 of 10 greedy runs, on this release's engine and on the older 0.1.40.3-nvfp4.1 one, and in 2 of 5 runs with Qwen's recommended sampling;
• our OrcaRouter packs, the all-NVFP4 GPTQ one setup installs and a Q8_0-down one, looped in none of 5 runs each on the same request.

So the model loops, not the engine. Setup now offers one NVFP4 model, and it is the default where the PC meets the requirements: orca-nvfp4, Maximilian228/OrcaRouter-Qwen3.8-Flash-Next-Uncensored-NVFP4-GPTQ-Strata.

• If you installed huihui-nvfp4, it keeps working as it is: START-HERE.bat starts it, UPDATE.bat updates its engine, and nothing of it is deleted. Each run prints one note with the reason and the replacement.
• To switch: START-HERE.bat --family orca-nvfp4 (Linux: ./setup.sh --family orca-nvfp4). The 51 GB n-gram table is the same file in both models and is linked from your huihui install instead of being downloaded again.
• --family huihui-nvfp4 now stops with the reason and names orca-nvfp4.

The page file is counted for sure. 0.1.41-nvfp4.2 counted a system-managed page file at what Windows lets it grow to. A growing file may not grow in time while the GPU driver charges its VRAM: in upstream issue #60, "System managed" and 4096-32768 MB still ran out of commit and a fixed 64 GB worked. The engine and setup now count only what is there for sure: a file's size now, or its configured initial size. Below 60000 MB in all they warn and ask for a fixed 64000 MB: Win+R sysdm.cpl > Advanced > Performance Settings > Advanced > Virtual memory > Change: Custom size, initial and maximum 64000 MB, then restart Windows.

Checked (RTX 5090 + Ryzen 9 9950X3D):

• The engine is 1217e500. Its first-token logits and 200 greedy tokens are 0.1.41-nvfp4.2's, with every expert in RAM and in the low-RAM mode.
• Tests: setup's 486 (with the bundle's and tools' 522), the server's 601, the memory platform test.

update.cmd in an earlier bundle updates to this release; a clone's UPDATE.bat (Linux ./update.sh) does too.

docs/NVFP4.md, "huihui-nvfp4 withdrawn, the page file counted for sure", has the details.

SHA-256 fdfe0537e8eae6bc0f4358cdf5b47c2d438db12069950ddcc96dcae905eacdd0

Fork 0.1.41-nvfp4.2: a 96 GB PC keeps every expert in RAM, the low-RAM mode starts 3x faster, the page file is checked

Fork 0.1.41-nvfp4.2: a 96 GB PC now keeps every expert in RAM, the low-RAM mode starts 3x faster, and the page file is checked.

A 96 GB PC keeps every expert in RAM. Windows lists a 96 GB PC as 93-95.6 GiB, so the low-RAM mode (until now on below 96 GB installed) started there although the 63 GiB of experts fit. It now starts only below 92 GiB installed, which means a 64 GB PC; setup follows the same rule. Measured on a 96 GB PC emulated on the 128 GB one (95.6 GiB reported as installed, 88 GiB left free by a RAM ballast; 3 pairs, huihui-nvfp4):

• a chat: 13.7 ms a round instead of the low-RAM mode's 20.3 (12.7 with all 128 GB);
• a 32K prompt: 3.9 s instead of 8.4;
• 18-20 GiB of RAM stayed free, nothing was paged out.

The low-RAM mode starts faster (64 GB PCs, or --low-ram). Its VRAM cache and its RAM copy of the experts are read from experts.bin unbuffered, with the file's mapped view closed meanwhile. At 88 GiB: ready in 17.5 s instead of 53, a peak working set of 47 GiB instead of 85, and Windows kept 40 GiB free instead of 1. The same tokens and logits.

The page file is checked. Windows lets all programs together commit only RAM + page file, and the engine commits ~100 GiB with all experts in RAM (~80 in the low-RAM mode). Below 60000 MB of page file in all, the engine prints a WARNING at start and setup a framed warning (--check and the install): short of commit the expert cache in VRAM is made smaller, so the model can run significantly slower (measured +17% and +38% a round), or not start. Set 64000 MB: Win+R sysdm.cpl > Advanced > Performance Settings > Advanced > Virtual memory > Change: Custom size, initial and maximum 64000 MB, then restart Windows.

Two texts fixed:

• The low-RAM start's warning about lent cache slots said prompts could read ~3x slower "than with --prefill auto", while the run already used auto. It now says what happens: the RAM budget keeps no copy of the slots a prompt borrows, so a long prompt reads them from the SSD (a 32K prompt read 14.3 GiB).
• Setup no longer suggests --prefill auto:32768: this fork's auto already goes up to 32768.

Checked (RTX 5090 + Ryzen 9 9950X3D):

• The engine is 3a219488. Its first-token logits and 200 greedy tokens are those of 0.1.41-nvfp4.1, both with every expert in RAM and in the low-RAM mode.
• Tests: setup, bundle and tools 513 + 174, the server's 601, the memory platform test.
• Not measured on a real 96 GB PC: the emulation reports the installed size to the engine (STRATA_EMULATE_RAM_GIB, a diagnostic) and takes the free RAM with locked pages.

update.cmd in a 0.1.41-nvfp4.1 bundle updates to this release; a clone's UPDATE.bat (Linux ./update.sh) does too.

docs/NVFP4.md, "A 96 GB PC", has every measurement.

SHA-256 104f141384a660d6966cef3510d4124e2833657c33413ca8f84b45bd20675991