NInfer Workstation 2026.11.11-v3port.22

NInfer Workstation 2026.11.11-v3port.22 updates the native Windows engine from stable .21 and integrates upstream through 13ebb92f6.

Changes since .21

• Opt-in static YaRN RoPE scaling for the Qwen3.5-family route, including Qwen3.8-27B and its MTP, up to 1,048,576 positions. Existing profiles keep their selected context settings. This does not add Flash-Next YaRN or scale DFlash drafts.
• Native FP4 representations and projection/MoE schedules, synchronization fixes, recovery under resource pressure, context-fragmentation capture, reduced redundant prompt work, Jinja lifetime/iteration fixes and malformed-tool-history validation.
• Retains native Windows, resident Flash-Next Text/MTP/Vision, Supervisor, structured output, logprobs, scoring, System One and rerank. GLM conversion remains unregistered; Spark/distributed source is not a qualified fork product.
• Corrected a retained GDN test fixture to validate its stream before graph capture. The scoring-state fixture now checks all three actual Device RoPE input axes instead of requiring a redundant Host encoding; exact position, fresh-state, repeated-score and retirement criteria are retained. Updated the benchmark guide to the current prompt/conversation CLI and prefix-warmup behavior.

Qualification

• Colab G4, Linux: RTX PRO 6000 Blackwell Server Edition, driver 580.82.07, CUDA 13.3.73, GCC 13.3.0, Python 3.11.17. Full native build; all 33 selected native contracts; binding and all 26 resident Flash-Next Engine cases; reclaim and scoring-state checks pass. Covers graph/eager and batch parity, Text/MTP/Vision, cache/state reuse, scoring, logprobs, structured output, long FP8 and shared-prefix materialization. Both fixture failures above were found during this qualification, corrected and rerun; no numerical criteria were loosened. The image also required its missing liburing-dev dependency.
• Flash-Next smokes: ordinary and full-head MTP3 at 582 and 2,202 prompt tokens, three measured requests per setting, all reaching 65 output tokens. Fully cached prompts after native prefix warmup; no fresh-prefill or HTTP TTFT claim. Decode medians: 134.16 / 277.96 tok/s (582 tokens) and 130.85 / 275.77 tok/s (2,202 tokens). MTP adds approximately 1.52 GiB Device allocation in this configuration. These are absolute observations without a matched preceding Flash build.
• Windows 11, RTX PRO 6000 Blackwell Workstation Edition: MSVC 19.51, CUDA 13.3.33, driver 621.14. Full Release applications/tests/benchmarks build; 62 focused CTests and seven full Qwen3.8-27B NVFP4 transaction, pressure/recovery/cancellation and connected-client shutdown checks pass. Corrected GDN fixture rerun passes. Python 3.11: 113 tests and eight subtests pass with 16 CUDA-only skips under CPU Torch. Dashboard request tests, TypeScript checks and production build pass. Existing CRT link warning remains.
• Packaged Windows acceptance: all four applications start with a clean system PATH; installed payload hashes match the manifest; isolated install, upgrade and uninstall preserve settings, operator files and existing registration; the installed runtime performs real structured generation. Acceptance harness registry-handle and display-name checks were corrected and repeated without changing the installer payload.
• Served acceptance, Qwen3.8-27B NVFP4/DFlash2: current native serving-contract smoke and all 14 functional checks pass: models, English/Portuguese/code, thinking, streaming, forced tools, JSON schema, logprobs, vision, System One, rerank, prefix reuse and a four-way 20-second burst (564 requests, zero errors). Configuration and tray settings are preserved. Functional checks do not establish benchmark quality or calibration equivalence.

Performance and limits

The matched Windows comparison is against fork merge 8bcb72c85, not stable .21: three fresh runs per arm, Qwen3.8-27B NVFP4, FP8 KV, graphs, one lane, 256 output tokens. Median prefill changed 41.2 to 22.2 ms for the 42-token ordinary request and 320 to 299 ms for the 4,002-token MTP3 request. Decode medians changed +0.8% and +0.9%; ordinary ranges overlap and MTP accepted different speculative work, so these are not uniform or isolated-kernel speed gains. Workspace peaks rose 0.38 MiB (9.9%) and 1.0 MiB (0.2%), inside the same 535 MiB reservation. Reported GPU/Host allocations matched at displayed precision.

The earlier measured Flash-Next ordinary-decode cost from the single resident graph remains unresolved and documented in performance results. Current absolute Flash-Next smokes do not establish a regression-free matched comparison. No fresh physical RTX 5090, Windows resident Flash-Next, GLM, Spark/distributed or expert-offload qualification is claimed. YaRN long-context evidence remains specific to its feature qualification on G4. Historical JevBench scores were not rerun for this engine; this release does not establish unchanged System One probabilities or calibration.

Installation

Windows 11 x64, NVIDIA Blackwell (sm_120a) and a driver supporting CUDA 13.3 are required. Download the unsigned Windows x64 installer and verify it against NInfer-2026.11.11-v3port.22-SHA256SUMS.txt. Quit NInfer Workstation from the tray before upgrading. Models and settings are preserved. Flash-Next requires the 96 GB configuration and a compatible v3 artifact; model weights are not included.

The installer bundles the runtime and third-party notices. release-manifest.json records the source revision and payload hashes. FFmpeg is dynamically linked LGPL software; the matching source archive and FFmpeg-SOURCE.txt are attached. This installer is unsigned.

NInfer Workstation 2026.11.07-v3port.21 (Windows, Blackwell)

NInfer Workstation is the Windows-native fork of the NInfer inference engine. Unsigned Windows x64 installer for NVIDIA Blackwell GPUs (sm_120a, e.g., RTX 5090, RTX PRO 6000 Blackwell).

Stable release. It replaces 2026.11.07-v3port.20, the first release to carry the NInfer Workstation name on GitHub, and puts that name into the app itself. The engine binaries (ninfer-serve.exe, ninfer.exe) are byte-identical to 2026.11.06-v3port.19 and 2026.11.07-v3port.20.

Upgrading

From 2026.11.06-v3port.19 or 2026.11.07-v3port.20, only the Supervisor and the installer change. Engine behaviour and System One probabilities are the same as in those releases.

• The Start menu folder moves from NInfer to NInfer Workstation. Shortcuts you added to the old folder stay there.
• The installed app keeps its location, settings and data folder (%LOCALAPPDATA%\NInfer).

From earlier releases, startup flags changed, and the engine rejects the old forms:

• The first Supervisor start replaces retired flags in engine.args and the model catalog. The previous file stays beside it as supervisor.json.pre-migration, and the engine log lists every change.
• --kv-capacity counted tokens, so it becomes --context-capacity auto. Set an explicit size such as 20GiB if the GPU is shared.
• Commands run outside the Supervisor need updating by hand. The startup migration table lists each replacement.
• System One probabilities can shift between builds. Recalibrate thresholds after upgrading from a release before 2026.11.06-v3port.19.

Changes since 2026.11.07-v3port.20

• NInfer Workstation name: the installer, the Apps list entry, the Start menu shortcuts, the tray and the dashboard header use the name. The engine keeps the NInfer name.
• Dashboard header fix: between about 820 and 920 px wide, the alerts and display-mode buttons covered the last tabs. Below 1000 px the tabs now take their own row, and below 1180 px the header shows only "NInfer".

Changes since 2026.11.06-v3port.19

• Settings on one page: six sections (Model, Memory, Requests, Access, Web frontends, Logs and launch) replace nine separate pages. An index shows each section's current values and unsaved changes, and search finds any setting by name, --flag or current value.
• Context capacity has a slider again: choose Auto or a fixed size, or type an exact size such as 20GiB. Beside each memory control, a panel shows what the choice buys, such as how many full-length requests the pool holds and how long each request can be with every lane active, and how much memory stays free for the desktop. The figures come from the engine's own plan; when they can only be estimated before a restart, they are labelled approximate.
• Plain wording: Save, Discard, and Save and restart.
• One dashboard look on every page: Monitor, Chat, Connect, the API test bench and Diagnose share the new design, in Day and Night modes and in narrow windows.
• Fixes:
  • The throughput log interval setting wrote a flag the engine rejects, so saving it stopped the engine from starting. A test now checks every Settings flag against the engine.
  • A smaller context pool and a larger desktop reserve can be saved together; before, the reserve was checked against the old pool and refused.
  • Four more retired engine flags are migrated at startup: --max-private-continuations, --max-shared-prefixes and --host-state-slots are dropped, and --host-kv-mib becomes --host-cache-capacity. A preset that still carried them failed to start.

Changes since 2026.10.14-v3port.11

• Upstream sync through a754fcf3:
  • Unified startup configuration, with size and duration quantities.
  • Unified Context storage for State and KV, sized by --context-capacity.
  • Native sessions and unified Engine observations.
  • A unified FP8 KV codec, which changes stored cache arithmetic.
• Supervisor: replaces retired engine flags at managed startup; an invalid saved command line no longer stops the Supervisor; a failing dashboard request costs only that request.
• Structured output: JSON response formats accept structural whitespace, with each run capped at 128 characters; a reasoning_prefix grammar no longer leaks </think> into content.
• Context: fixed a coverage metadata undercount that left checkpoint reserves slightly short.
• Windows: builds on MSVC before 19.51, and the test suite runs natively.
• Packaging: adds the libjpeg-turbo runtime and notices.

Qualification

• This commit: Windows 11, MSVC 19.51, CUDA 13.3. Only the Supervisor and launcher differ from the previous payload; the engine binaries are byte-identical. Both Supervisor tests pass: the C++ suite, and the dashboard script and capacity arithmetic check. The dashboard header was measured at 17 widths from 390 to 1600 px with no tab overlapping the brand or the controls.
• Upgrade from 2026.11.07-v3port.20 on an RTX PRO 6000 Blackwell: the Start menu folder moved to NInfer Workstation with three working shortcuts, the Apps entry reads "NInfer Workstation", the configuration was unchanged, and 14 of 14 serving checks pass afterwards (chat in English and Portuguese, code, thinking, streaming, tool calls, JSON schema, logprobs, vision, System One, rerank, prefix reuse and a four-way burst).
• Engine (identical binaries, qualified on 2026.11.06-v3port.19): full Windows build (315 projects); the non-model test suite passes with two tests skipped; Qwen3.8-Flash-Next passes all 26 real-model Engine cases on RTX PRO 6000 Blackwell. On an RTX 5090, the dense 27B passes the same 14 serving checks and 213 mixed requests without errors.
• System One: on the 231 public JevBench items, Qwen3.8-27B NVFP4 with FP8 KV answers 194 correctly with a single readout, 202 averaged and 219 with averaged readout plus reasoning escalation (Brier 0.092). Thresholds depend on the model file and KV format, so calibrate each installation separately.
• Not run on this commit: a fresh install on a clean machine, the real-model test suite, and a Linux build.

Known limits

• On Windows, GPU allocations count against system commit, and the default Host context pool needs about 9 GiB on top. If cudaMallocHost fails while RAM looks free, check free commit.
• Qwen3.8-Flash-Next needs the 96 GB configuration; it is not supported on 32 GB GPUs.

Requirements

• Windows 11 x64
• NVIDIA Blackwell GPU (sm_120a) with a driver supporting CUDA 13.3
• Local .ninfer v3 model artifacts

Verification

3089db901df7fb4e910a818c891755230bf4a480cc779eebc2b1907399b7b65b  NInfer-2026.11.07-v3port.21-windows-x64-unsigned.exe

NInfer Workstation 2026.11.07-v3port.20 (Windows, Blackwell)

NInfer Workstation is the Windows-native fork of the NInfer inference engine. Unsigned Windows x64 installer for NVIDIA Blackwell GPUs (sm_120a, e.g., RTX 5090, RTX PRO 6000 Blackwell).

Stable release. It replaces 2026.10.14-v3port.11 as the latest stable release and supersedes the 2026.11.06-v3port.19 pre-release. The engine binaries (ninfer-serve.exe, ninfer.exe) are byte-identical to 2026.11.06-v3port.19; this release adds the redesigned Supervisor.

Upgrading

From 2026.11.06-v3port.19, only the Supervisor changes. Engine behaviour and System One probabilities are the same as in that release.

From earlier releases, startup flags changed, and the engine rejects the old forms:

• The first Supervisor start replaces retired flags in engine.args and the model catalog. The previous file stays beside it as supervisor.json.pre-migration, and the engine log lists every change.
• --kv-capacity counted tokens, so it becomes --context-capacity auto. Set an explicit size such as 20GiB if the GPU is shared.
• Commands run outside the Supervisor need updating by hand. The startup migration table lists each replacement.
• System One probabilities can shift between builds. Recalibrate thresholds after upgrading from a release before 2026.11.06-v3port.19.

Changes since 2026.11.06-v3port.19

• Settings on one page: six sections (Model, Memory, Requests, Access, Web frontends, Logs and launch) replace nine separate pages. An index shows each section's current values and unsaved changes, and search finds any setting by name, --flag or current value.
• Context capacity has a slider again: choose Auto or a fixed size, or type an exact size such as 20GiB. Beside each memory control, a panel shows what the choice buys, such as how many full-length requests the pool holds and how long each request can be with every lane active, and how much memory stays free for the desktop. The figures come from the engine's own plan; when they can only be estimated before a restart, they are labelled approximate.
• Plain wording: Save, Discard, and Save and restart.
• One dashboard look on every page: Monitor, Chat, Connect, the API test bench and Diagnose share the new design, in Day and Night modes and in narrow windows.
• Fixes:
  • The throughput log interval setting wrote a flag the engine rejects, so saving it stopped the engine from starting. A test now checks every Settings flag against the engine.
  • A smaller context pool and a larger desktop reserve can be saved together; before, the reserve was checked against the old pool and refused.
  • Four more retired engine flags are migrated at startup: --max-private-continuations, --max-shared-prefixes and --host-state-slots are dropped, and --host-kv-mib becomes --host-cache-capacity. A preset that still carried them failed to start.

Changes since 2026.10.14-v3port.11

• Upstream sync through a754fcf3:
  • Unified startup configuration, with size and duration quantities.
  • Unified Context storage for State and KV, sized by --context-capacity.
  • Native sessions and unified Engine observations.
  • A unified FP8 KV codec, which changes stored cache arithmetic.
• Supervisor: replaces retired engine flags at managed startup; an invalid saved command line no longer stops the Supervisor; a failing dashboard request costs only that request.
• Structured output: JSON response formats accept structural whitespace, with each run capped at 128 characters; a reasoning_prefix grammar no longer leaks </think> into content.
• Context: fixed a coverage metadata undercount that left checkpoint reserves slightly short.
• Windows: builds on MSVC before 19.51, and the test suite runs natively.
• Packaging: adds the libjpeg-turbo runtime and notices.

Qualification

• This commit: Windows 11, MSVC 19.51, CUDA 13.3. The release build recompiles only the Supervisor. Both Supervisor tests pass: the C++ suite, and the dashboard script and capacity arithmetic check. The installer upgrades a 2026.11.06-v3port.19 installation with an unchanged configuration; on an RTX PRO 6000 Blackwell, 14 of 14 serving checks pass afterwards (chat in English and Portuguese, code, thinking, streaming, tool calls, JSON schema, logprobs, vision, System One, rerank, prefix reuse and a four-way burst).
• Engine (identical binaries, qualified on 2026.11.06-v3port.19): full Windows build (315 projects); the non-model test suite passes with two tests skipped; Qwen3.8-Flash-Next passes all 26 real-model Engine cases on RTX PRO 6000 Blackwell. On an RTX 5090, the dense 27B passes the same 14 serving checks and 213 mixed requests without errors.
• System One: on the 231 public JevBench items, Qwen3.8-27B NVFP4 with FP8 KV answers 194 correctly with a single readout, 202 averaged and 219 with averaged readout plus reasoning escalation (Brier 0.092). Thresholds depend on the model file and KV format, so calibrate each installation separately.
• Not run on this commit: the real-model test suite, and a Linux build.

Known limits

• On Windows, GPU allocations count against system commit, and the default Host context pool needs about 9 GiB on top. If cudaMallocHost fails while RAM looks free, check free commit.
• Qwen3.8-Flash-Next needs the 96 GB configuration; it is not supported on 32 GB GPUs.

Requirements

• Windows 11 x64
• NVIDIA Blackwell GPU (sm_120a) with a driver supporting CUDA 13.3
• Local .ninfer v3 model artifacts

Verification

36a104a9d0dbdca406cf447efc1f363b6fff8da185abbd8e8cfbda7063a79dee  NInfer-2026.11.07-v3port.20-windows-x64-unsigned.exe

NInfer 2026.11.06-v3port.19 (Windows, Blackwell)

Unsigned Windows x64 installer for NVIDIA Blackwell GPUs (sm_120a, e.g., RTX 5090, RTX PRO 6000 Blackwell).

Pre-release. Acceptance on this commit is still open, see #37. 2026.10.14-v3port.11 stays the latest stable release until then.

Upgrading from an earlier release

Startup flags changed, and the engine rejects the old forms.

• The first Supervisor start replaces retired flags in engine.args and the model catalog. The previous file stays beside it as supervisor.json.pre-migration, and the engine log lists every change.
• --kv-capacity counted tokens, so it becomes --context-capacity auto. Set an explicit size such as 20GiB if the GPU is shared.
• Commands run outside the Supervisor need updating by hand. The startup migration table lists each replacement.
• System One probabilities can shift between builds. Recalibrate thresholds after upgrading.

Changes since 2026.10.14-v3port.11

• Upstream sync through a754fcf3:
  • Unified startup configuration, with size and duration quantities.
  • Unified Context storage for State and KV, sized by --context-capacity.
  • Native sessions and unified Engine observations.
  • A unified FP8 KV codec, which changes stored cache arithmetic.
• Supervisor:
  • Replaces retired engine flags at managed startup.
  • An invalid saved command line no longer stops the Supervisor; the dashboard stays available to show the engine's error.
  • A failing dashboard request now costs that request instead of the whole app.
• Structured output:
  • Fixed a reasoning_prefix grammar that could leak </think> into content.
  • JSON whitespace is capped at 128 characters.
• Context: fixed a coverage metadata undercount that left checkpoint reserves slightly short.
• Windows:
  • Builds on MSVC before 19.51, which rejected five kernel launchers (C2326).
  • The test suite now runs natively, including six test fixtures that had not moved to the current Op APIs.
• Packaging: adds the libjpeg-turbo runtime and notices.

Qualification so far

• Windows MSVC, RTX PRO 6000 Blackwell: full build passes (315 projects). The non-model test suite passes, with two tests skipped.
• Qwen3.8-Flash-Next: all 26 real-model Engine cases pass on RTX PRO 6000 Blackwell, on the merge commit. Its model runs need the 96 GB configuration, so it is out of scope on the RTX 5090.
• RTX 5090, dense 27B: selected native checks and a paired comparison pass on the merge commit with local patches now included here. Generated IDs match in all 180 paired rows. Worst paired wave slowdown is 1.0%, with decode down as much as 1.36% (median).
• Not yet run on this commit: the real-model test suite, and RTX 5090 acceptance.

Known limits

• On Windows, GPU allocations count against system commit, and the default Host context pool needs about 9 GiB on top. If cudaMallocHost fails while RAM looks free, check free commit.
• A Linux build of this commit has not been run.

Requirements

• Windows 11 x64
• NVIDIA Blackwell GPU (sm_120a) with a driver supporting CUDA 13.3
• Local .ninfer v3 model artifacts

Verification

3cf6536f38876db0709be0572513c4e994423c3093bb59abcf13522b4475b559  NInfer-2026.11.06-v3port.19-windows-x64-unsigned.exe

Source commit: 6772bfdeb1c28dbcd17333a0a761b635dad41399

Third-party notices and LGPL source

The installer bundles unmodified LGPL shared FFmpeg libraries (avcodec-63.dll, avformat-63.dll, avutil-61.dll, swresample-7.dll, swscale-10.dll) from BtbN/FFmpeg-Builds (n9.0.2-3-ga5923073bf). The exact corresponding FFmpeg source archive (ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz) and FFmpeg-SOURCE.txt are attached to this release, and the notices are installed under licenses\. hwloc and libjpeg-turbo notices are installed there too.

NInfer 2026.10.14-v3port.11 (Windows, Blackwell)

Unsigned Windows x64 installer for NVIDIA Blackwell GPUs (sm_120a, e.g., RTX 5090, RTX PRO 6000 Blackwell).

Changes since 2026.10.07-v3port.10

• Speculative Decoding with top_logprobs (3876894):
  • Extended candidate_logprobs to compute device-side top-K log-probabilities across all accepted verification positions in a single CUDA-graph-capturable pass without host logits transfers.
  • Removed the top_logprobs > 0 speculative decoding fallback in Qwen3_5 / Qwen3_8 and Qwen4_Exp so MTP-5 and DFlash2-7 remain active during token probability inspection.
  • Added choices[0].logprobs emission on the final streaming chat.completion.chunk when stream: true and logprobs: true.
• Supervisor Chat Workspace & Streaming Performance (3876894, 58df025, 103098d):
  • Redesigned the Supervisor Chat tab into a two-pane workspace with a sticky right-hand Token Probabilities inspector, live streaming token telemetry, and throttled incremental markdown rendering.
• Single-Stream Speculative Decode & Prefill Optimizations (2b62e8e, 9718f98):
  • Extended Fp8A16SlicedKMmaSchedule<4, 8, 7> with Epilogue::kRowTokens so = 3..8$ FP8 GDN input projections fuse causal conv1d + SiLU in a single kernel (-48 graph nodes, zero workspace).
  • Raised nvfp4_linear_add = 17408$ A16 threshold to \le 8$ with KWarps = 8 (-56 graph nodes, zero workspace) and extended fp8_linear_swiglu A16 through \le 16$ (-8 graph nodes, zero workspace).
  • Added RTX PRO 6000 Blackwell (sm_120a, 188 SMs) dispatch and persistent tile schedule tuning across Qwen3.8-27B and Qwen3.8-Flash-Next.

Requirements

• Windows 11 x64
• NVIDIA Blackwell GPU (sm_120a) with driver supporting CUDA 13.3 (
  vcuda.dll)
• Local .ninfer v3 model artifacts

Verification

1a9ddecfa9f050123a6fbfe114872777d5c464c4071c056283e79049f1716f2a NInfer-2026.10.14-v3port.11-windows-x64-unsigned.exe

Source commit: 2b62e8e5e558ac1572c35f64269e1db30b85ef9d (workstation)

Third-party notices and LGPL source

The installer bundles unmodified LGPL v2.1+ FFmpeg shared libraries (�vcodec-63.dll, �vformat-63.dll, �vutil-61.dll, swresample-7.dll, swscale-10.dll) from BtbN/FFmpeg-Builds (
9.0.2-3-ga5923073bf). The exact corresponding FFmpeg source archive (fmpeg-n9.0.2-3-ga5923073bf-source.tar.gz) and FFmpeg-SOURCE.txt are attached to this release and installed under licenses.

NInfer 2026.10.07-v3port.10 (Windows, Blackwell)

Alpha, unsigned. Windows 11 x64 installer for NVIDIA Blackwell sm_120 GPUs (RTX 5090, RTX PRO 6000 Blackwell). Built from 61afb718 on workstation.

Install

Run NInfer-2026.10.07-v3port.10-windows-x64-unsigned.exe, open NInfer from Start, and select a downloaded .ninfer model. Windows will warn about an unknown publisher because the installer is unsigned; check it against NInfer-2026.10.07-v3port.10-SHA256SUMS.txt:

1648a298d4ef6ddd62d29eb2e9cb20ae836ad478e5031364cc651befc06fc8c4  NInfer-2026.10.07-v3port.10-windows-x64-unsigned.exe

Needs a current NVIDIA driver with CUDA 13.3 support. No CUDA Toolkit, Visual Studio or Python is needed. Settings and logs live in %LOCALAPPDATA%\NInfer and survive updates and uninstall. Quit NInfer from its tray menu before updating.

Models

This release reads NInfer v3 artifacts only. Model weights are not included:

• Qwen3.8-27B NVFP4: https://huggingface.co/neroued/Qwen3.8-27B-nvfp4-NInfer
• Qwen3.8-27B Uncensored NVFP4: https://huggingface.co/igorls/Qwen3.8-27B-Uncensored-NVFP4-NInfer
• Qwen3.8-Flash-Next: derive the v3 artifact from the published v2 file igorls/Qwen3.8-Flash-Next-mixed-NInfer with tools/convert/qwen4_exp. The v3 file is not a separate download. See docs/maintainer/qwen3.8-flash-next-artifact.md.

A v2 .ninfer file from an earlier download does not load; the Supervisor's model list marks it. Download the current file, or upgrade a 27B file in place with tools/upgrade_ninfer_v2_to_v3.py from the repository.

What's new since 2026.10.02-v3port.9

Qwen3.8-Flash-Next on the v3 engine. This build includes the native text, MTP and Vision package. MTP uses the full stored output head at K=1 through 5. The Supervisor accepts that range and rejects an optimized draft head for this model. Image and video input needs --vision on the engine arguments. Text-only models still run without that flag.

Prefill chunk size does not change Flash-Next results, except inside a final chunk shorter than 256 tokens, as with the previous artifact. A token's logits and continuation state stay bit-identical across larger chunkings.

Three process-fatal engine failures are fixed.

• A burst of requests that reuse a shared prefix could abort the process on both the 27B and Flash-Next with selected pressure target could not be sealed. That admission now waits, and the engine tries it again after the next execution step.
• Flash-Next MTP could overwrite a waiting request's prompt tokens (candidate token ledger does not match prompt length). The MTP round now uses its own buffer.
• After saving a shared prefix, Flash-Next MTP could leave its KV cache unmapped (KV committed frontier is invalid). It remaps that cache together with the text cache.

Recorded qualification. The independent-oracle numerical gate was accepted on October 6, 2026, and one failing metric stays on record: mean chosen-token difference, entirely on one long document. Comparison against the saved v2 trajectories was not accepted. On an RTX PRO 6000 Blackwell, the Flash-Next Op tests, the loading test, the real-artifact Engine cases and ninfer_bench passed. The serving check with a KV cache small enough to leave a desktop usable is still open. See docs/maintainer/upstream-ports.md#flash-next-v3 and docs/performance.md.

The 27B decision policy, hidden-row export, document reranking, web frontends and the v3 engine from 2026.10.02-v3port.9 are unchanged.

Licenses

NInfer is Apache-2.0. Bundled third-party licenses are in the licenses folder of the installation. FFmpeg n9.0.2-3-ga5923073bf is included as dynamically linked LGPL-3.0-or-later DLLs, which you may replace. Its corresponding source is attached here as ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz. FFmpeg-SOURCE.txt names the build and the build recipe for the libraries linked into it.

Assets

• NInfer-2026.10.07-v3port.10-windows-x64-unsigned.exe: installer
• NInfer-2026.10.07-v3port.10-SHA256SUMS.txt: installer checksum
• release-manifest.json: source commit plus SHA-256 of every bundled file
• ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz, FFmpeg-SOURCE.txt: FFmpeg corresponding source

NInfer 2026.10.02-v3port.9 (Windows, Blackwell)

Alpha, unsigned. Windows 11 x64 installer for NVIDIA Blackwell sm_120 GPUs (RTX 5090, RTX PRO 6000 Blackwell). Built from 2969beee on workstation.

Install

Run NInfer-2026.10.02-v3port.9-windows-x64-unsigned.exe, open NInfer from Start, and select a downloaded .ninfer model. Windows will warn about an unknown publisher because the installer is unsigned; check it against NInfer-2026.10.02-v3port.9-SHA256SUMS.txt:

88680a1db6f1333220277867bd5b57250584ae5013f8267526202c1bc221bcad  NInfer-2026.10.02-v3port.9-windows-x64-unsigned.exe

Needs a current NVIDIA driver with CUDA 13.3 support. No CUDA Toolkit, Visual Studio or Python is needed. Settings and logs live in %LOCALAPPDATA%\NInfer and survive updates and uninstall. Quit NInfer from its tray menu before updating.

Models

This release reads NInfer v3 artifacts only. Model weights are not included:

• Qwen3.8-27B NVFP4: https://huggingface.co/neroued/Qwen3.8-27B-nvfp4-NInfer
• Qwen3.8-27B Uncensored NVFP4: https://huggingface.co/igorls/Qwen3.8-27B-Uncensored-NVFP4-NInfer

A v2 .ninfer file from an earlier download does not load; the Supervisor's model list marks it. Download the current file, or upgrade it in place with tools/upgrade_ninfer_v2_to_v3.py from the repository. Qwen3.8-Flash-Next is not available in this release: its port to the v3 engine is in progress.

What's new since 2026.10.02-v3port.8

A decisions API that reaches and, on hard decisions, exceeds a dedicated decision model, from the same resident model that serves chat.

• System One decision policy. POST /v1/systemone accepts an opt-in policy object, with server defaults --systemone-readout, --systemone-reasoning-budget and --systemone-escalate-entropy. The default request keeps the single native readout and its exact prompt bytes, so existing clients see unchanged answers.
  • "readout": "averaged" decides each question from three prompts over one state (the native prompt, a lettered prompt with the option order rotated by one, and an evidence/criterion framing) and averages their renormalised distributions. It removes most of the option-order and letter-position sensitivity of a single readout at the cost of two extra prefills per question.
  • "reasoning_budget": N escalates a question whose averaged distribution has normalised entropy above escalate_entropy (default 0.66) to one bounded-thinking generation of N tokens and mixes its answer in at answer_weight (default 0.7). Escalation costs seconds per escalated question, so the budget is a per-request or per-deployment latency decision.
  • usage.policy reports the readout, the number of escalated questions and their reasoning tokens; usage.input_tokens bills every prompt.
  • Measured on the 231 public JevBench items with Qwen3.8-27B NVFP4 (same items, paired by scenario bootstrap): single readout 194 correct, Brier 0.216; averaged 203, Brier 0.173, p50 92 ms; averaged with a 1,024-token budget 218, hard tier 98 of 111, Brier 0.099, with 26% of items escalated, p50 103 ms, p95 5.6 s. Cloudflare's Clef decision head, run in BF16 on the same items, scores 201 with Brier 0.166; the escalated policy fixes 19 of those items and breaks 2 (95% interval +3.5 to +11.2 points). These are public-item results; no sealed score is claimed. See docs/serving.md#decision-policy and docs/research/decision-head-plan.md.
• Hidden-row export for training readouts. CausalScoreReadout::capture_hidden_rows returns the final-normalized hidden row of every scored position from Engine::score_tokens, and the new ninfer-hidden-export app (built from source; not part of the installer) writes those rows for token-id records as a safetensors file carrying the artifact, prefill chunk and KV format. The causal scoring Engine now keeps the caller's prefill chunk instead of forcing 1,024, so an export can match a serving configuration. See docs/perplexity.md#hidden-row-export.
• Decision policy study tooling. tools/bench/jevbench/decision_policy/ collects native, reordered, framed and bounded-reasoning readouts over a labeled corpus and evaluates averaging, calibration and confidence-gated escalation with dev-selected thresholds and paired statistics.
• Research record. docs/research/decision-head-plan.md records the study behind the policy: the Clef head is a pure add-on to the stock Qwen3.8-27B weights; on the quantized backbone its vector-similarity scoring collapses, and a fine-tune recovers only part of it; hidden states of the quantized backbone depend on the prefill chunk's token count (inherent to 4-bit and 8-bit activation quantization, not an engine defect); calibrators fitted on one workload do not transfer to another; the native readout with averaging and a reasoning budget is the route that wins.

Earlier changes since 2026.09.24-alpha.1

• v3 engine. The fork now builds on upstream NInfer's v3 engine and .ninfer v3 artifacts, merged through upstream d44ab584.
• Document reranking. POST /v1/rerank takes a Jina/Cohere-shaped body and scores each document with an in-process System One Choice. The model id is ninfer-choice-rerank-v1, and return_documents defaults to true. Rerank only reads the prompt cache, so it never evicts cached conversations. See docs/serving.md.
• Web frontends in the Supervisor. Each configured static web app, such as llama.cpp's web UI, is served on its own local port with the engine API forwarded on that port. A browser chat needs no proxy, CORS setup or API key in the page, and responses stream. Add, change or remove frontends in the dashboard under Settings > Web frontends; saving applies right away while the engine keeps running. The tray and the dashboard link to each frontend. See docs/windows-app.md#web-frontends.
• Supervisor dashboard stays fast with a large request log. The Supervisor now reads the engine request log once in the background and then follows it, instead of re-reading the whole file for every dashboard refresh. With a 778 MB log, a dashboard state request went from about 27 s to about 14 ms, and the Supervisor's peak memory from about 7.3 GB to 34 MB. After a start it reads the existing log once (about 10 s per GB) and shows "Reading log NN%" meanwhile.
• llama.cpp web UI compatibility. GET /props, Chat Completions without model, max_tokens: -1, and thinking_budget_tokens.
• CUDA synchronization blocks instead of spinning. Decode speed is unchanged, and about one CPU core is freed while decoding.

Licenses

NInfer is Apache-2.0. Bundled third-party licenses are in the licenses folder of the installation. FFmpeg n9.0.2-3-ga5923073bf is included as dynamically linked LGPL-3.0-or-later DLLs, which you may replace. Its corresponding source is attached here as ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz. FFmpeg-SOURCE.txt names the build and the build recipe for the libraries linked into it.

Assets

• NInfer-2026.10.02-v3port.9-windows-x64-unsigned.exe: installer
• NInfer-2026.10.02-v3port.9-SHA256SUMS.txt: installer checksum
• release-manifest.json: source commit plus SHA-256 of every bundled file
• ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz, FFmpeg-SOURCE.txt: FFmpeg corresponding source

NInfer 2026.10.02-v3port.8 (Windows, Blackwell)

Alpha, unsigned. Windows 11 x64 installer for NVIDIA Blackwell sm_120 GPUs (RTX 5090, RTX PRO 6000 Blackwell). Built from 97d7a785 on workstation.

Install

Run NInfer-2026.10.02-v3port.8-windows-x64-unsigned.exe, open NInfer from Start, and select a downloaded .ninfer model. Windows will warn about an unknown publisher because the installer is unsigned; check it against NInfer-2026.10.02-v3port.8-SHA256SUMS.txt:

153727e5b0b215916ac32b616c5ebc0d57882bb0eca4d5f941a9ee8f74a30651  NInfer-2026.10.02-v3port.8-windows-x64-unsigned.exe

Needs a current NVIDIA driver with CUDA 13.3 support. No CUDA Toolkit, Visual Studio or Python is needed. Settings and logs live in %LOCALAPPDATA%\NInfer and survive updates and uninstall. Quit NInfer from its tray menu before updating.

Models

This release reads NInfer v3 artifacts only. Model weights are not included:

• Qwen3.8-27B NVFP4: https://huggingface.co/neroued/Qwen3.8-27B-nvfp4-NInfer
• Qwen3.8-27B Uncensored NVFP4: https://huggingface.co/igorls/Qwen3.8-27B-Uncensored-NVFP4-NInfer

A v2 .ninfer file from an earlier download does not load; the Supervisor's model list marks it. Download the current file, or upgrade it in place with tools/upgrade_ninfer_v2_to_v3.py from the repository. Qwen3.8-Flash-Next is not available in this release: its port to the v3 engine is in progress.

What's new since 2026.09.24-alpha.1

• v3 engine. The fork now builds on upstream NInfer's v3 engine and .ninfer v3 artifacts, merged through upstream d44ab584.
• Document reranking. POST /v1/rerank takes a Jina/Cohere-shaped body and scores each document with an in-process System One Choice. The model id is ninfer-choice-rerank-v1, and return_documents defaults to true. Rerank only reads the prompt cache, so it never evicts cached conversations. See docs/serving.md.
• Web frontends in the Supervisor. Each configured static web app, such as llama.cpp's web UI, is served on its own local port with the engine API forwarded on that port. A browser chat needs no proxy, CORS setup or API key in the page, and responses stream. Add, change or remove frontends in the dashboard under Settings > Web frontends; saving applies right away while the engine keeps running. The tray and the dashboard link to each frontend. See docs/windows-app.md#web-frontends.
• Supervisor dashboard stays fast with a large request log. The Supervisor now reads the engine request log once in the background and then follows it, instead of re-reading the whole file for every dashboard refresh. With a 778 MB log, a dashboard state request went from about 27 s to about 14 ms, and the Supervisor's peak memory from about 7.3 GB to 34 MB. After a start it reads the existing log once (about 10 s per GB) and shows "Reading log NN%" meanwhile.
• llama.cpp web UI compatibility. GET /props, Chat Completions without model, max_tokens: -1, and thinking_budget_tokens.
• CUDA synchronization blocks instead of spinning. Decode speed is unchanged, and about one CPU core is freed while decoding.

Licenses

NInfer is Apache-2.0. Bundled third-party licenses are in the licenses folder of the installation. FFmpeg n9.0.2-3-ga5923073bf is included as dynamically linked LGPL-3.0-or-later DLLs, which you may replace. Its corresponding source is attached here as ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz. FFmpeg-SOURCE.txt names the build and the build recipe for the libraries linked into it.

Assets

• NInfer-2026.10.02-v3port.8-windows-x64-unsigned.exe: installer
• NInfer-2026.10.02-v3port.8-SHA256SUMS.txt: installer checksum
• release-manifest.json: source commit plus SHA-256 of every bundled file
• ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz, FFmpeg-SOURCE.txt: FFmpeg corresponding source

NInfer 2026.10.02-v3port.7 (Windows, Blackwell)

Alpha, unsigned. Windows 11 x64 installer for NVIDIA Blackwell sm_120 GPUs (RTX 5090, RTX PRO 6000 Blackwell). Built from e33dca1b on workstation.

Install

Run NInfer-2026.10.02-v3port.7-windows-x64-unsigned.exe, open NInfer from Start, and select a downloaded .ninfer model. Windows will warn about an unknown publisher because the installer is unsigned; check it against NInfer-2026.10.02-v3port.7-SHA256SUMS.txt:

13923f15f1318853095923d88472696a757cdaab9f75010aa4dbc85fb628ed7a  NInfer-2026.10.02-v3port.7-windows-x64-unsigned.exe

Needs a current NVIDIA driver with CUDA 13.3 support. No CUDA Toolkit, Visual Studio or Python is needed. Settings and logs live in %LOCALAPPDATA%\NInfer and survive updates and uninstall. Quit NInfer from its tray menu before updating.

Models

This release reads NInfer v3 artifacts only. Model weights are not included:

• Qwen3.8-27B NVFP4: https://huggingface.co/neroued/Qwen3.8-27B-nvfp4-NInfer
• Qwen3.8-27B Uncensored NVFP4: https://huggingface.co/igorls/Qwen3.8-27B-Uncensored-NVFP4-NInfer

A v2 .ninfer file from an earlier download does not load; the Supervisor's model list marks it. Download the current file, or upgrade it in place with tools/upgrade_ninfer_v2_to_v3.py from the repository. Qwen3.8-Flash-Next is not available in this release: its port to the v3 engine is in progress.

What's new since 2026.09.24-alpha.1

• v3 engine. The fork now builds on upstream NInfer's v3 engine and .ninfer v3 artifacts, merged through upstream d44ab584.
• Document reranking. POST /v1/rerank takes a Jina/Cohere-shaped body and scores each document with an in-process System One Choice. The model id is ninfer-choice-rerank-v1, and return_documents defaults to true. Rerank only reads the prompt cache, so it never evicts cached conversations. See docs/serving.md.
• Web frontends in the Supervisor. Each configured static web app, such as llama.cpp's web UI, is served on its own local port with the engine API forwarded on that port. A browser chat needs no proxy, CORS setup or API key in the page, and responses stream. The tray and the dashboard link to each frontend. See docs/windows-app.md#web-frontends.
• Supervisor dashboard stays fast with a large request log. The Supervisor now reads the engine request log once in the background and then follows it, instead of re-reading the whole file for every dashboard refresh. With a 778 MB log, a dashboard state request went from about 27 s to about 14 ms, and the Supervisor's peak memory from about 7.3 GB to 34 MB. After a start it reads the existing log once (about 10 s per GB) and shows "Reading log NN%" meanwhile.
• llama.cpp web UI compatibility. GET /props, Chat Completions without model, max_tokens: -1, and thinking_budget_tokens.
• CUDA synchronization blocks instead of spinning. Decode speed is unchanged, and about one CPU core is freed while decoding.

Licenses

NInfer is Apache-2.0. Bundled third-party licenses are in the licenses folder of the installation. FFmpeg n9.0.2-3-ga5923073bf is included as dynamically linked LGPL-3.0-or-later DLLs, which you may replace. Its corresponding source is attached here as ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz. FFmpeg-SOURCE.txt names the build and the build recipe for the libraries linked into it.

Assets

• NInfer-2026.10.02-v3port.7-windows-x64-unsigned.exe: installer
• NInfer-2026.10.02-v3port.7-SHA256SUMS.txt: installer checksum
• release-manifest.json: source commit plus SHA-256 of every bundled file
• ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz, FFmpeg-SOURCE.txt: FFmpeg corresponding source

NInfer 2026.10.02-v3port.6 (Windows, Blackwell)

Alpha, unsigned. Windows 11 x64 installer for NVIDIA Blackwell sm_120 GPUs (RTX 5090, RTX PRO 6000 Blackwell). Built from a53cf3e4 on workstation.

Install

Run NInfer-2026.10.02-v3port.6-windows-x64-unsigned.exe, open NInfer from Start, and select a downloaded .ninfer model. Windows will warn about an unknown publisher because the installer is unsigned; check it against NInfer-2026.10.02-v3port.6-SHA256SUMS.txt:

dc6518d47c1d27d459298dfe30a2a6d4e5502bd72c0f219ed149f7aa9a8ce67f  NInfer-2026.10.02-v3port.6-windows-x64-unsigned.exe

Needs a current NVIDIA driver with CUDA 13.3 support. No CUDA Toolkit, Visual Studio or Python is needed. Settings and logs live in %LOCALAPPDATA%\NInfer and survive updates and uninstall. Quit NInfer from its tray menu before updating.

Models

This release reads NInfer v3 artifacts only. Model weights are not included:

• Qwen3.8-27B NVFP4: https://huggingface.co/neroued/Qwen3.8-27B-nvfp4-NInfer
• Qwen3.8-27B Uncensored NVFP4: https://huggingface.co/igorls/Qwen3.8-27B-Uncensored-NVFP4-NInfer

A v2 .ninfer file from an earlier download does not load; the Supervisor's model list marks it. Download the current file, or upgrade it in place with tools/upgrade_ninfer_v2_to_v3.py from the repository. Qwen3.8-Flash-Next is not available in this release: its port to the v3 engine is in progress.

What's new since 2026.09.24-alpha.1

• v3 engine. The fork now builds on upstream NInfer's v3 engine and .ninfer v3 artifacts, merged through upstream d44ab584.
• Document reranking. POST /v1/rerank takes a Jina/Cohere-shaped body and scores each document with an in-process System One Choice. The model id is ninfer-choice-rerank-v1, and return_documents defaults to true. Rerank only reads the prompt cache, so it never evicts cached conversations. See docs/serving.md.
• Web frontends in the Supervisor. Each configured static web app, such as llama.cpp's web UI, is served on its own local port with the engine API forwarded on that port. A browser chat needs no proxy, CORS setup or API key in the page, and responses stream. The tray and the dashboard link to each frontend. See docs/windows-app.md#web-frontends.
• llama.cpp web UI compatibility. GET /props, Chat Completions without model, max_tokens: -1, and thinking_budget_tokens.
• CUDA synchronization blocks instead of spinning. Decode speed is unchanged, and about one CPU core is freed while decoding.

Licenses

NInfer is Apache-2.0. Bundled third-party licenses are in the licenses folder of the installation. FFmpeg n9.0.2-3-ga5923073bf is included as dynamically linked LGPL-3.0-or-later DLLs, which you may replace. Its corresponding source is attached here as ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz. FFmpeg-SOURCE.txt names the build and the build recipe for the libraries linked into it.

Assets

• NInfer-2026.10.02-v3port.6-windows-x64-unsigned.exe: installer
• NInfer-2026.10.02-v3port.6-SHA256SUMS.txt: installer checksum
• release-manifest.json: source commit plus SHA-256 of every bundled file
• ffmpeg-n9.0.2-3-ga5923073bf-source.tar.gz, FFmpeg-SOURCE.txt: FFmpeg corresponding source