Repository navigation
OMP NInfer v0.16.0 — native DFlash2 K7, RTX 3090 concurrency 2, and unified engine fixes
OMP NInfer v0.16.0 — native DFlash2 K7, RTX 3090 concurrency 2, and unified engine fixes
Prepared composition: fresh candidate route acceptance passed; product-tooling four-seat critical-review council (G4, review/runtime-ledger.json) binding outstanding; not published.
This is not a published product release. All three runtime components in v0.16.0
build from one frozen engine release-candidate source 26fa28d0e7834515508fc07ac503164a6effc820
(26fa28d0, referred to below as N16, tag rebase/ckpt-native-n16 in alphastorm/ninfer) on
upstream Neroued/ninfer master 64d10fbf9f8e6bfc79ab0570b94674c5b9997929: the native Windows
RTX 3090 lane (v0.8.0-qwen38-3090-beta.1, profile qwen38-3090-native-v0.8.0-beta.1) moves
to DFlash2 draft window 7 (K=7) with INT8 KV and concurrency 2 (C2) over one shared
131,072-token pool, the native Windows RTX 4090 lane (v0.8.0-qwen38-4090-beta.1, profile
qwen38-4090-native-v0.8.0-beta.1) moves to DFlash2 K=7 with INT8 KV and concurrency 1
(C1) at 131,072 tokens, and the RTX 5090 container lane (v0.7.0-qwen38-5090-beta.4)
rebuilds from the same source with runtime, checkpoint-export, io_uring, installer, and
observability fixes while retaining its qwen38-5090-v0.13.0 serving profile (with only
--binary-sha256 updated to the N16 server bab3fd77). Stock Oh My Pi
18.14.0 and the shared version 3 model artifact (81f924d4) carry unchanged from v0.15.0.
Because this release combines both native DFlash2 transitions, native C2 on the RTX 3090, and the
three-lane engine rebuild, v0.16.0 supersedes the v0.15.1 patch previously referenced in the
v0.15.0 release notes and roadmap.
The published product remains v0.15.0; use its
immutable installation guide
until publication. The prepared guide is bound to frozen candidate
56899320d416bafda5c15d7e454e9b593badc0a6;
it is not an installation recommendation before publication. Component publication, runtime
qualification, candidate-bound external installation, independent G4 review, and founder-owned
product publication are distinct gates; no predecessor hardware or client-route receipt is relabeled.
Owner-operated, exact-profile 0.x release; no SLA.
Manifest · Qualification ·
Compatibility · Glossary
Oh My Pi (coding-agent client) connects through OMP NInfer (qualified appliance and release layer)
to NInfer (inference engine) serving Qwen3.8 27B on one user-owned GPU.
Composition — one engine source (26fa28d0) across all three lanes
| Component | Exact identity | Serving boundary |
|---|---|---|
| Oh My Pi | Unmodified upstream v18.14.0, source 31147f9d1a76608549821c3759f8487b4d66c72d, tree 9a63b5bfbb5971962c44fca660744b0d75dcd2c2 |
Carried stock client (omp-darwin-arm64 60884395, omp-windows-x64.exe 620a5935, omp-linux-x64 e7dd2b60); fresh three-platform and five-route acceptance PASS, 31/31 blocks on 2026-11-01 (routes); macOS remains preview |
| RTX 5090 container | v0.7.0-qwen38-5090-beta.4 (v0.7.0-qwen38-5090-runtime-beta.4, v0.7.0-qwen38-5090-source.4); server bab3fd77, archive f3a35cdb, SPDX b09ad9a7, checksums cca65f0d, source archive 1ef87d52, image sha256:7a5a7bb88966512bcab2f711c1a7515bf0b8b834aaf0abd5456643493babb016 on base d4377ed0 |
Kept qwen38-5090-v0.13.0 (config 751d22dc, only --binary-sha256 changed from 55f0b95a to bab3fd77): BF16 KV (146,368 automatic tokens), 131,072 context, DFlash2 K7 (--lm-head-draft), two active requests (C2) + 2 extra device-state slots, 1,024-token prefill chunk, 180 s (180000 ms) pending timeout, 20,867.734375 MiB (21,881,405,440 B) shared Host budget, 28,672 MiB host RAM floor, 60,000 ms keep-warm, /checkpoints-v013 (24,576 MiB quota, 256 MiB staging, mandatory --session-checkpoint-require-origin-auth), vision + tools + JSON-schema output + preserved thinking; checked-first-build reproducibility exception remains specific to this lane |
| RTX 4090 native Windows | v0.8.0-qwen38-4090-beta.1; package 06526bdb (991,969,490 B), server c6f59e9e, public config f00cfac9, release-spec aab7a74d, SPDX 825a8d30, CycloneDX b8c8efcb, outer SHA256SUMS bf1650eb |
Profile qwen38-4090-native-v0.8.0-beta.1: INT8 KV, DFlash2 K7 (Q4_G64_FP16 draft MLP, host-mapped token embedding, proposal: "artifact-default"), one active request (C1) + 2 extra device-state slots, 131,072 context/KV, 1,024-token prefill chunk, 30 s (30000 ms) pending timeout, 15,747.734375 MiB (16,512,696,320 B) shared Host budget, 640 MiB admission headroom, 32,768 MiB host RAM floor, Group64 A8 and FP32 head logits off, 60,000 ms keep-warm, 450 W owner baseline, text + tools + JSON-schema output (no vision) |
| RTX 3090 native Windows | v0.8.0-qwen38-3090-beta.1; package 040d5923 (1,008,976,979 B), server 72cd776b, public config cbe67aae, release-spec 7d2406a3, SPDX 6f78e190, CycloneDX 3b217f69, outer SHA256SUMS 0fd18a35 |
Profile qwen38-3090-native-v0.8.0-beta.1: INT8 KV, DFlash2 K7 (Q4_G64_FP16 draft MLP, host-mapped token embedding, proposal: "artifact-default"), two active requests (C2) over one shared 131,072-token KV pool + 2 extra device-state slots, 1,024-token prefill chunk, 180 s (180000 ms) pending timeout, 20,867.734375 MiB (21,881,405,440 B) shared Host budget, 768 MiB admission headroom, Group64 A8 and FP32 head logits off, keep-warm off, 350 W serving cap (370 W restored on stop), text + tools + JSON-schema output (no vision) |
All three runtime lanes bind frozen source 26fa28d0e7834515508fc07ac503164a6effc820 on upstream
Neroued/ninfer master 64d10fbf9f8e6bfc79ab0570b94674c5b9997929 (short 8-hex identifiers are
SHA-256 prefixes for artifacts/configs or Git commit prefixes in alphastorm/ninfer unless
otherwise noted). The model remains the published
container version 3 artifact 81f924d440c27261d820c19a9f8d45794c5aee410f8a68bd358133fa8c0375da
(20,437,521,664 bytes, revision 1cbd84e7221e51186bd7f093a149912d2489625b, downloaded to
qwen3_8_27b-v3.ninfer). Native packages use CUDA 13.3 (nvcc 13.3.73, sm_89/sm_86) and MSVC
14.44.35207 (cl 19.44.35229); the RTX 5090 container uses CUDA 13.1 (nvcc 13.1.115, sm_120a)
and GCC 13.3.
What changes in v0.16.0
- Native DFlash2 K=7 on RTX 3090 and RTX 4090: Both native Windows lanes retain 8-bit
INT8KV
(33,792bytes/token) and a 131,072-token context ceiling while switching their speculative
decoding backend from MTP3 (K=3,proposal: "lm-head-draft"/lm_head_draft: true) to
DFlash2 draft window 7 (K=7, using the model artifact's bundled 5-layer block drafter with
proposal: "artifact-default"inrelease-spec.jsonand nolm_head_draftoverride in
server-config.json; the kept RTX 5090 CLI profile continues to pass--spec dflash2 --draft-tokens 7 --lm-head-draft). To fit 24 GB cards with the required admission headroom
(768 MiBon RTX 3090,640 MiBon RTX 4090), both native profiles use thesm_86/sm_89
load-time fit seam:Q4_G64_FP16conversion of the draft MLPgate/upweights (473,497,600
converted device bytes) and host-mappedtoken_embedding(1,350,860,800host-mapped bytes).
Each native DFlash2StateImageis195,897,344bytes (40 MiBlarger than MTP3's
153,954,304-byte image because it carries the local cyclic draft KV ring). On RTX 4090
(qwen38-4090-native-v0.8.0-beta.1), the prefill chunk changes from2,048to1,024
tokens, the shared Host budget updates from15,506,063,360bytes (14787.734375 MiB) to
16,512,696,320bytes (15747.734375 MiB, retaining11,264 MiBof Host KV plus 24 DFlash2
state images and the32,768 MiBhost RAM floor), and concurrency stays at one active request
(C1, 30 s pending timeout, two extra / three total device-state slots). - RTX 3090 concurrency 2 over one shared 131,072-token pool (
qwen38-3090-native-v0.8.0-beta.1):
The RTX 3090 native profile advances from one active request (C1, 30 s pending
timeout, four extra device-state slots,12,284,837,888-byte /11715.734375 MiBshared Host
budget) to two active requests (C2,engine.max_concurrency: 2,180,000 mspending timeout,
two extra / four total device-state slots) sharing one 131,072-tokenINT8KV pool, and raises
its Host KV portion from8,192to16,384 MiBplus 24 DFlash2 state images for a
21,881,405,440-byte (20867.734375 MiB) shared Host budget (matching the20867.734375 MiB
Host budget already used byqwen38-5090-v0.13.0on the RTX 5090). In
examples/manual-tunnel/fail-closed.yml,
providers.maxInFlightRequests.ninfer-native-3090changes from1to2(ninfer-native-4090
andninfer-heavystay1). Only one native lane runs at a time on shared loopback port18082;
C2means two concurrent requests sharing one 131,072-token KV pool inside the active RTX 3090
lane, not two independent full-size KV pools or simultaneous RTX 4090 and RTX 3090 serving. - First-fork export-State residency (
8c639fe0/80c0af77, EXP-105): Previously, an
off-worker durable checkpoint export heldReadLeasepins on deviceStateImageslots throughout
disk serialization, blocking the first post-export or post-resume fork until the write retired.
StateImageStore::prepare_host_exportnow materializes immutable state images into the counted
Host arena and releases device state replicas before off-worker disk serialization begins. On the
RTX 5090 (08257eec), post-resume first-fork server time to first token (TTFT) dropped from
5.03 s (5.026 swith4.850 ssource_waitonc3911a14;4.903 s/4.738 son
31583e3f) to 0.11 s (0.106 sTTFT,0.005 ssource_wait). On the RTX 3090 (08257eec),
first-fork server TTFT dropped from 15.4–17.8 s (14.86–17.22 ssource_wait) to
0.33–0.76 s (source_wait <= 0.033 s). - Shared-pool export-pin liveness and startup headroom (
ea2edc20,375f47c9, EXP-106):
EngineCore::reserve_resident_units()now waits whencheckpoints_.pending()or
checkpoint_exports_in_flight_ != 0can release pinned device KV pages instead of failing a
resumed concurrent resident with HTTP 500oldest resident cannot obtain its legal unit.
StartupMemoryBudget::fits()now enforces both physical free memory and configured-budget
headroom after CUDA graph initialization so startup replans or refuses rather than spending the
declared reserve. - Checkpoint-owner, delivery-ordering, and
io_uringlifetime fixes (c3911a14,e517dab8,
40a1b2d0):- Checkpoint owner replacement (
c3911a14andalphastorm/ninfer#49e517dab8): Fixes the
Native could not release an unreferenced checkpoint ownerfailure on fork-after-resume that
deferred native RTX 3090 DFlash2 out ofv0.15.0, retains binding waiters on failed state
adoption, orders error and cancellation response delivery after engine-state retirement
(#95), and adds media-decode range checks
(#96). - Linux
io_uringowned completion buffers and teardown barrier (40a1b2d0,
#116 handle-allocation and ring-teardown
lifetime itemsGA-R2/GA-U1; reaches the Linux RTX 5090 container lane, not Windows native
execution): Allocates the completion handle before publishing a checkpoint read batch to the
worker so a failed allocation cannot leave a worker writing retired caller buffers, and on fatal
ring reset retracts unconsumed submission queue entries (SQEs), issuesIORING_OP_ASYNC_CANCEL
per outstanding request, and drains both cancellation and original completion queue entries
(CQEs) — aborting fail-closed on a 30 s deadline if retirement cannot be proved — before
releasing memory.
- Checkpoint owner replacement (
- Native Windows installer and controller recovery (
577fecba,alphastorm/ninfer#489cb741dc,
#116): Hands off the GPU-owner lease
across running-lane upgrades and rollbacks (including power-cap transitions, closing
#94 and removing the historical stop-first
prerequisite), journals directory-creation cleanup intent beforemkdir, refuses a non-running
incumbent's retained inner GPU lease before cutover, publishes pre-spawn launch attribution with a
process-image census guard before GPU-owner restoration, and blocks destructive transaction
rollback when a launch-bound shutdown report is explicitly incomplete or records refused saves. - Per-position speculative decoding metrics (
f6668a5d, request-log schema 30): Prometheus
/metricsexposesninfer_spec_decode_position_tokens_total{position,kind}(attemptedand
acceptedacross zero-based draft positions, with startup warmup subtracted), andrequests.jsonl
emits additive optionalrequest_done.speculative.attempted_per_positionwhile retaining
request-log schema30. - Product tooling and residual closures (
3de36942,fa96e486,a376ffd7,
#117):scripts/verify_release.py,
scripts/bind_native_variant.py,scripts/compose_native_qualification.py, and
scripts/compose_route_acceptance.pyconsume authenticated packaged native policy
(release-spec.jsonschema 5 andserver-config.jsonschema 4), keep serial speed-gate campaigns
atC1while adding a separateC2acceptance arm (concurrency_probe.py --scenario fanout --fanout-concurrency 4, stock OMP subagent/session overlap at limit 2, and two
automatic-compaction/control-recall epochs), and close all 13v0.15.0product-council residual
items in #117 (P15-R01..P15-R13in
CHANGELOG.md), including predecessor-variant binding for native quality
screens, exact 1,120-screen / 80-determinism prompt membership seals, mandatory G4 ledger binding
in ready mode, recursiverepository_path/sha256verification, and normalized public.cast
files plus re-rendered GIF/MP4/poster demo assets (3,676OCR-inspected frames with zero
private-path markers).
Stock Oh My Pi 18.14.0 retained; 18.14.1 and 18.14.2 not adopted
v0.16.0 retains the unmodified upstream Oh My Pi 18.14.0 binaries (31147f9d, tree
9a63b5bf, rebuilt with Bun 1.4.4 for CVE-2026-91403 and CVE-2026-111497) pinned in
client-components.json. Upstream v18.14.1 and v18.14.2
(7efd7721, published 2026-10-31) are deliberately not adopted in this runtime release:
- Unchanged wire and compatibility contract: Across the 394 commits in
v18.14.0...v18.14.2,
zero commits touchedpackages/ai/src/providers/openai-responses.ts,
packages/ai/src/providers/openai-responses-state.ts, themodels.ymlschema bundle,
authHeader,compat.statefulResponses,compat.includeEncryptedReasoning,
compat.supportsReasoningSummary, or transport retry policy. - No security advisory: Neither
18.14.1nor18.14.2contains a security fix; the bundled
Bun 1.4.4 CVE fixes already shipped in18.14.0. - Deferred behavior changes:
18.14.2movesthinking-loop-guardto a catalog axis (exact
for Qwen with a 1,024-character floor on line-spanning cycles, removingcompat.thinkingLoopGuard)
and rejects local compaction summaries that end withstopReason: "length". Neither addresses an
observed defect on the qualified NInfer routes; adopting a new client binary will follow as a
separate client-patch candidate (v0.16.1, targeting18.14.2rather than18.14.1alone).
v0.16.0 experiment ledger and deferred or rejected candidates
Full narrative and historical context are in docs/PERFORMANCE.md
and ROADMAP.md.
| Entry | Recorded result | Release boundary |
|---|---|---|
| EXP-101 — RTX 3090 DFlash2 K7 powered quality screen | Preregistered 1,120-prompt screen and 80-prompt fresh-cache determinism run on v0.8.0-qwen38-3090-beta.1 (26fa28d0, package 040d5923, server 72cd776b, config cbe67aae, 350 W, port 18082, 50-prompt shards at 1 request) vs the sealed v0.14.0 MTP3 production arm (fedce425 / d08d262b, 25 sealed holds, state 9a9213cf): PASS (receipt, SHA-256 671659f0): 1,120 unique prompts (672 gate + 504 redaction pairs, overlapping on 56 control variants), both arms 80/80 deterministic with 0 missing/error samples, 771/1,120 paired content/reasoning outputs changed, and all 9/9 primary one-sided 95% bounds passed: recall lower -0.0073 and precision lower -0.0070 (margin > -0.02), unsupported upper +0.0089 (< +0.03), schema/tool-selection/tool-argument lower 0 (each > -0.02), duplicate upper +0.0059 (< +0.02), redaction lower -0.0119 (> -0.05), secret-leak ratio 1.0294 with upper bound 1.0673 (< 1.10, counts 545 → 561, not zero). |
Governing quality gate for the shipping RTX 3090 DFlash2 K7 C2 package. |
| EXP-102 — RTX 4090 DFlash2 K7 powered quality screen | Preregistered 1,120-prompt screen and 80-prompt fresh-cache determinism run on v0.8.0-qwen38-4090-beta.1 (26fa28d0, package 06526bdb, server c6f59e9e, config f00cfac9, 450 W, port 18082, 50-prompt shards at 1 request) vs the sealed v0.14.0 MTP3 production arm (5caa7780 / b5cfbf50, 25 sealed holds, state cc48a776): PASS (receipt, SHA-256 6231dccd): 1,120 unique prompts (672 gate + 504 redaction pairs, overlapping on 56 control variants), both arms 80/80 deterministic with 0 missing/error samples, 777/1,120 paired content/reasoning outputs changed, and all 9/9 primary one-sided 95% bounds passed: recall lower -0.0082 and precision lower -0.0093 (margin > -0.02), unsupported upper +0.0045 (< +0.03), schema/tool-selection/tool-argument lower 0 (each > -0.02), duplicate upper +0.0046 (< +0.02), redaction lower -0.0139 (> -0.05), secret-leak ratio 1.0525 with upper bound 1.0985 (< 1.10, counts 533 → 561, not zero). |
Governing quality gate for the shipping RTX 4090 DFlash2 K7 C1 package. |
EXP-103 — RTX 4090 K=6/K=5 draft windows and K=7 131K INT8 fit |
On v0.14.0 binary 5caa7780, config-only K=6 (0.8625–0.8642 cross ratios, median 0.8633) and K=5 (0.8668–0.8673, median 0.8671) failed the >= 0.95 floor on depth-0 case log-024@0 vs MTP3. DFlash2 K=7 131,072-token INT8 C1 fit passed (1,449,132,032 B available >= 671,088,640 B floor, 22,265.0 MiB peak, 32-token / 114-byte true-cold continuation 07b8db1f). |
K=6 and K=5 rejected; RTX 4090 uses K=7. |
EXP-104 — Opt-in FP32 head logits (--fp32-head-logits) and RTX 4090 floor |
--fp32-head-logits (+8,442,880 persistent bytes at K=7/C=2) passed 2/2 numerical tests on sm_86/sm_89 (83 FP32 reductions, max rel L2 0.001722), but fp32_head_logits: true failed log-024@0 on 31583e3f at 0.9016–0.9025 (median 0.9021). With --fp32-head-logits off (the default), DFlash2 K=7 passed log-024@0 at 0.9627–0.9635 (31583e3f) and 0.96186–0.96294 (08257eec); in the final 26fa28d0 shipping-package (06526bdb) 115-case ABBA speed gate at 131,072-token INT8 KV (+2 extra slots), the 108-case role corpus (role108) reached ×1.2736 (139.87 → 178.13 tok/s, all 115 cases ×1.2741) while log-024@0 measured 0.934 (139.41 → 130.19 tok/s) and was waived by the founder with disclosure. |
--fp32-head-logits rejected for shipping profiles and stays opt-in and off (false); default DFlash2 K=7 adopted on RTX 4090. |
EXP-105 — Export-State residency (prepare_host_export) |
Materializing immutable device StateImage replicas to Host before off-worker disk serialization cut first-fork source_wait on RTX 3090 (08257eec) from 14.86–17.22 s (15.4–17.8 s TTFT) to <= 0.033 s (0.33–0.76 s TTFT) and on RTX 5090 (08257eec) from 4.85 s (5.03 s TTFT) to 0.005 s (0.106 s TTFT). |
Shipped in 26fa28d0 across all three lanes. |
EXP-106 — Native C2 on RTX 3090 and RTX 4090 C2 refusal |
Waiting for in-flight exports in reserve_resident_units() (ea2edc20) resolved the oldest resident cannot obtain its legal unit HTTP 500. On RTX 3090 (08257eec), INT8 C2 over one shared 131,072-token pool passed startup (1,110,441,984 B available, 22,584 MiB peak), two-session 59,866-token true-cold durability, calibrated 59,879 + 59,879-token concurrent runs (18/18 HTTP 200 across 3 trials), and paired 2,048-token decode (1.385× median aggregate 57.68 → 79.90 tok/s, 0.722× pair wall 71.01 → 51.26 s; concurrent per-request 0.698× / 0.735× of C1). RTX 4090 INT8 C2 at 131,072 tokens refused startup admission. |
RTX 3090 ships C2 (qwen38-3090-native-v0.8.0-beta.1); RTX 4090 stays C1 (qwen38-4090-native-v0.8.0-beta.1). |
EXP-107 — Native rk4v4 4-bit KV (262,144-token context) |
Ported head-dimension-256 (D256) rk4v4 (17,920 B/token, 47.0% smaller than INT8) passed 7/7 FP64 op/graph/store executables (max rel L2 0.007773) and held-out perplexity on sm_86/sm_89 (ΔNLL = 0.00345 <= 0.02), and on RTX 3090 fit 262,144-token C1 (1,097,859,072 B available; exact 130,048/250,062-token retrieval in 197.86 / 503.57 s), 245,760 C2, and 229,376 C2. However, true-cold restore failed on all three RTX 3090 tuples (token 32 on 262,144 C1, cold 48117 vs restored 3635; tokens 16–32 on session 1 of 245,760/229,376 C2). RTX 4090 refused 262,144 C1 and 245,760 C2 at startup. |
Deferred to a follow-up release; --kv-dtype rk4v4 stays off and unqualified in all shipped profiles. |
Native Windows qualification at 26fa28d0
Both v0.8.0-qwen38-4090-beta.1 (06526bdb) and v0.8.0-qwen38-3090-beta.1 (040d5923) are
built from frozen commit 26fa28d0e7834515508fc07ac503164a6effc820 with separate closed 11-file
outer distribution sets (10 checksummed entries plus SHA256SUMS), packaged schema 5
release-spec.json and schema 4 server-config.json files, SPDX 2.3 file inventories, and
canonical CycloneDX 1.5 dependency inventories (12 resolved vcpkg packages with 13 DLL associations,
CUDA 13.3 runtime, vendored sources, MSVC CRT, and DirectStorage redistributables). In two clean
compile rounds per architecture on the Windows 11 build host, all 43/43 EXE/DLL binaries matched
raw bytes (17 EXEs and 26 DLLs per architecture), raw-runtime and package private-marker scans
reported 0 findings, and component cut preflight exited 0. Each outer distribution's immutable
package-build-receipt.json (b0f94a57 on RTX 4090, 9e194b8f on RTX 3090) retains its pre-hardware
hardware-pending status; subsequent hardware qualification receipts are distinct and copied
unchanged.
Pre-cut and shipping-package native gates
Gate at 26fa28d0 |
RTX 4090 (v0.8.0-qwen38-4090-beta.1, C1) |
RTX 3090 (v0.8.0-qwen38-3090-beta.1, C2) |
|---|---|---|
| Deterministic 2-round build & scans | 43/43 EXE/DLL byte-identical; 0 private markers | 43/43 EXE/DLL byte-identical; 0 private markers |
| Shipping-package 131,072-token INT8 fit & true-cold durability | PASS (receipt, 44c50008: 1,056,964,608 B / 1,008 MiB available >= 671,088,640 B / 640 MiB floor; 22,639 MiB peak, 1,504 MiB min free; 473,497,600 B converted device + 1,350,860,800 B host-mapped weights; cold 3,093 vs restored 3,066+27 32/32 token IDs & UTF-8 bytes identical; exact 130,048-token retrieval) |
PASS (receipt, 662619d7: 1,110,441,984 B / 1,059 MiB available >= 805,306,368 B / 768 MiB floor; 22,584 MiB peak, 1,742 MiB min free; 473,497,600 B converted device + 1,350,860,800 B host-mapped weights; two 59,866-token true-cold sessions restored at 59,840 cached + 26 computed tokens with 32/32 token IDs & UTF-8 bytes identical; exact 130,048-token retrieval) |
Role-corpus ABBA speed gate vs v0.14.0 MTP3 |
PASS with founder waiver on log-024@0 (speed 32acdecc, waiver 92b94c4a): 108-case role corpus (role108: 84 cases at 0 cached tokens + 12 at 32,000 + 12 at 64,000) weighted ×1.2736 (139.87 → 178.13 tok/s), all 115 cases (108 role + 7 code/reasoning supplementals) weighted ×1.2741 (138.94 → 177.02 tok/s), 7 supplementals weighted ×1.2778 (131.68 → 168.26 tok/s, min ×1.0804 on reasoning-invariant); 114/115 cases clear the >= 0.95 per-case decode ratio floor (lowest passing case 1.0128×), while depth-0 case log-024@0 measured 0.934 (139.41 → 130.19 tok/s, 0.933832×) below 0.95 and was waived by the founder with disclosure; retained content/reasoning/tool-call UTF-8 output bytes are 115/115 identical across within-engine repeats (A0=A3, B1=B2) and 43/115 identical across engines (31/84 at depth 0, 7/12 at 32K, 5/12 at 64K, 0/7 supplemental) |
PASS (receipt, c904d341): 84 depth-0 ABBA cases at 350 W vs v0.14.0 MTP3 (fedce425 / d08d262b); weighted ×1.2684 (1.268360×, 79.37 → 100.67 tok/s), minimum 0.9764× (0.976442× on log-024@0, 76.77/76.68 → 74.86/74.97 tok/s) >= 0.95 with no waived case; within-engine repeats 84/84 byte-identical in both arms (A0=A3, B1=B2) and cross-engine 36/84 byte-identical |
| Powered 1,120-prompt quality screen (EXP-102 / EXP-101) | PASS (receipt, 6231dccd): 1,120 unique prompts (672 gate + 504 redaction pairs, overlapping on 56 control variants); both arms 80/80 deterministic, 0 missing/error samples; 777/1,120 paired content/reasoning outputs changed; 9/9 primary bounds passed (recall lower -0.0082, precision lower -0.0093 vs -0.02; unsupported upper +0.0045 vs +0.03; schema/tool_selection/tool_argument lower 0 vs -0.02; duplicate upper +0.0046 vs +0.02; redaction lower -0.0139 vs -0.05; secret_leak ratio 1.0525, upper 1.0985 < 1.10, counts 533 → 561, not zero) |
PASS (receipt, 671659f0): 1,120 unique prompts (672 gate + 504 redaction pairs, overlapping on 56 control variants); both arms 80/80 deterministic, 0 missing/error samples; 771/1,120 paired content/reasoning outputs changed; 9/9 primary bounds passed (recall lower -0.0073, precision lower -0.0070 vs -0.02; unsupported upper +0.0089 vs +0.03; schema/tool_selection/tool_argument lower 0 vs -0.02; duplicate upper +0.0059 vs +0.02; redaction lower -0.0119 vs -0.05; secret_leak ratio 1.0294, upper 1.0673 < 1.10, counts 545 → 561, not zero) |
Agent-probe parity vs v0.14.0 (p16-agent-parity/1 / /2: all probe oracles pass with 0 errors/timeouts, per-position cached - prompt >= incumbent, fork/resume server TTFT <= incumbent + max(0.25 s, 25%)) |
PASS 83/83 positions (warm_arrival 12/12 both orders, agent_mix 51/51, sibling_continue 5/5, two-restart restore 4/4, fleet_probe fanout 11/11 across two 4-branch fanout rounds; min (cached - prompt) delta 0, min TTFT bound margin 0.219 s, max source_wait <= 0.0178 s) |
PASS 83/83 positions (warm_arrival 12/12 both orders, agent_mix 51/51, sibling_continue 5/5, two-restart restore 4/4, fleet_probe fanout 11/11 under p16-agent-parity/2; all (cached - prompt) deltas 0, min TTFT bound margin 0.169 s; C2 concurrency_probe --fanout-concurrency 4 5/5 HTTP 200; calibrated 2 × 59,879-token concurrent test 18/18 HTTP 200 across 3 trials, 0 HTTP 500; summary 8218925c) |
Upgrade, rollback, and repair rehearsal from v0.14.0 v0.7.0-qwen38-{4090,3090}-beta.3 |
PASS 3/3 scenarios from v0.7.0-qwen38-4090-beta.3 (87038cfb, server 5caa7780, config b5cfbf50 → 06526bdb, summary 38a3a380): (1c) running-lane upgrade (211.93 s, incumbent leases retired on managed Stop, candidate healthy with previous_release preserved), (2c) forced post-activation first-Start port-ownership refusal with rollback (211.37 s, 4/4 transaction-created release/secret/cache/stage targets deleted, incumbent restored healthy), and (3c) interrupted install at candidate_activated before task registration followed by -RepairInterruptedInstall (4/4 targets deleted) and same-instance reinstall (309.80 s); canonical states unchanged, 450 W restored; no product-route acceptance is inferred |
PASS 3/3 scenarios from v0.7.0-qwen38-3090-beta.3 (a7637f66, server fedce425, config d08d262b → 040d5923, summary 631428b5): (1c) running-lane upgrade at 350 W (220.32 s, incumbent leases retired on managed Stop, candidate healthy with previous_release preserved), (2c) forced post-activation first-Start port-ownership refusal with rollback (206.96 s, 4/4 transaction-created targets deleted, incumbent restored healthy), and (3c) interrupted install at candidate_activated followed by -RepairInterruptedInstall (4/4 targets deleted) and same-instance reinstall (322.25 s); canonical states unchanged, 370 W restored; no product-route acceptance is inferred |
Managed 15-phase native qualification
Each native package runs the 15-phase hardware qualification chain (tools/qualification/qualify_native.py in alphastorm/ninfer: (1) preflight, (2) neutral_build, (3) private_path_scan, (4) package, (5) transfer_install, (6) normal protocol and (7) pool-pressure pressure_protocol authenticated schema 2 smokes, (8) context_128k exact 130,048-token retrieval, (9) restart explicit-save and graceful-stop process restart, (10) rollback, (11) security state-root ACL regression, (12) omp stock client end-to-end turn, (13) benchmark_c1 at the qualified power cap, (14) receipt binding, and (15) restore owner power restoration); the table below summarizes the primary measured phase outputs in rtx4090.json (69ebf60c, qualified 2026-10-31T21:22:21Z) and rtx3090.json (public sanitized 48c7f635, raw 038fe713, qualified 2026-11-01T05:56:27Z):
Managed scratch qualification at 26fa28d0 |
RTX 4090 (rtx4090.json) |
RTX 3090 (rtx3090.json) |
|---|---|---|
| C1 decode, 4,541 prompt / 1,024 completion tokens (DFlash2 K7) | 254.27 tok/s (83.88% DFlash2 acceptance, 6.21 s wall, 22,639 MiB peak, 460.2 W peak) |
148.43 tok/s (83.88% DFlash2 acceptance, 11.69 s wall, 22,584 MiB peak, 350.5 W peak) |
| C1 prefill | 2,139.14 tok/s | 974.15 tok/s |
| Exact 130,048-token retrieval | 83.35 s (83.352 s, exact ORCHID=493817; COLOR=COBALT) |
188.99 s (188.994 s, exact ORCHID=493817; COLOR=COBALT) |
| Explicit save / graceful-stop cached tokens | 1,647 / 1,650 (643,966,448 B save; 6.315 s stop-event flush) |
1,647 / 1,650 (643,966,448 B save; 7.34 s stop-event flush) |
| Process restart | 42.21 s (42.213 s, 10 files / 643,998,587 B) |
46.88 s (46.882 s, 10 files / 643,998,585 B) |
| Normal / pressure schema 2 protocol & state security | passed / passed (12,758.578125 MiB pressure Host; 910 ms populated-root status, DACL protected) |
passed / passed (17,878.578125 MiB pressure Host at 370 W, C2 tool override 317b98ee; 898 ms populated-root status, DACL protected) |
| Rollback directions & owner power restored | not_applicable (0 directions, isolated scratch lineage) / 450 W (0 processes, lease inactive, task Ready) |
not_applicable (0 directions, isolated scratch lineage) / 370 W (0 processes, lease inactive, task Ready) |
Public native packages retain port 18082; installed scratch qualification equivalents change only
listen.port (18210 and pressure 18200 on RTX 4090, yielding scratch config 418687a4 and
scratch package 1f23e5d2; 18211 and pressure 18201 on RTX 3090, yielding scratch config
d114ddbb and scratch package c9624222) and do not execute or
activate the canonical public route. On RTX
4090, two pre-GPU setup attempts were infrastructure-invalid (the initial CPU hold lacked a private
host-lock environment marker before any phase ran, and the initial package step omitted the
referenced installed/ license/SPDX input tree) and each was corrected once in a fresh output root
without repeating any passed phase or GPU workload. On RTX 3090, because the frozen 26fa28d0
tools/smoke/agent_protocol.py helper asserted max_concurrency == 1 in --contract product, the
normal protocol and pool-pressure pressure_protocol phases ran with the founder-approved
agent_protocol.py C2 session-isolation precondition correction (SHA-256
317b98ee0fe110dbefd59b50a4508c2a27da5e7b457f2b819ad98ecd48868913, qualify_native.py wrapper
c06ad19eb7c776cb574734a4bc2d043740574bc11f752b3a04c7d5c1fa0b2ab1, open fork PR
alphastorm/ninfer#56 at 10a258bb), relaxing only
the two max_concurrency == 1 equality predicates to int >= 1 while keeping all quiescence and
request-correlated committed_decode_tokens delta checks unchanged; the committed public receipt
rtx3090.json (48c7f635) is a documented privacy-sanitized copy of
raw receipt 038fe713. The isolated 15-phase chain installs into a
fresh scratch root with no predecessor (rollback: not_applicable, 0 directions), so predecessor
upgrade/rollback against v0.7.0-beta.3 is proven separately by the upgrade rehearsal and
public-install promotion gates. Both v0.8.0-beta.1 native packages bind their packaged
release-spec.json and server-config.json in releases/v0.16.0/profiles/rtx{4090,3090}/.
RTX 5090 exact bytes, reproducibility exception, and lane gates
Server bab3fd77e8b5cafb792d0e4648e5e2b0d935105557a0b08fb2541799a25ffce7 (bab3fd77,
326,828,656 B), binary archive
f3a35cdb7859a576fa2ff7b474dfbd7ce5db37c7c756094cfd736d47fd284de0 (579,765,099 B), SPDX
b09ad9a7d06607f25600349f9e3aec751aaa5fe7387d9a756270abbcda47f7d7 (3,596 B), checksums
cca65f0d3d3b3f0da14ebce077dbc7e31a2402059c86e6f5e86238a2304eada6 (251 B), and source archive
1ef87d525976b770ae3ba9794587750bd2ba926dccc64e86d7cc2081d2674737 (12,656,433 B, release
v0.7.0-qwen38-5090-source.4) are the checked first build at 26fa28d0 (v0.7.0-qwen38-5090-beta.4).
The published runtime image is
ghcr.io/alphastorm/ninfer-runtime@sha256:7a5a7bb88966512bcab2f711c1a7515bf0b8b834aaf0abd5456643493babb016
(receipt release v0.7.0-qwen38-5090-runtime-beta.4). Runtime dependency base d4377ed0 is reused
through runtime-base-04b603aa-20261025 (original source 04b603aa, ancestor of 26fa28d0;
Dockerfile and runtime-image publication workflows are byte-identical to v0.14.0). The kept
strict-origin deployment profile is qwen38-5090-v0.13.0, configuration
751d22dc405595c17ef129b9b57d3582908567e47e6aa39507c02b9b9da82a47.
The exact shipping bytes (bab3fd77) passed all 11/11 exact-source RC phases (639.894 s, zero
failures or skips): (1) host (66/66 host tests), (2) dense_real (11/11 dense real-model
tests), (3) loader (1/1), (4) shared_mtp_fp8 (1/1), (5) overlap (1/1 32 GB C2
durable-pressure overlap), (6) four_boot_restart (four-boot schema 3 restart), (7) product_shape,
(8) exact_release_profile (exact release profile startup), (9) product_smokes
(serve_contract.py and agent_protocol.py --contract product), (10) help_candidate, and
(11) help_control (60 flags vs v0.7.0-qwen38-5090-beta.3's 59, adding opt-in
--fp32-head-logits, default off). Startup KV capacity is unchanged across all three RC profile
starts at 146,368 automatic KV tokens (1,074,790,400 B available against 1,073,741,824 B
reserve, 1,048,576 B margin) and 21,881,405,440 Host context bytes. In the isolated 84-role
ABBA T1 output-parity and serial decode comparison against v0.14.0
v0.7.0-qwen38-5090-beta.3 (2c6f423b / image 760fb6be),
84/84 request SHA-256 hashes and 84/84 output SHA-256 hashes matched across all four arms
(336/336 HTTP 200), with median decode 347.20 → 346.95 tok/s (ratio ×0.999282, per-case
0.9963–1.0019, passing the >= 0.99 fixes-only parity bar).
Published-image lane qualification (qualify_rtx5090_profile.py on digest 7a5a7bb8) and the full
N16 agent-probe campaign against the v0.14.0 incumbent (v0.7.0-qwen38-5090-beta.3, image
760fb6be, server 55f0b95a):
- Published-image lane receipt (
releases/v0.16.0/qualification/rtx5090.json, SHA-2562200fa44,2026-11-01T00:15:14Z):
PASS (0transient retries). Anonymous OCI digest-chain verification and an isolated
empty-DOCKER_CONFIGpull ofghcr.io/alphastorm/ninfer-runtime@sha256:7a5a7bb88966512bcab2f711c1a7515bf0b8b834aaf0abd5456643493babb016
passed, and the/usr/local/bin/ninfer-servebinary extracted from the image matched
bab3fd77e8b5cafb792d0e4648e5e2b0d935105557a0b08fb2541799a25ffce7. Startup reported BF16 KV with
automatic capacity 146,368 tokens (131,072max context, DFlash2K=7, optimized
--lm-head-draftproposal head) and 30,398 MiB idle/final VRAM. Coldprefill_curveat
130,048 prompt tokens (17completion tokens,0prefix-cache hits) returned the exact
plantedORCHID=493817; COLOR=COBALTneedle in 56.62 s wall (56.617 s, 2,297.0 tok/s
wall, 56.51 s /56.505 sserver prefill,30,400 MiBpost-gate VRAM). Single-request
decode(118prompt / 2,048 completion tokens) measured 181.78 tok/s server decode
(11.267 sserver) and 180.25 tok/s wall (11.362 swall) at 0.244 DFlash2 acceptance
(1,291 / 5,281drafted tokens across756verify rounds,0non-speculative fallback steps,
per-position accepted draft tokens[510, 312, 190, 118, 75, 49, 37]across draft positions
1..7) and 2.71 tokens/round. Theagent_protocolgate passed (authenticated session
identity, stateful continuation,2forks, parent delete200, deleted continuation404
previous_response_not_foundbefore and after Docker restart with no resurrection, surviving
descendant continuation, and live sibling continuation200). - N16 agent-probe campaign vs
v0.14.0incumbent (p16-agent-parity/2):
Every probe oracle and exact planted-key retrieval passed across both published images with0
errors, timeouts, or retries and0(cached - prompt)deficit on every paired position.
warm_arrivalpassed 12/12 positions across both orders (resume-firstandfork-first, min
TTFT bound margin0.084 s; onresume-first/fork_after_resume, server TTFT dropped from
5.159 s with4.997 ssource_waiton thev0.14.0incumbent to 0.278 s with0.000 s
source_waitonbeta.4, andfork-first/fork_after_resumemeasured0.119 → 0.113 s),
agent_mixpassed 51/51 (min TTFT margin0.249 s),sibling_continuepassed 5/5 (min
TTFT margin0.250 s), andfleet_probefanoutpassed 11/11 (post_restart_branch_0server
TTFT dropped from 5.437 s with5.248 ssource_waiton the incumbent to 0.300 s with
0.000 ssource_waitonbeta.4; min TTFT margin0.243 s). Four-way concurrent fanout
(concurrency_probe.py --scenario fanout --fanout-concurrency 4) passed 5/5 positions under
founder-approved parity rulep16-agent-parity/2, a judge correction that pairs
simultaneously submitted concurrent branches by server admission order
(request_start.timestamp_unix_ms) while keeping label pairing for sequential positions: under
C2(two active slots), two of the four concurrent branches are admitted immediately (0.232 vs 0.234 srank 1 and0.286 vs 0.288 srank 2,29computed tokens from the private endpoint) and
two wait for a slot (1.003 vs 1.001 srank 3 and1.165 vs 1.188 srank 4,23computed tokens
from the long anchor), whereas client-label pairing in/1read aC2admission-order swap
betweenbranch-1andbranch-3as failures. In two-restartrestore, two post-restart
positions exceeded the per-position TTFT boundincumbent + max(0.25 s, 25%)(0.526 scandidate
vs0.241 sincumbent,0.491 sbound, atrestore_0in the initial run;0.520 scandidate vs
0.242 sincumbent,0.492 sbound, atrestore_1in run 0 of the corrected three-run capture,
while runs 1 and 2 passed4/4positions each) and were waived by the founder with
disclosure (rtx5090-restore-waiver.json, SHA-256
66ba8104): the first request after a server restart sometimes takes~0.5 s(0.39–0.53 s)
instead of~0.24 s(0.21–0.24 s) in both engines, and thebeta.4runtime hits the slow case
somewhat more often (4/8post-restartrestore_0/restore_1requests across four two-restart
runs vs2/8on the incumbent). Both qualification runs restored the productionv0.14.0
container (image760fb6be), external health HTTP200,30,398 MiBGPU memory, and healthy
open fleet admission within the window cap (1,904.680 sinitial run,1,243.615 scorrected
run; an earlier release-candidate probe run was invalidated after6/12warm_arrivalrequests
when unpruned WSL2 build trees filled the host backing drive, after which VHDX compaction
reclaimed282.8 GBand added a 150 GiB host-free-disk preflight floor).
Founder-approved 5090-only reproducibility exception: As in v0.14.0, two clean fixed-path WSL2
builds of 26fa28d0 (nvcc 13.1.115, GCC 13.3, sm_120a) are not bit-identical: across 1,242
captured ELF files (56,881,590,888 bytes per set), 740 matched and 502 differed (665/962
object files and 75/280 other ELF files matched; all three application binaries differed). The
founder-approved reproducibility exception applies to the RTX 5090 container binaries, which ship the
exact preserved first-build bab3fd77 ELF set that passed the 11-phase RC and 84/84 T1 comparison
above. Native Windows sm_86/sm_89 shipping closures independently passed 43/43 second-build
EXE/DLL byte comparisons.
Profiles, state, upgrade notes, and known issues
Profiles and checkpoint stores
RTX 4090 uses qwen38-4090-native-v0.8.0-beta.1 (public config f00cfac9, 131,072-token INT8 KV,
DFlash2 K7, C1, 30 s pending timeout, two extra device-state slots, 1,024-token prefill chunk,
16,512,696,320-byte / 15747.734375 MiB shared Host budget, 640 MiB admission headroom, 32,768 MiB
host RAM floor, 60,000 ms keep-warm, 450 W owner baseline). RTX 3090 uses
qwen38-3090-native-v0.8.0-beta.1 (public config cbe67aae, shared 131,072-token INT8 KV, DFlash2
K7, C2, 180 s pending timeout, two extra device-state slots, 1,024-token prefill chunk,
21,881,405,440-byte / 20867.734375 MiB shared Host budget, 768 MiB admission headroom, keep-warm
off, 350 W serving cap with 370 W restored on stop). RTX 5090 keeps qwen38-5090-v0.13.0 (config
751d22dc, 146,368 automatic BF16 KV tokens, 131,072 context, DFlash2 K7 with --lm-head-draft,
C2, 180 s pending timeout, two extra device-state slots, 1,024-token prefill chunk,
21,881,405,440-byte / 20867.734375 MiB shared Host budget, 28,672 MiB host RAM floor, 60,000 ms
keep-warm).
Native stores use cache/<release-instance-id>/session-checkpoints-v3 under each immutable component
instance (qwen38-4090-native-v0.8.0-beta.1-26fa28d0e783 and
qwen38-3090-native-v0.8.0-beta.1-26fa28d0e783); the container retains /checkpoints-v013.
MTP3 durable sessions on either native lane (and prior v0.7.0-qwen38-5090-beta.3 container checkpoints) do not
carry across the v0.16.0 runtime and backend upgrade. Keep prior checkpoint roots untouched for
rollback; never import generations across different runtime binaries, speculative backends, or
profiles. Resumed OMP sessions replay the authoritative local transcript for a fresh prefill. Strict
checkpoint origin authentication (--session-checkpoint-require-origin-auth) remains mandatory.
Within one API-key principal, prompt_cache_key names a session capability; use unique high-entropy
keys and do not share the principal among mutually untrusted clients. Saves remain best effort, and
Windows NTFS ntfs-write-through is not POSIX fsync or a power-loss guarantee.
Upgrade notes
- Running native lanes upgrade directly (the
v0.14.0RTX 3090 stop-before-upgrade prerequisite
is dropped): Withalphastorm/ninfer#48(9cb741dc) and the P16 installer handoff fixes
(577fecba) inv0.8.0-qwen38-{3090,4090}-beta.1,Install-Release.ps1stops a running
incumbent lane through its own controller and hands off the GPU-owner lease across equal or
changed power caps, retaining thev0.7.0-qwen38-{3090,4090}-beta.3release asprevious_releasefor
Control-Release.ps1 -Action Rollback. Operators no longer need to manually stop a running RTX
3090 lane before upgrading. - Update
fail-closed.ymlfor RTX 3090 concurrency 2: When upgrading an existing
~/.omp/agent/config.yml, updateproviders.maxInFlightRequests.ninfer-native-3090from1to
2to matchexamples/manual-tunnel/fail-closed.yml
(ninfer-beta: 2,ninfer-main: 2,ninfer-native-3090: 2,ninfer-native-4090: 1,
ninfer-heavy: 1). Keepninfer-native-3090: 2only againstv0.8.0-qwen38-3090-beta.1or newer
(max_concurrency: 2,180 spending timeout).
Known issues and operating trade-offs
-
RTX 5090 first-request latency after a server restart (founder waiver on two-restart
restore):
In bothv0.14.0(v0.7.0-qwen38-5090-beta.3) andv0.16.0(v0.7.0-qwen38-5090-beta.4), the
first request against a checkpointed session after a server restart sometimes takes~0.5 s
(0.39–0.53 sserver TTFT) instead of~0.24 s(0.21–0.24 sserver TTFT), and thebeta.4
runtime hits the slow case somewhat more often (4/8post-restartrestore_0/restore_1
requests across four two-restart runs vs2/8on thev0.14.0incumbent). Two positions in the
two-restartrestoreprobe exceeded the per-position TTFT parity boundincumbent + max(0.25 s, 25%)(0.526 scandidate vs0.241 sincumbent,0.491 sbound, atrestore_0;0.520 s
candidate vs0.242 sincumbent,0.492 sbound, atrestore_1, while all probe checks and
exact planted-key retrievals passed) and were waived by the founder with this disclosure
(rtx5090-restore-waiver.json, SHA-25666ba8104). -
RTX 4090 single-case decode floor waiver on
log-024@0(0.934vsv0.14.0MTP3): Across
the four 4-arm ABBA speed campaigns (16arms total at 450 W) onv0.8.0-qwen38-4090-beta.1
(06526bdb,rtx4090-speed.json32acdecc), DFlash2K=7
raised weighted decode throughput to ×1.2736 on the 108-case role
corpus (role108:84cases at0cached tokens +12at32,000+12at64,000,139.87 → 178.13 tok/s), ×1.2741 across all 115 cases (138.94 → 177.02 tok/s), and ×1.2778 on
the 7 code/reasoning supplementals (min ×1.0804), with114/115cases clearing the>= 0.95
per-case decode ratio floor (lowest passing case1.0128×) and 115/115 within-engine repeats
(43/115 cross-engine) byte-identical. Depth-0 caselog-024@0decoded at 0.934 (139.41 → 130.19 tok/s,0.933832×), below the0.95every-case floor (291DFlash2 rounds with732 / 2,032accepted drafts vs306MTP3 rounds with717 / 918accepted drafts), and was waived by
the founder with this disclosure
(rtx4090-speed-waiver.json, SHA-25692b94c4a). -
RTX 3090
C2shared-pool capacity and per-request decode trade-off: The two concurrent
requests onv0.8.0-qwen38-3090-beta.1share one 131,072-tokenINT8KV pool
(kv_capacity: 131072), not 131,072 resident tokens per request (two ~60K-token requests fit
concurrently in device KV; two 120K-token requests exceed device KV and require host spill or
replay). While two 2,048-token decode requests overlap, each individual request decodes at
0.70–0.73× of single-requestC1speed (40.18and42.81 tok/svs57.58and
58.26 tok/s, ratios0.6979×and0.7349×), while combined throughput rises to 1.385×
(79.90vs57.68 tok/s) and two-request pair completion wall drops to 0.722× (51.26vs
71.01 s; solo decode on theC2profile is unchanged at57.94vs58.30 tok/s,0.9938×). -
Deferred fork-council residuals (
alphastorm/ninfer
#50–#54):
The26fa28d0G4 fork council (CR-20261031T162716Z-26fa28d0) closed in round 1 with zero
in-scope P0/P1 residuals after lead controls; its six merged disposition rows (L1–L6in
review/runtime-ledger.json) are deferred to the next runtime release across fivealphastorm/ninfer
bug issues (#50tracks bothL1andL2;#51–#54trackL3–L6):- alphastorm/ninfer#50 (P2, L1 + L2) —
Unreported or forced-stop shutdown during an unfinished install rollback: In
Control-Release.ps1,Stop-ManagedProcessthrows only when a launch-bound shutdown report
explicitly statescomplete: falseorrefused > 0. If the report is missing or unreadable
(graceful_unreported), ifStopfalls back to forced process termination
(graceful_wait_expired,graceful_nonzero_exit,terminated,signal_failed), or if a
later cleanStart/Stopclears a prior refused-save stop,Stopreturns success. Because
controllerRollbackonly swaps release pointers without deleting files and destructive
cleanup inInstall-Release.ps1is confined to an unfinished install transaction's own
candidate directories, the only reachable loss is the candidate's recomputable checkpoint cache
inside an install that never completed; the incumbent's checkpoints and the OMP client
transcript are never deleted. - alphastorm/ninfer#51 (P3, L3) — Installer
repair quiescence census skips blankExecutablePath: DuringInstall-Release.ps1 -Repair,
when state no longer names the candidate release, the process census filters by readable
ExecutablePathand drops entries whose path is blank. Windows executable file locks and
port-ownership checks still cause cleanup and incumbent restart to fail visibly if a candidate
server process is still alive. - alphastorm/ninfer#52 (P3, L4) — Checkpoint
export retirement under host memory exhaustion: Inengine_checkpoint.inl, if allocating the
small retirement command node throwsstd::bad_allocduring process-wide host OOM, the
in-flight export counter is not decremented inline; the server is already failing under host
memory exhaustion and the controller's stop timeout terminates the process. - alphastorm/ninfer#53 (P3, L5) — Opt-in FP32
head-logit flag omitted fromengine_identity():engine.cppomitsfp32_head_logitsfrom
engine_identity()whilesession_checkpoint_service.cppincludes it, so enabling
--fp32-head-logitswould cause checkpoint restore to fail closed with a fingerprint mismatch
and re-prefill from the transcript. Every shipped profile keepsfp32_head_logitsoff
(false), and packaged configs are hash-asserted at startup. - alphastorm/ninfer#54 (P3, L6) — Uninstall
power-limit record whenprior_pausedis true: RunningUninstallremoves the inner
GPU-owner lease record while the outer lease hasprior_paused=true, losing the recorded
pre-serving power limit and leaving the card at its safe qualified cap (350 Won RTX 3090);
operators can restore the default limit after uninstall withnvidia-smi -i <gpu> -pl 370.
- alphastorm/ninfer#50 (P2, L1 + L2) —
-
RTX 3090 15-phase
agent_protocolC2precondition override and sanitized public receipt
(alphastorm/ninfer#56): The RTX 3090 normal
(protocol) and pool-pressure (pressure_protocol, at the370 Wbaseline restored by the
preceding managedStop) 15-phase qualification smokes used the founder-approvedC2
session-isolation
precondition correction (agent_protocol.pySHA-256
317b98ee0fe110dbefd59b50a4508c2a27da5e7b457f2b819ad98ecd48868913,qualify_native.pywrapper
c06ad19eb7c776cb574734a4bc2d043740574bc11f752b3a04c7d5c1fa0b2ab1, fork PR
alphastorm/ninfer#5610a258bb); quiescence and
request-correlatedcommitted_decode_tokensdelta checks were unchanged. Its raw lane receipt
SHA-256038fe7132d3188ec4b1259c2933f3dacb23ec066345e723ce1b9baad873fef53remains private; the
committed public receiptrtx3090.json(SHA-256
48c7f63572e38b279dcc941c9f1a2acbf858ec95f1ffe3f0eb8bbc20a8111702) removes the 15 host-lock
resource names and replaces both qualification-tool path fields with the basename
agent_protocol.py, leaving all other JSON values, tool reason/hash, phase outcomes, thresholds,
and identities unchanged.
Fresh candidate acceptance and G4 review
Fresh acceptance passed on frozen candidate
56899320d416bafda5c15d7e454e9b593badc0a6
on 2026-11-01, completed at 2026-11-01T12:30:58Z.
Runtime qualifications name the current component cuts; every client/route and native public-install
receipt is fresh for this product candidate, not inherited from v0.15.0 or an earlier candidate.
All final windows passed their first attempt: RTX 4090 once, RTX 3090 once, and RTX 5090 one initial
attempt with no corrected window. Owner-operated receipts are tester-equivalent evidence, not
independent external-user outcomes.
| Acceptance evidence | Observed result |
|---|---|
| Exact stock OMP 18.14.0 platform and live tool/continuation/restart/resume/fail-closed checks | PASS on freshly downloaded macOS arm64 (preview), Windows x64, and Linux x64 under WSL2: published binary hashes and omp/18.14.0 matched; one typed read and linked result per client, visible marker, completed turn, exact continuation nonce, runtime identity, and fail-closed outage without model response or provider fallback. The macOS documented server restart returned its exact durable nonce. macOS and Windows each recorded three image inputs against the bound RTX 5090 runtime; no Linux/native vision claim. |
| Final-candidate guide execution: five routes / 31 executable quickstart blocks plus WSL2 checks | PASS 31/31 in documented routes: RTX 5090 host 2/2, macOS 10/10, Windows 5/5, native RTX 4090 7/7, native RTX 3090 7/7; Linux/WSL2 stock-client checks also passed. Every clone_commit is 56899320d416bafda5c15d7e454e9b593badc0a6. Pre-publication candidate-clone substitutions are explicit. Subsequent guide edits are prose-only; every executable block remains byte-identical. |
RTX 4090 and RTX 3090 public-install identities and RTX 3090 C2 route probes |
PASS in separate RTX 4090 and RTX 3090 public-URL checks: seven anonymously downloaded manifest assets per lane, closed 10-entry checksum sets, all hashes and served identities (06526bdb / 040d5923) matched. The downloaded installers returned already_installed, accepted exact qualified bytes without lifecycle-pointer changes or install-time start requests, then authenticated status/completion returned HTTP 200 and anonymous status HTTP 401. Both routes passed 22/22 behavior gates, sequential four-branch regression with zero transient retries; RTX 3090 also passed four-way fanout under C2, both stock OMP parallel scenarios, two committed automatic compactions with control recall, and structured outage. Promoted N16 canonical roots/stages were preserved/restored; first-lineage rollback is not_applicable, zero directions, not a predecessor upgrade/rollback claim. |
| Independent container and native restoration audits and observed hold durations | PASS in restoration: RTX 5090 hold 412.552 s, the same incumbent container and image sha256:760fb6bea925eb26d4fa53ab87a126494b69ba7bcfed515d23dee7afa65cdb2a, unchanged checkpoint mounts, HTTP 200, healthy Fleet and unless-stopped; native driver windows 499.977 s RTX 4090 / 1,225.308 s RTX 3090, 18/18 independent Restore and 18/18 final Audit checks per lane, exact stopped canonical states/stages and 450 / 370 W restored. |
Private-infrastructure deviation: the owner-approved temporary RTX 5090 acceptance backend
restore arm updated/started only the exact incumbent container, with no remove/create/run/relaunch
fallback. Candidate drivers and called backend scripts were unchanged. After health, Fleet
reconciliation and restart-policy checks, the backend's original bytes were restored and the
temporary key revoked. This is a same-container infrastructure restoration deviation, not a
runtime/component change or a publication authorization.
Stock-OMP parallel-proof oracle correction: acceptance gates engine/lane properties,
attributable value delivery and session isolation; exact marker punctuation and whole-answer
formatting are recorded as non-gating compliance observations. The final RTX 3090 subagents proof
passed despite non-exact parent and scout formatting, while both scouts' values reached their own
results and the parent and no cross-session value leaked. The sessions scenario passed all four
own recalls with separate session IDs. No retry, output filtering, special input case or weakened
isolation/attribution gate was used.
The composed acceptance binds these fresh
observations alongside the 26fa28d0 runtime qualifications. The
bound review ledger records:
- G4 fork council (
CR-20261031T162716Z-26fa28d0, sequenceomp-ninfer-v0.16.0-fork-26fa28d0,
base2af6b3de→26fa28d0): All four selected seats (daybreak-blue,gemini-argon,
gpt-astra,claude-mythos) completed in round 1 with zero aborts or truncations; closed without
remediation by accountable-lead disposition (fork-council ledger SHA-256
b2871c3664a0c4434dedc3aa2b3f2fdd7f5b301c9d3cecc7d1ce463c835aabe6, bound in
review/runtime-ledger.json, record SHA-256cbb0c780,
subject digest42d8473c), with no in-scope residual P0/P1 and six deferred P2/P3 rows (L1–L6) filed across
five issues inalphastorm/ninfer(#50 forL1 - G4 product-tooling council (
CR-20261101T123454Z-56899320-product, sequence
omp-ninfer-v0.16.0-product-56899320, frozen candidate
56899320d416bafda5c15d7e454e9b593badc0a6): All four selected seats (daybreak-blue,
gemini-argon,gpt-astra,claude-mythos) completed in round 1: 4/4 completed, 0 missing,
0 aborted, no retry or remediation dispatch. Closed by accountable-lead disposition after
lead-verified composition mitigation of the source-pin/profile-snapshot P1; no in-scope residual
P0/P1 remains. All 21 source items (17 evidence and 4 questions) map to 16 disposition rows;
11 P2/P3 rows (P4–P14) are deferred in
#128, withP4also retaining
#109.
Product disposition ledger SHA-256:
405c537aad60bfc5f81f38e979ce2f0c3afe42f80f27011ea11f97d4970427eb;
combined publicreview/runtime-ledger.jsonSHA-256:
ab195970955c58b584ddbcf2143072b56ced28e714251e975c0f67986722e627.
Final manifest evidence pins, strict ready verification, and founder-owned publication are separate
controls; acceptance and G4 approval grant no publication authority.
Support boundaries and next work
macOS remains preview; Linux client evidence is WSL2-scoped. Native lanes serve sequentially on
port 18082 (two concurrent requests on RTX 3090 run within the single active RTX 3090 lane, never
simultaneously with the RTX 4090 lane). Native vision, RTX 4090 C2, rk4v4 4-bit KV or
262,144-token native context, --fp32-head-logits, Group64 A8 prefill, small-T tensor-core native
decode, multi-GPU inference, multi-tenancy, silent cloud fallback, and independent external-user
outcomes are not qualified.
This release closes omp-ninfer #94,
#95,
#96,
#116, and
#117 (and incorporates alphastorm/ninfer
#48 and #49 across all three shipped runtime components). Carried runtime/tooling deferrals
#88–#90,
deterministic RTX 5090 nvcc module IDs, a corrected repeated native Group64 A8 screen, the
new fork-council P2/P3 deferrals in alphastorm/ninfer
#50–#54,
and post-release fork hygiene PRs
#55–#56
remain open.
Next: root-cause the rk4v4 4-bit KV true-cold restore divergence (preregistered
restored-input-3066 / prefill-3093 / generated-step-32 KV, StateImage, and logit capture on
diagnostic branch 6e980a5c, EXP-107) for a 262,144-token native-context follow-up release, and
evaluate stock Oh My Pi 18.14.2 (7efd7721) in a separate client-patch release (v0.16.1).