Skip to content

Releases

Qwen3.8 Flash-Next 1.0.5 — DFlash2 on four B70s

Choose a tag to compare

@wu1ff wu1ff released this 23 Oct 05:25

DFlash2 on 4× Intel Arc Pro B70 32 GB: Intel AutoRound W4A16 target, external BF16 draft with seven proposals, TP4 + expert parallelism, native 262,144-token context and four request slots. NVMe-backed disk BF16 PLE, APC, reasoning, automatic tools and up to two images are supported. The qualified host used Ubuntu 26.04 amd64 and 256 GB RAM.

Accepted integrated-serving-v1 image, published unchanged:
ghcr.io/wu1ff/qwen38-flashnext-b70@sha256:6e2ab4e686680a5208ff0119c4637341cba81942bd39b6576ea14dc8f547b83b

BetterBench v0.6.0: 336/336 measurements, 32 warmups, 160.11 weighted tok/s, 139.39 ms median TTFT; C1/C2/C4 aggregate 146.50/227.12/316.01 tok/s. Warm model/OS/PLE caches; zero prefix-cache hits. Results describe this workload run.

Model pack 1.0.5 works with unchanged b70ctl 1.0.1. /work uses the container's writable overlay, temporary files use /tmp, and /models and /draft are read-only. The previous MTP5 :disk image is preserved.

Complete model-pack sources, experimental archive, graphics and existing source-access/notices are attached with SHA256SUMS.

Qwen3.8 Flash-Next 1.0.4 — Intel AutoRound + dynamic MTP5

Choose a tag to compare

@wu1ff wu1ff released this 20 Oct 20:17

Intel AutoRound on four B70s with one qualified dynamic MTP5 / 262144-context profile.

BetterBench: 120.38 weighted tok/s; C1/C2/C4 aggregate 116.77/180.36/242.62 tok/s.

Complete model-pack sources and SHA256SUMS are attached. Existing source-delivery, bundled libgfortran and Intel recipient-term findings remain documented and unresolved.

Qwen3.8-Flash-Next Disk model pack 1.0.3

Choose a tag to compare

@wu1ff wu1ff released this 11 Oct 18:41

Flash-Next model pack 1.0.3 promotes the qualified Disk BF16 PLE runtime for 4× Intel Arc Pro B70, TP4 + expert parallelism. One image supports Base and MTP3 with all eight existing profiles at 32768, 65536, 131072 and native 262144 context. Vision (image2/video0), tools, reasoning, APC/ALIGN, the B70 residency shim and --exclude-tools-when-tool-choice-none remain part of the qualified contract.

Install through b70ctl → Model Packs → Browse Available Packs after refreshing the public catalog. For manual import, download qwen38-flashnext-b70-1.0.3.tar.gz and SHA256SUMS, run sha256sum -c SHA256SUMS, extract into a new directory and use Import Local Pack. Select the target, four cards, context and Base or MTP3 in Run Model. Startup typically takes several minutes; watch /health and Runtime logs.

The archive SHA256 is 9ece92a5c8e12dd9c94cd49bd77061e73933fd0c13d72db68a95c37f1f3523fe. It includes pack.json, README.md, RUNTIME_RECIPE.md and the complete cumulative patches/** source/build export: final overlays, native sources/headers/dispatchers, dependency locks, checksum manifests, build recipes, notices and historical provenance. Native build instructions are in RUNTIME_RECIPE.md and patches/disk/README.md; exact Intel compiler packages are acquired separately under their applicable terms. Model weights are separately acquired from the revision-pinned checkpoint.

The existing GHCR package has the sole tag disk. Use the immutable image reference:

ghcr.io/wu1ff/qwen38-flashnext-b70@sha256:b13fed6dd54051395f603409ada440c74afcbd5331810eee3acd62315d7787e3

This is the verified Controller Docker image ID on the supported containerd image store. Its separate OCI config digest is sha256:2399885f6fbc59cdb602fe869350ddd80e085f62da25d55859d683b5e3450baf; classic image stores can expose that config ID and fail the existing Controller identity check. The model-pack release does not rebuild or republish the image.

Base retains four slots, MBT1024 and the independently qualified aggregate-prefill cap256 during active decoding. MTP3 retains depth3, MBT1024, eager/KV-only draft prefill, post-capture warmup, fused metadata, device split planning and native tiled reduction. Disk maps the original BF16 table and uses bounded pinned staging (20 MiB across TP4 at MBT1024); this is not total host-memory usage or a minimum-RAM guarantee.

Both modes passed the retained native-context serving campaign, tools/reasoning/vision/APC/cancellation checks and focused Controller qualification of the notice-corrected image. Descriptive final-image coding samples measured Base45.48–50.24 tok/s and MTP3108.77–122.21 tok/s across 64K/160K cold and reused APC requests. These are n=1 samples with differing output trajectories and natural OS cache history. Accepted controlled combined tuning improved warm generation about1.8–2.9% in matched fixtures; neither result establishes a universal speedup. Native-ceiling checks used262112 input+32 output; sustained ceiling-length generation, storage-cold endurance and concurrency throughput/fairness remain unmeasured. Full measurement definitions, findings and historical limitations are retained in the recipe.

PR#4–#8 remain closed unmerged with their individual dispositions preserved. KaloyanNaumov/Lumnus research and configuration influence is acknowledged; no identifiable submitted executable source ships in Disk. Original co-author trailers remain historical records and are not assigned to this independently developed release. The retained source notices and third-party licenses continue to apply.

The Vanilla1.0.2 release and historical source documentation remain available. Its GHCR image was intentionally superseded and is unavailable there; rollback requires a saved local Vanilla image and pack. Controller updates can retire older packs and unshared owned images, so preserve local recovery assets before updating. There is no automatic runtime rollback or catalog downgrade.

Qwen3.8-Flash-Next B70 pack 1.0.2 — runtime source patches

Choose a tag to compare

@wu1ff wu1ff released this 01 Oct 19:59

Pack-only source/provenance release for qwen38-flashnext-b70 1.0.2.

The archive now includes patches/** beside README.md, RUNTIME_RECIPE.md, and pack.json. The source bundle covers the shipping Qwen4Exp XPU gate, pinned-host PLE, FULL graph capture fix, HC K-split, final Astra GDN speculative metadata, seven-expression GDN index64 correction with native library source/build recipe, B70 residency shim source/build recipe, and the production dense-QSA serving config and preparator. Each patch has retained provenance and checksums.

The production runtime remains ghcr.io/wu1ff/qwen38-flashnext-b70:1.0.0 at registry digest sha256:85512b52c09fa660a2e7fe441417129e7c47fac727fd85f66ccea6b65e0a9122. No image rebuild or GHCR push occurred. All eight resolved profile command, environment, mount, image, and digest contracts are byte-identical to pack 1.0.1.

Validation: development validator 8/8, ordered patch application and byte comparison against retained overlays, native patch applied to vllm-xpu-kernels 0.1.14.1 commit 6d92b1bfbf32767ecda8e819613eb151e70030ad, patch checksums verified, deterministic archive regeneration, Controller archive extraction/load and go test ./... passed.

The archive is deterministic (sorted GNU tar entries, epoch timestamps, regular mode 0644, nameless gzip). Verify it with the attached SHA256SUMS.

Qwen3.8-Flash-Next B70 pack 1.0.1 — tool-calling launch-contract fix

Choose a tag to compare

@wu1ff wu1ff released this 27 Sep 21:59

Pack-only release (1.0.4 precedent): runtime id/image/digest unchanged — ghcr.io/wu1ff/qwen38-flashnext-b70:1.0.0 @ sha256:85512b52…; no GHCR push, image bytes identical to 1.0.0.

What changed

The 1.0.0 launch contract omitted vLLM's tool-calling flags, so OpenAI-compatible requests with tools + "tool_choice": "auto" failed with HTTP 400 ("auto" tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set). Pack 1.0.1 adds, once at the shared runtime layer of pack.json (inherited identically by Base and MTP3 and all 8 profiles):

--enable-auto-tool-choice --tool-call-parser qwen3_xml --reasoning-parser qwen3

Both parsers ship inside the unchanged image and the pinned chat template emits exactly the Qwen3 XML tool syntax the parser reads (static audit of the exact pinned stack; no custom template, no other flags).

Qualification (b70ctl path, 262144, released digest)

  • Base 15/15 PASS: HTTP 200 on the original 400 request shape, valid OpenAI tool_calls (name + JSON-exact arguments), tool-result round trip (role: tool), streaming tool calls, tool_choice: none plain text, reasoning/content separation (message reasoning field, stream + non-stream), ZEBRA-42 vision regression exact, APC hit 2,496/2,698 with exact recall, clean logs, 4/4 devices.
  • MTP3 16/16 PASS: depth-3 active, 3-draft invariant exact (408 = 3×136), same tool surface end-to-end, zero runtime/spec/parser exceptions, 4/4 devices.

Archive qwen38-flashnext-b70-1.0.1.tar.gz is deterministic (GNU tar, epoch, 0644, sorted, nameless gzip); SHA-256 in SHA256SUMS.

Qwen3.8-Flash-Next B70 Pack 1.0.0 (Base / MTP3)

Choose a tag to compare

@wu1ff wu1ff released this 27 Sep 20:58

Initial release of the Qwen3.8-Flash-Next (Qwen4Exp) B70 model pack.

One image, both modes: ghcr.io/wu1ff/qwen38-flashnext-b70:1.0.0
(registry digest sha256:85512b52c09fa660a2e7fe441417129e7c47fac727fd85f66ccea6b65e0a9122) serves Base (speculation off, util 0.85) and MTP3 (MTP depth 3, util 0.91) — launch configuration only.

  • Model: devan-carlin/Qwen3.8-Flash-Next-W4A16 @ 40b8f18d… (ungated, 31 files, ≈180 GB incl. the 102.4 GB PLE table in the same revision)
  • 8 profiles, TP4 on 4× Arc Pro B70: w4a16-{base,mtp3}-tp4-{32768,65536,131072,262144}
  • Context ladder law: the 256K boot of each mode qualifies its lower tiers; no separate lower-context boots
  • Hard requirements baked into the image: pinned-host PLE, B70 residency shim, dense-QSA serve prep, TP4+EP, W4A16 bf16, FULL_AND_PIECEWISE graphs [1,2,4,8,256], APC ON / ALIGN, vision image:2 (video unqualified/off)
  • Includes the GDN convolution state-index 64-bit correctness correction (deep-prefill DEVICE_LOST fix)

Deferred: concurrency tuning, video. Full provenance in the pack RUNTIME_RECIPE.md.

Qwen3.8-27B Model Pack v1.0.6

Choose a tag to compare

@wu1ff wu1ff released this 25 Sep 04:08

The Qwen3.8-27B pack runtime moves to ghcr.io/wu1ff/qwen38-27b-b70:1.0.6
(sha256:f3006020add52dedfed08947b05234ef4c058b47158afe61c0e79f9ef60a6f4a);
the previous authority sha256:c0c9b8f3… remains pullable as tag 1.0.3.

1.0.6 is the 1.0.3 runtime bytes plus exactly one further delta — the
two-file SD conv-state migration fix (patches/013). Root cause of the
spontaneous !-degeneration seen under APC/ALIGN: every conv-state
migration at an ALIGN advance or checkpoint with accepted ≥ 2 copied
from the wrong block-table source column (src_col + accepted − 1
instead of src_col + token_bias), corrupting temporal state from the
next token on. The accepted=1/bias=0 path — all of Base and MTP1, and
every APC-off posture — was already byte-identical and is untouched.

Everything else is unchanged from pack 1.0.5: launch contract,
DFlash2 APC-ON posture, graph policy, utilization envelopes, and model
revisions. Qualification: live b6 gate, the 18-probe degeneration
battery clean on TP4 FP8 uncensored 262144 (0/18 enter), production
lanes FP8 TP2 64K and INT4 TP1 32K (2/2 PASS), and representative
TP1/TP2/TP4 qualification through the pack import and render path
(96/96 deterministic profiles).

Qwen3.8-27B Model Pack v1.0.5

Choose a tag to compare

@wu1ff wu1ff released this 22 Sep 20:39

Doc-only correction of the v1.0.4 short-prefix APC table, using the law measured by the final closeout matrix (one boot, 15 exact tokenizer-verified shared-prefix boundaries).

Measured law (dFlash2 APC, TP2 geometry): with B = floor(shared_tokens/1024), the reconciled prefix-cache hit is 1024 x (B-1) tokens — one block is always dropped for the EAGLE lookahead — and one block further is lost when the divergence lands in the final token of a hash block (shared_tokens mod 1024 == 1023). So: shared < 2048 restores nothing; 2048 restores 1024; 3071 dips to 0; 3072 restores 2048; 4096 restores 3072; 5120 restores 4096; 7168 restores 6144. A zero hit on short shared prefixes is cache geometry, not a failure.

No runtime change: the image stays ghcr.io/wu1ff/qwen38-27b-b70:1.0.3 @ sha256:c0c9b8f3..., and the launch contract is identical to v1.0.4 (stale --mm-encoder-attn-backend TORCH_SDPA removed; vision via XPU-default FLASH_ATTN).

Qwen3.8-27B Model Pack v1.0.4

Choose a tag to compare

@wu1ff wu1ff released this 22 Sep 20:23

Pack-only release on the unchanged 1.0.3 runtime bytes (image ghcr.io/wu1ff/qwen38-27b-b70:1.0.3, digest sha256:c0c9b8f3... — no runtime tag or byte change).

Vision fixed (VISION-LAUNCH-FLAG-FIX). The stale --mm-encoder-attn-backend TORCH_SDPA launch override is removed: it made the head-dim-72 ViT encoder request an SDPA primitive oneDNN 3.12 cannot create, killing every image request. With no override the XPU platform default selects FLASH_ATTN for the ViT and vision works. Live-qualified on INT4 dFlash2 TP2 64K (cold and warm image grounding, prefix-cache-composed vision, image-grounded tool call); every other profile carries the fix by launch equivalence. Exception: INT4 TP1 32K's vision memory envelope is unqualified — large-image/multimodal use is not recommended on that profile.

Also documented: INT4 TP2 256K is a maximum-capacity profile (KV pool ~1.01x its context — one near-full-context request at a time), and short shared prefixes (< ~3072 tokens) do not produce APC reuse on dFlash2 (hybrid block/eagle-drop/Mamba-chunk geometry — expected, not a failure).

Qwen3.8-27B Model Pack v1.0.3

Choose a tag to compare

@wu1ff wu1ff released this 22 Sep 19:00

Qwen3.8 27B B70 pack 1.0.3 — DFlash2 ALIGN prefix caching + XPU Mamba pointer-overflow fix.

Runtime authority: ghcr.io/wu1ff/qwen38-27b-b70:1.0.3 @sha256:c0c9b8f382298bdd90f78ae2f4700637241c7a8e2b6c7ab591dc933c76b73cbf (retag-only promotion of the qualified align-apc-c1 bytes; the previous authority 314786fd remains published as tag 1.0.2).

What changed: DFlash2 Automatic Prefix Caching is ON (--enable-prefix-caching; the Qwen3.8 hybrid config resolves the Mamba cache mode to ALIGN). The enabling runtime fix is the retained upstream vLLM PR #48109 [Bugfix][XPU] Fix Mamba state pointer overflow — Level Zero device pointers at or above 2^63 crashed the int64 Mamba state-address stores; the reinterpretation preserves the exact 64-bit pattern and is inert wherever the crash path is not hit. Qualification (verdict PROMOTE): 12/12 delta lanes across INT4 TP1/TP2/TP4 and FP8 TP2/TP4, standard and uncensored artifacts each, with real FA+Mamba prefix-cache hits, tool calls, restarts, 951/951 strict proposal rows (zero misses), and a clean 12/12 health campaign.

Utilization envelopes: INT4 DFlash2 TP1 32K now serves at 0.91 and INT4 DFlash2 TP2 256K at 0.86 — pack-envelope corrections independently required by the previous runtime as well (not an ALIGN tax); every other DFlash2 profile stays at 0.82.

Unchanged: all five model revisions and file inventories, the 96-profile matrix, Base/MTP1 contracts, the oneCCL launch environment, and the dFlash2 graph policy. Text and tools are qualified; vision remains separately unqualified and should not be assumed on these lanes.