Siliang Engine v0.1.4 (Experimental)

Siliang Engine v0.1.4

v0.1.4 is a correctness and Windows release-packaging hotfix on top of v0.1.3.
It does not introduce new model-performance claims or change the recommended
v0.1.3 model profiles.

What changed

• Fixed a cross-request K/L2 ownership bug in the SLFU hot-demotion path. With
  K-prefill disabled, prompt processing for a later request can legitimately
  repopulate L2 with an expert that is still warm in K from the preceding
  decode. A later K eviction used to attempt a second L2 store for that expert
  and fail closed with duplicate-resident. The transition worker now
  quiesces L2 before ownership changes and reuses an already-resident L2 copy
  instead of creating a duplicate.
• Expanded fail-closed K/L2 diagnostics so rejected victim stores report the
  rejection stage, layer, expert, and slot rather than collapsing into a
  generic swap failure.
• Enabled the embedded llama.cpp Web UI in Windows release builds. Release
  builds first build the UI from source with npm, may fall back to the verified
  prebuilt UI source, and fail the build if LLAMA_UI_HAS_ASSETS=1 is not
  present in the generated UI header.
• Persisted runtime CPU-dispatch evidence in packaged release provenance so the
  selected packaged CPU backend remains auditable after CI packaging.

Release qualification

The v0.1.4 hotfix was qualified on the Windows CUDA reference workstation with
fresh CPU and CUDA builds.

• Fresh Windows CPU Release build: completed; embedded Web UI verified.
• Fresh Windows CUDA 13.2 sm_75 qualification build: completed; embedded Web
  UI verified.
• Embedded Web UI runtime smoke: GET / returned HTTP 200 with embedded HTML.
• DeepSeek V4 Flash K/L2 multi-request regression:
  • request 1: 128/128 decode tokens, ~1.74 tok/s;
  • next prompt: 221 tokens processed on the same server process;
  • request 2: 32/32 decode tokens, ~1.61 tok/s;
  • no duplicate-resident, K/L2 transition failure, fail-closed event, or
    compute error was observed.

The throughput values above are smoke-test observations only. v0.1.4 retains
the model qualification and limitations documented for v0.1.3; see
v0.1.3.md for the broader performance evidence.

Compatibility and boundaries

• The public expert-cache configuration remains the v0.1.3 typed CLI/server
  interface.
• Existing v0.1.3 model recommendations remain unchanged.
• Use --parallel 1 whenever L1 K/R/P is enabled.
• L1 K/R/P remains intentionally incompatible with LoRA in this prototype.
• The Siliang expert arena remains Windows-only; Linux CPU and macOS Metal CI
  continue to protect the upstream-compatible non-arena paths.

Downloads

• Windows x64 CPU
• Windows x64 CUDA 13.2 multi-arch
• SHA256SUMS

Siliang Engine v0.1.3 (Experimental)

Siliang Engine v0.1.3

v0.1.3 brings the bounded MoE memory hierarchy into the normal Siliang Engine
CLI/server lifecycle and qualifies it across several model families on the
Windows CUDA reference workstation.

The main goal of this release is not to establish a new absolute performance
ceiling. It is to move the previously experimental memory hierarchy into a
bounded, configurable, release-gated runtime path and verify that it behaves
correctly across different MoE architectures.

What changed

• Added typed expert-cache configuration shared by llama-cli and
  llama-server; no hidden environment setup is required.
• Added managed host L2 policies plus bounded CUDA K residency, per-schema R
  exchange slots, and a global pinned P elevator.
• Canonicalized Siliang lifetime-frequency admission as slfu;
  cumulative-lfu remains accepted as a legacy alias.
• Added exclusive K/L2 turnover controls (--admit-k-cold and
  --demote-k-hot) for SLFU.
• Added the architecture-guarded DeepSeek4 FRONT rolling path. The release
  candidate fences a FRONT bank overwrite behind completion of preceding CUDA
  consumers; this closes a determinism bug found during release qualification.
• Added an opt-in routed-MoE prompt path using bounded route unions. It remains
  experimental and is qualified separately from decode.
• Kept router scoring and mixture weights on the GPU; the CPU mapper receives
  selected expert IDs only for cache control and physical-slot translation.
• Added context-owned lifecycle integration, failure-closed capacity checks,
  aggregate route telemetry, and release/runtime gates.
• L1 K/R/P remains intentionally incompatible with LoRA in this prototype.

v0.1.3 release qualification

The table below is new v0.1.3 server evidence, not a restatement of the
older benchmark claims.

These are natural-generation qualification runs rather than route-replay
benchmarks. No frozen route trace, route oracle, or replayed expert sequence was
used. Each repetition started a fresh llama-server process and the measured
request was issued directly, without a separate generated-token warmup request.

The measured decode used 256 generated tokens with temperature=0, top-k=1,
seed 42, and cache_prompt=false. Routed-expert activity therefore follows the
model's actual autoregressive generation. This does not exclude llama.cpp's
normal internal startup/empty-run initialization.

The exact commands are in
../CONFIGURATION.md.

Model / path	Median decode	Range	Result
Gemma4 26B-A4B, K1440/R16/P16	21.651 tok/s	21.253-21.672	3/3 complete, identical token hash
Qwen3 30B-A3B, K1440/R16/P16	19.261 tok/s	17.360-20.164	3/3 complete, identical token hash
Qwen3.6 35B-A3B, K1440/R16/P16	9.279 tok/s	7.549-9.534	Correct, but not the recommended path
Qwen3.6 35B-A3B, no expert cache	11.011 tok/s	10.210-11.392	3/3 complete, identical token hash; recommended v0.1.3 path
Ornith 1.0 35B, K1920/R16/P16	13.967 tok/s	13.948-14.004	3/3 complete, identical token hash
GPT-OSS 120B, 18 GiB managed L2	3.344 tok/s	3.335-3.356	3/3 complete; all runs observed low host-memory headroom

GPT-OSS is retained as a supported managed-L2 path.

The current 3.344 tok/s median should not be interpreted as a matched
regression against the historical 4.052 tok/s result. The historical
benchmark issued a separate 48-token generated warmup before each 256-token
measured request, while the v0.1.3 qualification above did not. In addition,
all three current repetitions were observed under low host-memory headroom.

The two values therefore characterize different execution conditions; the
current value is retained as release qualification evidence rather than
promoted as a new performance ceiling.

DeepSeek4 current profile

The v0.1.3 DeepSeek4 decode profile is:

& "<llama-server.exe>" -m "<deepseek4-expert-major.gguf>" `
    -ngl 99 -ncmoe 43 -nkvo --no-op-offload `
    -c 4096 -b 512 -ub 512 -t 2 -tb 2 --parallel 1 `
    --expert-cache `
    --expert-cache-l2-mib 8192 `
    --expert-cache-l2-policy lru `
    --expert-cache-l1-k 216 `
    --expert-cache-exchange-r 12 `
    --expert-cache-elevator-p 12 `
    --expert-cache-l1-policy slfu `
    --admit-k-cold on `
    --demote-k-hot on `
    --expert-cache-roll deepseek4 `
    --no-expert-cache-prefill

During the release gate, the original asynchronous FRONT overwrite path produced
different greedy continuations across fresh starts. Isolation showed:

• --no-expert-cache: deterministic 3/3;
• managed L2 only with FRONT off: deterministic 3/3;
• K216/L2/R/P with FRONT off: deterministic 3/3;
• FRONT enabled before the fence: divergent output after roughly 19-44 tokens.

The release-candidate FRONT completion fence then produced an identical token
hash in 3/3 fresh 64-token runs at 1.936-1.983 tok/s.

A separate full-depth release gate generated all 2,048 tokens at
1.94436 tok/s (514.31 ms/token). The run reached the 2,048-token route
checkpoint with zero runtime failure.

Host-memory headroom was low during that run, so this is primarily a
depth/stability result, not a clean isolated throughput ceiling.

The historical 2.774 tok/s DeepSeek4 result discussed below is also a real
natural-generation benchmark, but it is not a matched before/after comparison
with this profile. It used an 18 GiB arena and 256-token fully-cold repetitions,
while the current release profile uses 8 GiB L2 plus K216/R12/P12 and FRONT and
was additionally exercised over a 2,048-token decode.

The raw tok/s values therefore represent different operating points and should
not be interpreted as evidence of a v0.1.3 throughput regression.

Historical performance evidence

The earlier README/PERFORMANCE measurements remain part of the project evidence
and are intentionally not rewritten as v0.1.3 claims.

They answer different questions from the release qualification above. The
v0.1.3 qualification uses natural fresh-server generation with no separate
generated-token warmup request and no route replay/oracle. Historical
experiments used their own frozen protocols, including different arena
capacities, fully-cold process starts and, for GPT-OSS, a separate generated
warmup request.

Raw tok/s values across the two sections are therefore not matched
before/after regression measurements.

Notable retained results include:

• DeepSeek V4 Flash 0731 expert-major: 2.774 tok/s median versus
  2.274 tok/s for the stock layout with the same 18 GiB arena (+22.0%).
  This was a matched fully-cold natural-generation benchmark: three fresh
  processes per arm, Windows standby-list purge before every process start,
  and exactly 256 generated decode tokens per request with no separate
  generated-token warmup. All six outputs were byte-identical and reached the
  fixed 256-token limit.

• DeepSeek V4 Flash stock GGUF: 2.291 tok/s with the 18 GiB arena
  versus 1.375 tok/s mmap on the same stock file (+66.6%). This is a
  separate matched same-file historical experiment isolating the arena path
  rather than the expert-major layout.

• GPT-OSS 120B: 4.052 tok/s managed-arena median versus
  1.972 tok/s mmap on the same repacked file (+105.5%). This comparison
  used three fresh processes per arm and a matched protocol on both sides:
  each process first generated a separate 48-token warmup request, followed by
  a 256-token measured request. All six measured cells produced 256 nonempty
  tokens and the same retained decoded text. The 48 warmup tokens were not
  included in the reported decode throughput.

• Source-research DS4 natural-depth observations: single 2,000-token starts
  at 12/14/16 GiB L2 completed at
  2.12587 / 2.20073 / 2.28674 tok/s respectively. These were actual
  autoregressive generations, not frozen-route or route-replay experiments,
  but they were single starts with different generated work and remain
  research observations rather than matched release benchmarks.

See ../PERFORMANCE.md for the complete protocols,
limitations, raw repetitions, output-equivalence evidence and retained artifact
identities.

Model boundaries

• Gemma4 26B-A4B: positive v0.1.3 K1440 qualification.
• Qwen3 30B-A3B: positive v0.1.3 K1440 qualification.
• Qwen3.6 35B-A3B: the K1440 path is correct but slower than the matched
  no-cache control; the shipped recommendation is --no-expert-cache.
• Ornith 1.0 35B: positive v0.1.3 K1920 qualification.
• GPT-OSS 120B: supported through managed L2; the current release receipt
  is pressure-qualified and uses a different protocol from the historical
  warm-runtime performance benchmark.
• DeepSeek4: K216/R12/P12 + 8 GiB L2 + FRONT is the current release profile;
  FRONT determinism is explicitly gated and a full 2,048-token decode completed
  successfully.
• M4: remains inconclusive/no-go; no useful integrated architecture path is
  claimed in v0.1.3.

Whole-L2 CUDA registration, heterogeneous batch-copy experiments, route replay
oracles as product inputs, DirectStorage rolling, generic whole-model static
rolling, and the cost-model/pre-bake solver remain outside v0.1.3.

Route replay and frozen-route experiments remain research tools: none of the
release-qualification throughput rows above depend on them.

Server use

Use --parallel 1 whenever L1 K/R/P is enabled. The OpenAI-compatible server
can then be addressed normally at /v1.

DeepSeek non-reasoning integrations can additionally use:

--reasoning off --reasoning-format deepseek

when the client expects plain content...

Siliang Engine v0.1.2 (Experimental)

Siliang Engine v0.1.2

This release fixes and hardens the Siliang build and release pipeline.

Previous release builds still carried assumptions from the original development
machine. v0.1.2 moves the public build back onto the portable backend model used
by llama.cpp, while keeping the Siliang MoE arena as an opt-in layer on top.

What changed

• Fixed the portable release build pipeline.
• Removed the sm75-only CUDA release target. The CUDA package now uses the
  multi-architecture selection provided by the pinned llama.cpp / CUDA
  toolchain.
• Disabled host-native CPU compilation for release artifacts and ship the
  runtime-selectable CPU backend variants instead.
• Added release-time backend/loadability checks and a machine-readable build
  configuration receipt to each package.
• Added Linux CPU and macOS Metal compatibility builds to CI so Siliang changes
  cannot silently break the upstream multi-platform build surface.
• Updated the Siliang Engine logo.

The Siliang RAM arena itself is still a Windows feature in v0.1.2. Linux and
macOS are compatibility/build gates for the underlying fork; they are not yet
Siliang arena release targets.

Downloads

• Windows x64 — CPU
• Windows x64 — CUDA 13.2 multi-arch
• SHA256SUMS

The CUDA package is no longer specific to the RTX 2070 / compute capability
7.5 build host. The exact CUDA architecture set selected during the tagged build
is recorded in provenance/SILIANG-BUILD-CONFIG.json inside the package.

Siliang Engine v0.1.1 (Experimental)

Changes since v0.1.0

• Fixed Windows release packages to include the required OpenSSL runtime DLLs, license text, and runtime provenance; added packaging regression coverage.

Downloads

• Windows x64 CPU ZIP
• Windows x64 CUDA 13.2 sm75 ZIP
• SHA256SUMS

Experimental notes

• Windows-only experimental release.
• The CUDA package targets CUDA 13.2 and sm75 (Turing / RTX 20-series).
• The Siliang arena/cache path remains opt-in through SILIANGEM_CACHE_MIB; leaving it unset keeps the ordinary path.