Repository navigation
LexiEngine Inkling 2
State of version b753bf3, November 21, 2026. Speed and accuracy against llama.cpp: Benchmarks.
LexiEngine's runtime for Inkling-2-Mini (Apache-2.0, released 2026-11-20): the text backbone on one Vulkan device,
served by lexi-engine --backend vulkan. Source:
| Part | Where |
|---|---|
| Device runtime and server wrapper |
crates/server/src/inkling_vulkan.rs (InklingDevice, InklingVulkanRuntime, MemoryPlan) |
| Kernels |
crates/vulkan-ops/src/shaders/inkling.comp, inkling_attn.comp, inkling_attn_cm.comp, inkling_expert.comp, inkling_expert_cm.comp; Rust side crates/vulkan-ops/src/inkling_ops.rs
|
| Runner (checks and speed) | crates/server/examples/inkling_vulkan.rs |
| CPU reference |
crates/model-inkling (tool inkling-reference) |
| Routing and report |
crates/server/src/main.rs (ahead of the generic factories), crates/server/src/rcc_served_architectures.rs (Vulkan lane only) |
| Tokenizer | pre-tokenizer inkling2 = the GPT-4o splitting, crates/tokenizer/src/lx36_pretokenizers.rs
|
| llama.cpp patches used for conversion and the baseline | at the end of this page |
24 layers, width 2,304, 24 query heads and 8 key/value heads of 128 values, vocabulary 200,058 (tables padded to 201,024), 128 routed experts of width 1,152 with 8 used per token, context up to 1,048,576 in the file's metadata.
The loop. Layers 6 to 17 (the core) run three times with shared weights. A forward pass applies layers 0-5 once, the core three times (each pass starting again from the stream that entered the core; passes 2 and 3 take a gated message from the pass before through low-rank controllers of rank 64 on the query, key, value and expert inputs), then layers 18-23 once: 24 + 2 x 12 = 48 layer applications, each with its own cache slot. Slots 0-23 are the first pass; slot 24 + 12 (p - 2) + (l - 6) is core layer l in pass p. Residuals are mixed by blocks of 6 layers (attention residuals with learned queries over the block outputs so far).
Attention. Layers 5, 11, 17 and 23 see every position (global); the other 20 see a window of 1,024. Global slots:
layers 5 and 23 once, 11 and 17 three times = 8 slots; windowed slots: 40. Three query heads share a key/value head.
No rotary positions: a relative bias by distance (16 values per head, distances below 1,024), queries and relative rows
clipped to a norm (qk_clip 22.63, relative_clip 8) and scaled on global layers by
tau = 1 + 0.1 ln(max(1, (position + 1) / 8,192)); a per-head sink logit (one extra softmax term), and a sigmoid output
gate. Keys and values pass a 4-tap causal convolution (static taps, plus taps computed from the input on global layers)
before they are cached; the attention and expert outputs pass 4-tap convolutions too. A logit soft cap of 30.
Experts. Router with a bias, 8 of 128, route weights scaled by 8; gate and up fused, activation "situ" with betas 4 and 25.
The runtime reads llama.cpp's GGUF layout for inkling2_loop. The file used for every number here is
Inkling-2-Mini-NVFP4exp-Q8.gguf (17.6 GB), converted with the patch in section 8:
- 40 expert tensors in ggml NVFP4 (GGUF type 40: blocks of 64 values, 4-bit E2M1 levels, a UE4M3 scale per 16): the
vendor's 4-bit values and block scales repacked unchanged (decoded rows checked bit-identical to the checkpoint),
with the vendor's per-expert factor amax / denominator in a per-expert
.scaletensor that the kernels apply. - 8 expert tensors the vendor stores in MXFP8, and the whole trunk (attention, controllers, router, embedding, head): Q8_0 (blocks of 32 with a binary16 step), re-quantized by the converter from the vendor's MXFP8 / BF16.
- The draft head (
mtp.safetensors) and the vision tower are not in the file.
The expert kernels read NVFP4 and Q8_0 rows; the dense products read Q8_0 through the engine's exact row kernels (runs of up to 8 tokens) and its matrix-core Q8_0 kernel (prompt passes of a multiple of 16).
-
Prompt passes of up to 2,048 tokens (a multiple of 16; a tail of fewer than 16 runs in steps of up to 8). The
experts of a pass are grouped by expert on the device and computed as 16x16x16 binary16 matrix products (NVFP4
levels times their block scale are exact in binary16; Q8_0 rows are rounded to binary16; the inputs and hidden rows
are binary16). Attention of the global layers in a pass runs on matrix cores too: queries prepared once in
binary16 (rotated as the keys are), keys walked in blocks of 64; since 2026-11-21 the windowed layers too (from the
block holding the earliest query's window, older keys masked;
LEXI_INKLING_WINDOW_CORES=0keeps the scalar kernel with its online softmax over blocks of 256 keys). Since 2026-11-21 (evening) each subgroup of that kernel keeps its share of the weighted value sums (16 dimensions x 16 queries of each of the three heads) in matrix-core registers across the key blocks: a query's rescale by exp(old largest - new) multiplies the accumulator components of its column (on RDNA3 a lane holds one column of a 16x16 F32 accumulator), and the block's values are added on the matrix cores, so the sums no longer go through shared memory every block (shared memory 52 -> 40 KB;LEXI_INKLING_ATTN_REGACC=0keeps the earlier kernel). Passes of a multiple of 32 tokens useinkling_attn_cm32.comp: the same with 32 queries a workgroup, so a key block is decoded once for 32 queries and the 24 score products of a block divide evenly over the 8 subgroups (6 accumulators a subgroup, 59 KB shared;LEXI_INKLING_ATTN_Q32=0: 16 queries). -
Controllers (passes 2 and 3 of the core take a message through four low-rank controllers per layer): a set of
controllers in two commands, the low-rank values of each (normalized input, message) pair, then each input row
x - tanh(forget) x + tanh(gain) message with gain and forget stored transposed, in passes and single tokens alike
(since 2026-11-21;
LEXI_INKLING_FUSED_CONTROL=0: the earlier five commands per controller). -
Single tokens (generation, and runs of up to 8): the experts per (token, route) with F32 inputs, each row read
by a subgroup of 32 lanes taking consecutive 16-value units (since 2026-11-21;
LEXI_INKLING_EXPERT_LANES=1gives a row per lane, the earlier kernel), down rows per route with a combine step. The keys a token sees are split into up to 128 chunks (each a multiple of 256 positions), one workgroup per key/value head and chunk (taking the tokens of a short run one after another), and a second dispatch merges the chunks; so at depth the whole card works on one token's keys. -
One recorded forward per token shape (since 2026-11-21): runs of up to 8 tokens read their position from a
device word instead of the recorded commands, so a token's commands are the same at every position, and the queue
replays the recorded command buffer of the previous token of that shape instead of recording it again
(
LEXI_INKLING_STEP_POSITION=0,LEXI_INKLING_REPLAY=0switch them off; the logits are bitwise equal either way). - The head computes logits for the last token (or every token of a 16-token pass for scoring), applies the soft cap on the device; the server's device sampler picks the token there.
mtp.safetensors holds a 3-layer block drafter (the vendor calls it DSpark) with a recurrent ("Markov") head; its
wiring follows the public SGLang implementation. With LEXI_INKLING_DRAFT naming the checkpoint folder (config.json
with mtp_config, and mtp.safetensors), the runtime loads it next to the main model (Q8_0 matrices; 1.45 GB with
F32 matrices at first, smaller since):
- Every pass keeps four feature rows per position: the stream (the running sum of the current residual block) just
before layer applications 5, 23 and 41 in pass order, and the final normed row. Each draft layer mixes them with its
own softmax weights (
fuse_weight), normalizes, and writes keys (per-head norm, rotary positions, theta 10^6) and values for those positions into the draft's own context cache (8-bit rows; layers 0 and 2 windowed to 1,024). - A speculative step at position p with anchor token a (the last token the main model produced): a block of a's normed embedding and 7 mask rows runs through the 3 layers (pre-norm attention over the context and the block, not causal inside the block; SiLU MLP of 9,216); the 7 draft rows go through the output table once; the recurrent head (state of 256: gated update from [state, embedding of the previous token, the draft row, the mixed features of position p - 1]) adds its output, folded into a 257-row table computed at load (the output table times its output projection and bias), and the largest logit is the proposal. All on the device; 8 token ids are read back.
- Verification: the anchor and the 7 proposals run as one 8-token pass in which every token is computed exactly as
alone (per-token row, expert and split-key attention kernels); the main model's greedy token after each position is
chosen on the device; the matching prefix plus one token are kept. The short convolutions' rows are restored for a
partial acceptance by replaying the kept tokens' saved input rows over the rows saved before the pass. A run whose
attention chunk size would change inside it (every 32,768 positions) takes a plain step instead. So the output is
always that of plain greedy decoding (checked by the runner's
specmode). - Verification length: 8 tokens (the anchor and all 7 proposals) below 131,072 positions, 4 from there on, because
deep in the context every token of the pass walks all keys of the global layers (
LEXI_INKLING_SPEC_RUN=N, 2 to 8, fixes it). Measured at 16,384 (8 better) and 261,632 (4 better) only.LEXI_INKLING_SPEC_POLICY=adaptivechooses the length per step by expected tokens per millisecond from measured step times and acceptance; it was slower than the lengths by depth in both measurements (76.1 against 78.7 tok/s at 16,384; 33.6 against 42.1 at 261,632, where a few early rejections made it settle on plain steps), so it is not the default. - Served through the greedy speculative step of the scheduler: requests with temperature 0 use it; sampled requests decode plainly.
4-bit rows by default and the only format the server uses: llama.cpp's q4_0 layout per head of 128 values, 4 blocks
of 32 with a binary16 step and levels (code - 8) x step: 18 words = 72 bytes a head. Before rounding, keys are rotated
with a Walsh-Hadamard transform over the head's 128 values and values per half of 64 (as llama.cpp does); queries are
rotated alike and attention outputs rotated back. LEXI_INKLING_KV=f32 in the runner keeps rows as computed, for
checking the arithmetic only (never a serving or benchmark setting). There is no 8-bit row format.
A windowed slot keeps window + 3,072 positions (one pass of 2,048 writes its rows before they are read); a global slot keeps every position. The short convolutions keep their last 3 input rows per slot.
| Item | Arithmetic | Bytes |
|---|---|---|
| Weights | every tensor of the file on the device | 17.62 GB |
| Global caches | 8 slots x keys and values x 262,144 positions x 8 heads x 72 bytes | 2,415,919,104 |
| Windowed caches | 40 slots x 2 x 4,096 positions x 8 heads x 72 bytes | 188,743,680 |
| Caches together | 2.60 GB | |
| Prompt scratch | planned at 480,000 bytes per pass token (2,048 tokens: 0.98 GB); measured: 0.58 GB of stream/projection rows plus the grouped-expert rows | about 1 GB |
| Kept free | pipelines, attention chunk parts, driver | 0.54 GB |
Draft head (with LEXI_INKLING_DRAFT) |
Q8_0 matrices, the 257-row fold table, 8-bit context rows for 262,144 positions, scratch | 1.45 GB measured (the plan counts 1.38 GB) |
MemoryPlan (in inkling_vulkan.rs) puts every weight and cache row on the first device and takes the largest pass
of 2,048 / 1,024 / 512 / 256 tokens that fits beside them; a context whose weights and caches do not fit is refused
with these figures (the runtime never moves cache rows to system memory). On the RX 7900 XTX (25.73 GB reported free)
262,144 positions load with passes of 2,048, and so do 524,288 positions (5.02 GB of caches; loaded and measured
2026-11-21, below); 1,048,576 need 9.85 GB (refused on a 24 GB card, below). The unit tests
caches_at_262144_positions and the_plan_takes_the_largest_pass_that_fits_and_refuses_what_cannot hold this
arithmetic, including 524,288 on the XTX's reported 25.73 GB.
Measured load (2026-11-21, lexi-engine --backend vulkan -c 262144, commit 199a3602): "weights 17.62 GB, caches 2.60
GB (262144 positions), scratch 0.58 GB (passes of 2048); loaded in 13.8 s".
| Item | Bytes |
|---|---|
| Weights | 17.62 GB |
| Global caches: 8 slots x keys and values x 524,288 positions x 8 heads x 72 bytes | 4,831,838,208 |
| Windowed caches (as at 262,144) | 188,743,680 |
| Caches together | 5.02 GB |
| Prompt scratch, passes of 2,048 (the plan takes 2,048 on the RX 7900 XTX) | 0.58 GB measured |
| Draft head at this capacity (its global layer's 8-bit context rows double; passes of 1,024 then, scratch 0.30 GB) | 2.00 GB measured |
1,048,576 positions would need 9.85 GB of 4-bit caches beside 17.62 GB of weights, about 28.1 GB with scratch and margin: more than a 24 GB card holds, so the plan refuses it (it would take a larger card or weights kept off the card).
Measured load at 524,288 (runner, LEXI_INKLING_CONTEXT=524288 LEXI_INKLING_CHUNK=2048): "weights 17.62 GB, caches
5.02 GB (524288 positions), scratch 0.58 GB (passes of 2048); loaded in 13.1 s"; the card showed 23.73 of 25.75 GB
in use during the prompt.
lexi-engine --backend vulkan -m Inkling-2-Mini-NVFP4exp-Q8.gguf -c 262144 --host 127.0.0.1 --port PORT
- The chat completions endpoint
/v1/chat/completions, streaming and not, with the file's tokenizer and chat template. The template prints"}}"inside an expression; the engine's template lexer ends a tag only outside string literals since 2026-11-21 (before, the template failed to parse). - One sequence at a time; prompts are handed to the runtime one pass at a time (a long prompt can be cancelled
between passes). Refused on this runtime: other backends, plan-only, decisions, activation capture, embeddings,
adapters, saved profiles,
--parallelabove 1. -
Prompt reuse (since 2026-11-21, append-only): a prompt that starts with every token the caches hold keeps them
and runs only its new tokens; otherwise a prompt that starts with the previous request's whole prompt (a
conversation's next turn, whose rendering of the previous answer differs from the generated tokens) goes back to a
checkpoint taken when that request generated its first token; anything else runs from position 0. The checkpoint
holds what later positions overwrite: the short convolutions' last 3 input rows per slot (3.8 MB) and the windowed
slots' rows of the 1,024 positions before it (47 MB with 4-bit rows), copied to system memory; the global slots keep
every position anyway. The reused count is reported as
usage.prompt_tokens_details.cached_tokens. Checked with the runner'sresumemode (A 500 tokens, then 5,003 tokens that wrap the windowed rings, then A + a new tail and A + those tokens + the tail): both continuations bitwise equal to the same pieces run from a reset, with unrounded and with 4-bit caches; with every kernel in its per-token exact form also equal to a fresh run of the whole sequence (KL 0). With the default kernels a fresh run of the whole sequence differs in the last digits (last-row KL up to 2.5e-3, same top token) because other pass boundaries send some tokens to the binary16 prompt kernels. Two served chat turns at 262,144: turn 2 reported 33 cached prompt tokens (the whole first prompt). - The template does not read
add_generation_prompt: the prompt ends after the last message and the model opens its own turn. It acceptsreasoning_effort(template variable; default 0.9). - The model writes its answer in turn tokens:
<|message_model|><|content_thinking|>thought<|end_message|> <|message_model|><|content_text|>answer<|end_message|>. The chat endpoint returns the thought asreasoning_contentand the answer ascontent, streaming and not (since 2026-11-21; checked: "What is 17 * 23?" gives the arithmetic inreasoning_contentand "391" incontent, the same streamed and not). - The Panel lists the model when the engine's architecture report lists
inkling2_loop(Vulkan lane). A Lexi Engine instance's settings carryINKLING_DRAFT(the draft head's checkpoint folder, passed asLEXI_INKLING_DRAFT; Vulkan only),LORA_ADAPTER/LORA_SCALE(one GGUF adapter at start) andLE_DECISION_TIMEOUT_MS(withLE_DECISIONS=1), checked when the settings are saved (since 2026-11-21).
lexi-engine --decisions (and --decision-admin with operator credentials, --decision-artifact-bank DIR with a
0700 folder) serves /v1/decisions, /v1/decisions/batch, /v1/decisions/capabilities and the admin routes
(/admin/decisions/calibrate/l1, /admin/decisions/fit/l2, artifacts) on this runtime, through the same Decider
the CPU runtime uses.
- The primitive (
InklingVulkanRuntime::decision_capture): run the prompt and return the last token's full logits (after the soft cap) and its stream rows after chosen layer applications. Refused while a generation is active. A prompt that extends the held tokens runs only its tail; each decision prompt's end is checkpointed, so a label's further tokens and the next label continue from it instead of running the prompt again. - Shape: 48 blocks of 2,304 values, feature stage
stream_after_application_before_output_mix. Block b is layer application b in execution order: 0-5 the layers before the core, 6-17, 18-29 and 30-41 the core's three passes, 42-47 the layers after it. The value is the stream (the running sum of the current residual block) of the last prompt token right after that application, before the output mix and the final norm. Artifacts bind to this numbering. - Label scores: the CPU runtime's arithmetic (log-softmax in f64 of each label token after the prompt extended by the label's earlier tokens).
- Capabilities: next-token log-probabilities, label-span scoring, hidden-state capture, last-token pooling; no early stop (every capture runs the whole forward), no shared-prefix batching.
- Identity: the fingerprint of the file's metadata with the SHA-256 of its bytes as the stable source identity
(
gguf-sha256:...; taken from a saved fingerprint when the file's size and time match, else hashed at start, about half a minute for 17.6 GB), and the tokenizer metadata's digest. - Limits of the API, not of the runtime: a decision state is at most 64 KiB and a rendered decision prompt at most
256 KiB (about 16,000 and 64,000 tokens). The limits stay: they bound what one request may make the server read,
parse and hold. That is why a decision over long context (a question after a 261k-token conversation) needs another
entry point: one that asks about a context the runtime already holds (the prompt-reuse checkpoint of the running
conversation) instead of resending it as state text. That entry point does not exist yet; the runner's
decide-depthmode checks the primitive at that depth. - Time limit:
--decision-timeout-ms N(default 30,000) bounds every decision and admin operation. One that outlives it is cancelled: the runtime stops at its next model call (for a decision prompt of a few hundred tokens, within about half a second), the operation fails without changing the artifact bank, and only then does the request answer 408 ("... timed out after N ms and was cancelled; nothing was changed"), so the next request finds the server free. L1 calibration on this runtime takes minutes for a few dozen rows (measured below): set the timeout to fit, for example 1,800,000. - Artifact bank (
--decision-artifact-bank DIR): owned through an exclusive lock onDIR/.active-bank.lockthat the kernel holds while the server runs. A second server on the same folder is refused; when the owner stops, however it stops, the kernel releases the lock and the next start takes the folder with no file to clear.
lexi-engine --lora FILE (several allowed, each with its scale as the generic path parses it) applies GGUF LoRA
adapters (llama.cpp's convention as lexi-engine-lora reads it: general.type = adapter, adapter.type = lora,
adapter.lora.alpha, F32 pairs blk.N.attn_{q,k,v,output}.weight.lora_a [input, rank] and .lora_b [rank,
output]) to the attention projections q, k, v and output of each physical layer; the core layers' terms apply on
all three passes. Other targets and embedding adapters are refused; the rank must be a multiple of 4. The delta is
scale x alpha / rank x B (A x), computed beside the Q8_0 product as two F32 row products and an add, in the prompt
and the single-token paths alike; adapters on one projection add. inkling-lora-synth writes a synthetic adapter for
checks (fixed seed; --zero for B = 0), and inkling-reference --gguf FILE --lora ADAPTER computes the same terms on
the CPU.
| Switch | Where | Effect |
|---|---|---|
LEXI_INKLING_PASS=N |
server | prompt pass of N tokens (a multiple of 16, at most 2,048) instead of the memory plan's choice |
LEXI_INKLING_DRAFT=DIR |
server, runner spec
|
load the draft head from the checkpoint folder DIR (speculative decoding for greedy requests) |
LEXI_INKLING_EXPERT_LANES=1 |
runtime | per-token experts with a row per lane (the kernel before 2026-11-21) |
LEXI_INKLING_SPEC_RUN=N |
runtime | tokens of a verification pass (2 to 8; default 8 below 131,072 positions, 4 beyond) |
LEXI_INKLING_WINDOW_CORES=0 |
runtime | prompt attention of windowed layers on the scalar kernel |
LEXI_INKLING_ATTN_CHUNKS=0 |
runtime | single tokens walk all their keys in one workgroup per head (the kernel before 2026-11-21) |
LEXI_INKLING_ATTN_CORES=0 |
runtime | prompt attention of global layers on the scalar kernel |
LEXI_INKLING_EXPERT_CORES=0 |
runtime | prompt experts per (token, route) instead of grouped matrix products (slow; for checks) |
LEXI_INKLING_FAST_ROWS=0 |
runtime | dense products of passes on the exact row kernels instead of the matrix-core Q8_0 kernel |
LEXI_INKLING_PROFILE=1 |
runtime | waits after every stage and records device time per stage (slower; not a speed figure) |
LEXI_INKLING_FUSED_CONTROL=0 |
runtime | the controllers as five commands each (pair, three F32 row products, apply) instead of two commands per set |
LEXI_INKLING_ATTN_REGACC=0 |
runtime | matrix-core prompt attention with the sums through shared memory every key block (the kernel before) |
LEXI_INKLING_ATTN_Q32=0 |
runtime | prompt attention with 16 queries a workgroup also for passes of a multiple of 32 tokens |
LEXI_INKLING_W4A8=1 |
runtime | runs of up to 8 tokens: 4-bit experts' gate and up products with 8-bit inputs (integer dot products); off by default |
LEXI_INKLING_SPEC_POLICY=adaptive |
runtime | verification length chosen per step by expected tokens per millisecond instead of by depth |
LEXI_INKLING_STEP_POSITION=0, LEXI_INKLING_REPLAY=0
|
runtime | positions in the recorded commands / every token's commands recorded again |
LEXI_INKLING_INPUT8=experts|projections|all |
runtime | accuracy ablation: those inputs rounded to 8 bits per block of 32 before the F32 kernels (no speed change) |
LEXI_INKLING_CONTEXT=N |
runner | cache positions (default the longest sequence plus 512) |
LEXI_INKLING_CHUNK=N |
runner | tokens per prompt pass (default 256; 2,048 for speed runs) |
LEXI_INKLING_PROMPT_RUNS=N |
runner bench
|
prompt repetitions (default 2; the last is the warm one) |
LEXI_INKLING_KV=f32 |
runner | unrounded cache rows, for checking the arithmetic only |
GGML_VK_ALLOW_SYSMEM_FALLBACK=0 stays set for every run that loads a model.
The checkpoint ships safetensors with 40 expert tensors NVFP4 (E2M1 values, an E4M3 scale per 16, a per-expert
amax and denominator), 8 MXFP8, the rest BF16 / F32, and no model code. llama.cpp 29b52d398 (build 12600, the first
with Inkling 2) converts it by decoding the experts to F32 and re-quantizing them: at 8 bits that file is 27.8 GB
and does not fit a 24 GB card. The patch at the end of this page (llama.cpp-29b52d398-inkling2.diff) adds three things:
-
conversion/inkling2.py: withINKLING2_NATIVE_NVFP4=1the NVFP4 experts are repacked into ggml NVFP4 blocks unchanged (gate and up rows de-interleaved into the fused tensor) and the per-expert factor goes into a.scaletensor; the products' inputs stay at full precision (prec_a4false) unlessINKLING2_A4=1. -
src/llama-model.cpp: the loop's extra cache slots share the physical layer's expert scale tensors. Without it llama.cpp runs passes 2 and 3 of the core with unscaled experts and produces garbage with this file (upstream does not hit it because its own conversion writes no scale tensors). -
src/llama-graph.cpp:LLAMA_NVFP4_ACC_F32keeps F32 accumulation for NVFP4 products (diagnostic).
Conversion (the trunk type follows --outtype; the run that wrote the measured file used q8_0, its exact command
line was not recorded):
git -C llama.cpp checkout 29b52d398 && git -C llama.cpp apply llama.cpp-29b52d398-inkling2.diff
INKLING2_NATIVE_NVFP4=1 python3 llama.cpp/convert_hf_to_gguf.py CHECKPOINT_DIR --outtype q8_0 --outfile Inkling-2-Mini-NVFP4exp-Q8.gguf
The result: 744 tensors, 17.6 GB, architecture inkling2_loop. The same file runs in LexiEngine and in the patched
llama.cpp.
The CPU reference (crates/model-inkling) is a port of llama.cpp 29b52d398's inkling2 / inkling2-loop graph. It
reads the vendor checkpoint directly (experts decoded exactly) or, with --gguf, the converted file's weights:
cargo build --offline --locked --release -p lexi-engine-model-inkling -p lexi-engine-model-maxwell -p lexi-engine-server --features vulkan --example inkling_vulkan
inkling-reference CHECKPOINT_DIR IDS OUT [--gguf FILE] [--cache-4bit] [--inputs exact|fp4] [--last N] [--score BASELINE]
maxwell-compare BASELINE OTHER IDS # KL, same top token and perplexity of two logits files for the same ids
Token files hold one sequence per line, comma-separated ids. Checks after a change of the runtime or its kernels:
| Check | Command | Must give |
|---|---|---|
| Arithmetic |
LEXI_INKLING_KV=f32 inkling_vulkan logits GGUF IDS OUT, then maxwell-compare REF OUT IDS against inkling-reference ... --gguf GGUF
|
mean KL 0.00000 |
| Same prompt twice | inkling_vulkan prefill GGUF IDS OUT |
"same prompt twice identical: true" |
| Second sequence after a reset |
inkling_vulkan logits GGUF IDS OUT with two lines in IDS, compared row by row with each line run alone |
equal |
| 4-bit accuracy |
inkling_vulkan logits and passes on 2 x 512 tokens against the vendor-checkpoint reference, second halves (510 rows) |
about llama.cpp's figure (0.0037) |
| Old and new kernels | the same command with LEXI_INKLING_ATTN_CHUNKS=0 LEXI_INKLING_ATTN_CORES=0
|
same generated tokens |
| GPU kernel tests | cargo test -p lexi-engine-vulkan-ops --features vulkan --test maxwell_attn --test maxwell_hc --test qwen4exp_exact_rows |
pass (shared row kernels) |
The reference's --inputs fp4 mode (4-bit inputs to the 4-bit experts, as the vendor requires) is a reading of the
checkpoint's quantization config; there is no vendor code to check it against, and it is not verified.
Accuracy against the exact CPU reference (vendor checkpoint, F32 weights as stored, F32 caches), text = a 2 x 512-token document, the 510 rows llama-perplexity scores:
| Run | Commit, date | Mean KL | Median | Same top token |
|---|---|---|---|---|
| LexiEngine single tokens | 2c0f15fe kernels, 2026-11-21 | 0.00385 | 0.00011 | 98.24% |
| LexiEngine prompt kernels (16-token passes) | 2c0f15fe kernels, 2026-11-21 | 0.00338 | 0.00009 | 99.22% |
| LexiEngine single tokens | 2ac529bd, 2026-11-20 | 0.00425 | 0.00009 | 98.24% |
| LexiEngine prompt kernels | 2ac529bd, 2026-11-20 | 0.00343 | 0.00010 | 98.82% |
llama.cpp 29b52d398 + patch, -ctk q4_0 -ctv q4_0
|
2026-11-20 | 0.00371 | 0.00007 | 98.82% |
Arithmetic (unrounded caches, against the reference computed from the same file): 48 tokens KL 0.00000 (top logits equal to 7 digits); 1,024 positions one at a time with the split-key attention KL 0.00000, worst 0.000 (the earlier single walk: 0.00002, worst 0.002), 2026-11-21.
Speed: see the numbers recorded for each commit in the table below. Single runs; "prompt" is the warm second prompt in the process unless stated.
| Commit, date | Prompt 4,096 | Prompt 16,384 | Generation at 4,096 | Generation at 16,384 | Prompt 261,632 (first in process) | Generation at 261,632 |
|---|---|---|---|---|---|---|
| 2ac529bd, 2026-11-20 | 1,531 tok/s | 1,324 | 35.1 | 31.8 | 385.6 (678.5 s) | 11.0 |
| e2184ab5, 2026-11-21, idle box | 1,676.4 | 1,568.9 | 39.92 | 38.96 | 791.0 (330.8 s) | 33.26 |
| 394bb21b, 2026-11-21, idle box | 2,015.6 | 1,877.5 | 43.83 | 42.78 | 851.8 (307.2 s) | 35.40 |
| a4c86246 (recorded forward), 2026-11-21, idle box | 2,004.1* | 1,884.2* | 50.36 | 49.12 | 868.3 (301.3 s) | 39.31 |
| this commit (controllers, prompt attention in registers with 32 queries), idle box | 2,244.5* | 2,147.8* | 52.37 | 50.93 | 1,145.7 (228.4 s) | 40.24 |
the same with LEXI_INKLING_W4A8=1
|
2,242.5* | 2,146.3* | 55.44 | 53.88 | see below | see below |
* one cold prompt per process (LEXI_INKLING_PROMPT_RUNS=1), 128 generated tokens. At 261,632: load 17.62 + 2.60 +
0.58 GB in 12.9 s. The prompt at 261,632 by attention kernel (switches; the first figure on the round's previous build, which also
held the fused mix and norm, neutral for prompts): sums through shared memory 297.9 s (878.2 tok/s), sums in
registers with 16 queries 274.7 s (952.5), with 32 queries 228.4 s (1,145.7).
Speculative decoding with the draft head (LEXI_INKLING_DRAFT=src inkling_vulkan spec GGUF IDS COUNT, greedy,
2026-11-21; the speculative tokens are identical to plain greedy decoding in every run). The commit after 394bb21b
(the draft attention's key tiles padded against shared-memory bank conflicts) changes only the draft head:
| Prompt | Commit | Plain | Speculative | Tokens per verification pass | Per step: proposing / verifying |
|---|---|---|---|---|---|
| 16,384 tokens, 256 generated | 394bb21b | 42.65 tok/s | 78.64 tok/s | 6.29 (runs of 8) | 8.9 / 69.4 ms |
| 16,384 tokens, 256 generated | after 394bb21b | 42.52 tok/s | 79.06 tok/s | 6.29 (runs of 8) | 7.7 / 70.2 ms |
| 261,632 tokens (262,144 positions), 128 generated | 394bb21b | 35.36 tok/s | 37.40 tok/s | 3.28 (runs of 4) | 27.5 / 59.5 ms |
| 261,632 tokens (262,144 positions), 128 generated | after 394bb21b | 35.68 tok/s | 40.88 tok/s | 3.28 (runs of 4) | 19.4 / 60.1 ms |
| 261,632 tokens (262,144 positions), 128 generated | this commit | 36.55 tok/s | 41.84 tok/s | 3.20 (runs of 4) | 16.7 / 58.9 ms |
At 261,632 with runs of 8 (4c9637ed and earlier builds): 30.2 tok/s, slower than plain; runs of 3: 33.2. Served at
262,144 (lexi-engine, the Fibonacci request at temperature 0, prompt included; 394bb21b): 42.3 tok/s without the draft
head, 49.5 with it, the same answer.
At 524,288 positions (2026-11-21, code of the controllers / prompt-attention commit; prompt p523264.ids: the
engine's source text of the 262,144 prompt followed by 34 Panel source files, 523,264 tokens; r524k10.ids: the
same length with ten planted facts at 1%, 10%, ..., 80%, 99%):
| Measurement | Result |
|---|---|
| Load (runner, passes of 2,048) | weights 17.62 GB, caches 5.02 GB, scratch 0.58 GB, 13.1 s |
Prompt 523,264 tokens (bench) |
682.6 s = 766.6 tok/s |
| Generation at depth 523,264, 64 tokens | 33.71 tok/s (29.25 ms a token) |
Retrieval, ten facts (ask) |
10 of 10; that prompt 690.2 s = 758.1 tok/s |
Draft head (spec, passes of 1,024, 2.00 GB) |
speculative 32.07 tok/s against 31.09 plain with the draft loaded (3.37 tokens a step; proposing 26.3 ms, verifying 78.0 ms); identical to plain greedy |
Served (lexi-engine -c 524288, greedy chat, 1,143 tokens, prompt included) |
43.9 tok/s without the draft head (passes of 2,048), 51.0 with it (the plan takes passes of 1,024); the same answer |
At this depth the draft head does not beat plain decoding without it (32.1 against 33.7 tok/s): its own attention over
the whole context costs about 26 ms a step. Commands: LEXI_INKLING_CONTEXT=524288 LEXI_INKLING_CHUNK=2048 LEXI_INKLING_PROMPT_RUNS=1 inkling_vulkan bench GGUF p523264.ids 64; LEXI_INKLING_CONTEXT=524288 inkling_vulkan ask GGUF r524k10.ids 8; LEXI_INKLING_DRAFT=src LEXI_INKLING_CONTEXT=524288 LEXI_INKLING_CHUNK=1024 inkling_vulkan spec GGUF p523264.ids 128.
Commands for the e2184ab5 row: LEXI_INKLING_CHUNK=2048 inkling_vulkan bench GGUF pN.ids 64 (two prompts, the second
warm), and LEXI_INKLING_CONTEXT=262144 LEXI_INKLING_CHUNK=2048 LEXI_INKLING_PROMPT_RUNS=1 inkling_vulkan bench GGUF p261632.ids 64 (load: weights 17.62 GB, caches 2.60 GB, scratch 0.58 GB, 12.9 s).
Retrieval at depth (runner mode ask, e2184ab5 and again 394bb21b, 2026-11-21): five facts ("the access code for vault NAME is NNNNNN")
planted at 2%, 25%, 50%, 75% and 97% of a 261,120-token prompt at 262,144 positions, then asked back one after another
as plain text: 5 of 5 answered exactly (the same with an 8,192-token prompt). A second prompt with ten facts at 1%, 10%, 20%, ..., 80% and 99% of 260,864 tokens (the commit after 394bb21b):
10 of 10 answered exactly. Again with the prompt attention in registers and 32 queries a workgroup (this commit): 10 of
10, the 260,864-token prompt in 230.1 s (1,133.6 tok/s).
Typed decisions (2c0f15fe kernels plus the decision commit, 2026-11-21):
- The primitive against the CPU reference, 48 tokens, every kernel exact (
LEXI_INKLING_FAST_ROWS=0,LEXI_INKLING_EXPERT_CORES=0,LEXI_INKLING_ATTN_CORES=0,LEXI_INKLING_WINDOW_CORES=0), unrounded caches: stream rows after applications 0 / 5 / 23 / 41 / 47 differ by at most 3.8e-6 / 3.8e-5 / 1.5e-5 / 3.4e-5 / 1.5e-5 (largest values 9.9 / 176 / 131 / 50 / 36.5), logits by 1.3e-5 (largest 27.6), the same top token. With the default binary16 prompt kernels the rows differ by up to 2.0 (of 50) and the logits by 0.28, the same top token. With 4-bit caches on both sides (the reference rounds its rows without the rotation, so the two round differently): up to 6.2 in a row and 1.7 in the logits, the same top token. - Served at 262,144 positions (
--decisions --decision-admin), raw level on the labelled holdout rows: yes/no on product facts 28 of 48, four options (largest number) 21 of 32, yes/no on Rust facts 6 of 16; probabilities sum to 1 within 2e-16; about 0.43 s a decision. Fitted on the separate fit rows through the admin API (operator key) and used through/v1/decisionsat levelsl1andl2, holdout correct (raw / l1 / l2): product facts 28 / 24 / 24 of 48, four options 21 / 30 / 25 of 32, Rust facts 6 / 15 / 8 of 16. L1 calibration took 80.5 / 117.5 / 29.2 s (48 / 32 / 16 rows) and an L2 fit 20.7 / 15.3 / 7.5 s, so the admin calls need--decision-timeout-msabove its 30 s default (at 30 s, before the change described in section 6, the call answered 408 while the calibration kept running, and further operations answered 429decision_busy). The API refused, as designed: a 70 KiB state (400, "decision state text is 71680 bytes, limit is 65536"), an L2 fit with 2 rows (400, "L2 requires at least 8 labeled states for this question"), calibration without the operator key (403). The L2 heads on 48 and 16 rows did not beat raw scoring on the yes/no sets: small fit sets. - After a 261,120-token prompt (runner mode
decide-depth; the prompt in 300.8 s, then about 0.95 s a question with the prompt's checkpoint restored for each label): four yes/no questions about facts planted in the prompt, 3 of 4 right (true claims 0.748 and 0.823 yes; a false claim 0.431 yes, right; another false claim 0.585 yes, wrong).
LoRA (synthetic rank-16 adapter from inkling-lora-synth, scale 1, 2026-11-21): device against the reference with
the same adapter, 48 positions one at a time, unrounded caches: KL 0.00000, the same top token throughout; the binary16
prompt kernels (16-token passes) against it: KL 0.00018; the adapter's own effect against the base model KL 0.00095
(same top 97.92%). A zero adapter (B = 0) gives output bitwise equal to the base, single tokens and 16-token passes
(4-bit caches). Cost with a 4,096-token prompt (one cold prompt per process): prompt 1,843.9 tok/s against 2,042.2
without (-9.7%), generation 44.29 against 43.94 (no measurable cost). Served with --lora at 262,144 positions: /health lists the adapter with its
SHA-256, and the chat answer is unchanged in form (reasoning and content separated).
Host recording against replay (runner mode replay-ablation, 64 decode steps at one position, each step's command
buffers either recorded as usual or replayed from a recording): at 4,096 positions 22.42 ms a step recorded, 19.25 ms
replayed (22.38 recorded again); at 16,384: 22.92 / 19.84 / 22.93. About 3.1 ms a token (14%) is host-side command
recording: the size of the gain available from recording a token's whole forward once. With the recorded forward
(position word and replay, the commit after the decision commits): 128 generated tokens at 4,096 positions 50.25
tok/s (43.54 recording every token), at 16,384 49.12 (42.47), the generated ids identical, logits bitwise equal.
Per-token work, 2026-11-21 evening, each change alone against the same baseline in one binary (switches; 4-bit caches; 128 generated tokens after a 4,096 / 16,384-token prompt, one cold prompt per process; accuracy = 4-bit run against the vendor reference, 510 rows, single tokens / 16-token passes):
| Change (switch) | Generation 4k / 16k | Prompt 4k / 16k | Accuracy | Kept |
|---|---|---|---|---|
| none (baseline) | 50.36 / 49.12 tok/s | 2,004 / 1,884 tok/s | 0.00419 / 0.00351 | |
controllers in two commands per set (FUSED_CONTROL) |
52.31 / 50.89 | 2,101 / 1,973 | 0.00428 / 0.00343 | yes |
| mix and norm in one command per token | 50.18 / 48.73 | 2,018 / 1,896 | bitwise equal | no (slower) |
| per-token expert loops unrolled | 50.10 / 48.63 | 2,004 / 1,887 | bitwise equal | no (slower) |
| one-token history updates folded into cache-row and branch commands | 50.38 / 49.10 | 2,005 / 1,886 | bitwise equal | no (no gain) |
matrix-core prompt attention, sums in registers (ATTN_REGACC) |
unchanged | 2,094 / 1,975 | 0.00419 / 0.00359 | yes |
8-bit inputs to the 4-bit experts (W4A8, with the unrolled loop) |
52.81 / 51.37 | unchanged | 0.00372 / 0.00351 | option, off |
| the attention projections of one input in one command (bitwise equal) | 1.0 ms a token slower | unchanged | bitwise equal | no (slower) |
32 queries a workgroup in prompt attention (ATTN_Q32, on top of the kept ones) |
unchanged | 2,245 / 2,148 against 2,200 / 2,070 | 16-token passes unaffected | yes |
The 8-bit input rounding alone (LEXI_INKLING_INPUT8, same kernels): 0.00419 none, 0.00385 experts, 0.00371
projections, 0.00384 all: no measurable cost. W4A8 cannot pass the arithmetic check (KL 0 against the CPU reference
with unrounded caches): against the reference rounding the same inputs (inkling-reference --inputs q8) it gives KL
0.00024 at 48 positions and 0.00094 at 1,024, and two device paths that differ only in the rounding formula differ
by 0.00052, because an input that differs in its last bit flips a value at a rounding boundary by a whole step and
the loop carries it. So it stays off by default. At 261,632 positions REGACC takes the prompt from 878.2 to 959.9
tok/s (297.9 -> 272.6 s; the generated tokens after the prompt differ, the sums being added in another order).
The 2c0f15fe kernels at 16,384 tokens (2026-11-21, development A/B in one session, single prompt per process): prompt 1,573.5 tok/s and generation 37.35 tok/s, against 1,409.8 and 32.69 with the earlier kernels switched back on; the 64 generated tokens identical.
llama.cpp 29b52d398 + patch on the same file and card (llama-bench -ngl 99 -fa on -ctk q4_0 -ctv q4_0 -b 2048 -ub 512, 2026-11-20, a disk copy running beside it): prompt 425 / 397 tok/s at 4,096 / 16,384, generation 25.4 / 25.0.
At 262,144 it loads (about 22.3-22.9 GB on the card plus 3.9-4.5 GB reported in system memory) but did not finish a
261,632-token prompt in 55 minutes (two passes, llama-bench) nor a 129,374-token prompt in 45 minutes (one pass,
llama-cli, -c 131072); no rate exists for it at depth.
- Numerical accuracy is verified against the reference only at short context (up to 1,024 positions); at 16,384 the new and old attention kernels agree on 64 generated tokens; deep in the context the retrieval checks find 5 of 5 and 10 of 10 planted facts at about 261,000 tokens and 10 of 10 at 523,264 (still a small sample).
- Contexts measured: up to 524,288 positions on the 24 GB card (the draft head then forces 1,024-token passes and does not beat plain decoding); 1,048,576 is refused (section 5).
-
LEXI_INKLING_W4A8=1(8-bit expert inputs, about 1.1 ms a token faster) is off by default: no 8-bit-input path can pass the KL-0 arithmetic check against the reference. - The draft head is read from the vendor checkpoint folder (
LEXI_INKLING_DRAFT, config.json and mtp.safetensors): it is not in the converted GGUF (the Panel passes it through the instance settingINKLING_DRAFT). Deep in the context its gain is small (41.8 vs 36.6 tok/s at 261,632) and at 523,264 it is slower than plain decoding without it (32.1 vs 33.7): its own attention over the whole context costs about 17 and 26 ms a step there. The switch from runs of 8 to runs of 4 at 131,072 positions is not measured between 16,384 and 261,632. - The draft head is used for greedy requests only (sampled requests decode plainly; the scheduler's sampled speculative step is not implemented for this runtime). Its windowed context rows are not part of the prompt-reuse checkpoint, so right after a restored turn its first proposals may be worse (the output is unaffected).
- The vendor's 4-bit inputs to the 4-bit experts are not implemented; the runtime uses binary16 (prompt passes) or F32 (single tokens) inputs there. Not checkable against vendor code.
- Vision (the image tower) is not implemented.
- The 8 MXFP8 expert tensors are Q8_0 in the file (re-quantized by the converter), not on the vendor's grid.
- Raw completions (
/v1/completions) show the model's turn tokens (<|content_thinking|>and so on) as text: the runtime's tokenizer spells them so the chat endpoint can put the thought inreasoning_contentand the answer incontent. - Decisions through the API are bounded by its 64 KiB state and 256 KiB prompt limits (a decision after a 261k-token context is checked through the runner only; section 6).
- One sequence at a time; prompt reuse only for a prompt that extends what the caches hold or starts with the previous request's whole prompt (section 6); 4-bit cache rows only (no 8-bit format).
Changes to llama.cpp (MIT) commit 29b52d398; apply with git apply.
Patches to llama.cpp 29b52d398 ("model : Inkling 2 Mini", build 12600), MIT licensed upstream code, used to
convert Inkling-2-Mini with its experts on the vendor's 4-bit grid and to run that file as the comparison baseline.
Apply with: git -C llama.cpp checkout 29b52d398 && git -C llama.cpp apply llama.cpp-29b52d398-inkling2.diff
See ../runtimes/INKLING-2-VULKAN-20261121.md, section 8.
diff --git a/conversion/inkling2.py b/conversion/inkling2.py
index 63e73a6d5..599c864ed 100644
--- a/conversion/inkling2.py
+++ b/conversion/inkling2.py
@@ -12,6 +12,8 @@ import torch
if TYPE_CHECKING:
from torch import Tensor
+import os
+
from .base import LazyTorchTensor, ModelBase, MmprojModel, TextModel, gguf, logger
@@ -158,6 +160,38 @@ class Inkling2Model(TextModel):
raise ValueError(f"Inkling2: {name}_scale: unsupported stored block shape {block}")
if list(block) != config.get("weight_block_shape"):
logger.warning("Inkling2: %s: hf_quant_config.json weight_block_shape %s conflicts with stored weight/scale shapes %s/%s; using %s", name, config.get("weight_block_shape"), tuple(tensor.shape), ss, list(block))
+ if nvfp4 and os.environ.get("INKLING2_NATIVE_NVFP4") == "1":
+ # Local addition: keep the packed experts on their own grid. The 4-bit values and the E4M3 scale per
+ # 16 values are repacked into ggml's NVFP4 blocks unchanged; the per-expert factor amax / denominator
+ # goes into the per-expert scale tensor the MoE step applies.
+ amax_t, den_t = (LazyTorchTensor.to_eager(self.model_tensors[n]()) for n in companions[1:])
+ den = float(den_t.float().item())
+ packed = LazyTorchTensor.to_eager(tensor).view(torch.uint8)
+ scales = LazyTorchTensor.to_eager(scale_tensor).view(torch.uint8)
+ if bool(((scales & 0x80) != 0).any()) or bool(((scales & 0x7F) == 0x7F).any()):
+ raise ValueError(f"Inkling2: {name}_scale: negative or NaN E4M3 scale, cannot keep as unsigned")
+ blocks = []
+ for e in range(experts):
+ w, sc = packed[e], scales[e]
+ if w13:
+ # stored rows alternate gate and up; the fused tensor holds the gate rows, then the up rows
+ w, sc = torch.cat([w[0::2], w[1::2]]), torch.cat([sc[0::2], sc[1::2]])
+ raw, _ = self._nvfp4_pack(w, sc)
+ blocks.append(raw)
+ merged = np.stack(blocks, axis=0)
+ target = name.replace(".w13_weight", ".gate_up_proj").replace(".w2_weight", ".down_proj")
+ new_name = self.map_tensor_name(target, try_suffixes=()) + ".weight"
+ logger.info(f"Repacked {new_name} natively as NVFP4, shape [{experts}, {rows}, {cols}]")
+ self.gguf_writer.add_tensor(new_name, merged, raw_dtype=gguf.GGMLQuantizationType.NVFP4)
+ self._write_scales_tensor(new_name.replace(".weight", ".scale"), [float(a) / den for a in amax_t.float().tolist()])
+ # Inputs of these products stay at full precision unless INKLING2_A4=1 (4-bit inputs in the backend
+ # gave garbage with this file on Vulkan, 2026-11-20).
+ if os.environ.get("INKLING2_A4") != "1":
+ self._prec_a4[new_name] = False
+ del blocks, merged
+ for n in [name] + required:
+ del self.model_tensors[n]
+ continue
amax = denominator = None
if nvfp4:
amax, denominator = (self.model_tensors[n] for n in companions[1:])
diff --git a/src/llama-graph.cpp b/src/llama-graph.cpp
index 269b920fb..833696ca8 100644
--- a/src/llama-graph.cpp
+++ b/src/llama-graph.cpp
@@ -1897,7 +1897,7 @@ ggml_tensor * llm_graph_context::build_lora_mm_id(
prec_policy.apply(res);
- if (w->type == GGML_TYPE_NVFP4) {
+ if (w->type == GGML_TYPE_NVFP4 && getenv("LLAMA_NVFP4_ACC_F32") == nullptr) {
ggml_prec_set_acc(res, GGML_PREC_BF16);
}
diff --git a/src/llama-model.cpp b/src/llama-model.cpp
index 4c77f6f68..0971c27a5 100644
--- a/src/llama-model.cpp
+++ b/src/llama-model.cpp
@@ -2006,6 +2006,19 @@ bool llama_model_base::load_tensors(llama_model_loader & ml) {
layer.nextn.shared_head_head_in_s = create_tensor(tn(LLM_TENSOR_NEXTN_SHARED_HEAD_HEAD, "input_scale", i), {1}, TENSOR_NOT_REQUIRED);
}
}
+
+ // local fix: cache slots that share a physical layer's expert tensors (looped models) were copied before
+ // the scales above were loaded, and no scale tensor carries their slot index: share the scales as well
+ for (int i = 0; i < n_layer_all; ++i) {
+ for (int j = 0; j < i; ++j) {
+ if (layers[j].ffn_down_exps && layers[j].ffn_down_exps == layers[i].ffn_down_exps) {
+ if (!layers[i].ffn_up_exps_s) { layers[i].ffn_up_exps_s = layers[j].ffn_up_exps_s; }
+ if (!layers[i].ffn_gate_exps_s) { layers[i].ffn_gate_exps_s = layers[j].ffn_gate_exps_s; }
+ if (!layers[i].ffn_down_exps_s) { layers[i].ffn_down_exps_s = layers[j].ffn_down_exps_s; }
+ break;
+ }
+ }
+ }
// output scales
if (output) {
// weight scaleLexiSuite · Local OS / Kernel / Engine / Panel · Development / testing; not production-qualified
Getting Started · Security · Troubleshooting · Release Notes · Licence
Verify release checksums, finish first-boot enrollment on a trusted network, and respect the current proprietary licence.
- Home
- Getting Started
- Install from USB
- Write a stick
- Install to an internal disk
- Install on a computer you already use: Linux, macOS, Windows
- Updating
- Ternary models (experimental)
- First Boot
- Secure Boot
- Kernels
- Hardware
- Overview / Home
- Navigation and Help
- Models
- Instances
- LexiChat
- Builds and Engines
- Optimizer
- Fit Lab
- Benchmarks and Evals
- Training
- Decisions
- TokenGate
- Inference Fabric
- Media Generation
- Agents and MCP
- Power and Hardware
- Network and Access
- Storage, Backup and Updates
- Files, Terminal and Passwords
- Diagnostics
- Fleet
- Virtual Machines
- LexiOS and Secure Boot
- macOS