Benchmarks

Benchmarks: LexiEngine against llama.cpp, and how to reproduce it

LexiEngine is measured against the newest llama.cpp release on the same card, with the same model file and the same settings, every time the engine changes. This page has the current results, the cases where llama.cpp is still ahead, the older results, and the method.

Current results (November 13 and 14, 2026)

The Qwen tables: LexiEngine version afaf544, November 13 (the Qwen path is unchanged in 14d173b). The Gemma tables: version 14d173b, November 14. Both against llama.cpp 1.0.0 (build b12359, released November 13, 2026; Vulkan, flash attention on the RX 7900 XTX; CUDA on the RTX 2060). One Radeon RX 7900 XTX (24 GB) capped at 291 W and one RTX 2060 (6 GB) in an Intel i7-9700 desktop on Linux 7.3, nothing else running beside any run, one slot, greedy. Generation is tokens per second after the first token.

Qwen3.8 27B Q4_K_M, 262,144-token context, 4-bit K/V cache, draft-head speculation on both engines

This is one fixed configuration, the one the model is served with every day (batch 2048/512, at most 2 drafted tokens). Every engine change is measured in this table before and after.

Prompt tokens	Generation, LexiEngine	Generation, llama.cpp 1.0.0	Prompt processing, LexiEngine	Prompt processing, llama.cpp 1.0.0
62	76.8 tok/s	65.3 tok/s	323 tok/s	180 tok/s
76 (256-token answer)	74.8 tok/s	63.4 tok/s	432 tok/s	190 tok/s
3,631	71.4 tok/s	57.0 tok/s	916 tok/s	725 tok/s
33,002	70.0 tok/s	55.8 tok/s	832 tok/s	605 tok/s
133,561	50.4 tok/s	device lost	604 tok/s
253,291	40.1 tok/s	no result	455 tok/s
261,896 (the full window)	39.7 tok/s	no result	445 tok/s

One request per row. llama.cpp 1.0.0 lost its Vulkan device (vk::Queue::submit: ErrorDeviceLost) while reading the 133,561-token prompt and returned nothing on the 261,896-token one. LexiEngine answered both. That is at this table's configuration; with a smaller batch (1024/256) llama.cpp 1.0.0 read the 133,561-token prompt at 319 tok/s and generated at 37.4 tok/s, and lost its device on the 253,291- and 261,896-token prompts as before. The version before this one (5db82d6) generated at the same speeds and read the 33,002-, 133,561- and 261,896-token prompts at 765, 489 and 335 tok/s.

An agentic coding session on the same configuration

32 turns on one conversation, about 2,400 new tokens a turn on a cached history that grows to about 71,500 tokens.

	LexiEngine	llama.cpp 1.0.0
Whole session	197 s	305 s
Time to first token, turn 1	2.5 s	3.3 s
Time to first token, turn 32	4.0 s	7.8 s
Generation at turn 32	57.2 tok/s	42.0 tok/s

A 9B Qwen-family model (Qwythos 9B Q4_K_M), 524,288-token context, no speculation

4-bit K/V cache, batch 2048/512, on both engines.

Prompt tokens	Generation, LexiEngine	Generation, llama.cpp 1.0.0	Prompt processing, LexiEngine	Prompt processing, llama.cpp 1.0.0
50	126.1 tok/s	106.5 tok/s	500 tok/s	485 tok/s
3,619	122.9 tok/s	103.3 tok/s	2,758 tok/s	2,719 tok/s
32,990	112.5 tok/s	93.0 tok/s	2,519 tok/s	2,186 tok/s
133,549	89.5 tok/s	67.5 tok/s	1,815 tok/s	1,053 tok/s
253,279	73.3 tok/s	50.9 tok/s	1,352 tok/s	622 tok/s

One request per row, greedy. LexiEngine is ahead at every depth and the lead grows with depth: 18% in generation on a short prompt, 44% at 253,000 tokens, where it also reads the prompt more than twice as fast (first token after 3 min 7 s against 6 min 47 s).

Gemma 4 31B Q4_K_M at a long context, 4-bit K/V cache

New in this version: Gemma 4 runs with the 4-bit K/V cache at contexts up to the model's 262,144 tokens. On a 24 GB card the 31B's weights (18.7 GB) leave room for a window of about 205,000 tokens; the engine computes that limit from the card and refuses a larger one with the figure. Measured at a 210,176-token context, batch 2048/512, with llama.cpp 1.0.0 started the same way (all layers on the GPU, its faster placement at this context):

Prompt tokens	Generation, LexiEngine	Generation, llama.cpp 1.0.0	Prompt processing, LexiEngine	Prompt processing, llama.cpp 1.0.0
3,595	31 to 33 tok/s	33.4 tok/s	817 tok/s	661 tok/s
32,966	29.0 tok/s	29.8 tok/s	548 tok/s	447 tok/s
133,525	no result in this version	21.4 tok/s		178 tok/s
201,371	no result in this version	18.2 tok/s		127 tok/s

The engine reads prompts 23% faster at both measured depths and generates 1% to 8% slower (its generation varied by about 7% between runs). Known fault in this version: on the 31B a prompt beyond about 130,000 tokens is read to the end and then stops with an out-of-memory refusal on its last short pass, so the two deep rows have no engine result; the cause is found (the last passes asked for new buffer sizes while the driver's free-memory figure lagged) and the correction is in test for the next version. llama.cpp 1.0.0 answered both, after 12 min 32 s and 26 min 30 s. At the full 262,144-token window neither engine fits the 31B in 24 GB: llama.cpp ran with part of its buffers in system memory, at 104 tok/s prompt and 15 tok/s generation.

Gemma 4 E4B at its full 131,072-token context with the same cache: generation 125.8 tok/s after 3,595 tokens and 108.4 after 32,966 (llama.cpp 1.0.0: 117.7 and 105.0); prompt processing 5,318 and 4,237 tok/s (llama.cpp 1.0.0: 5,715 and 4,283).

Default settings, both cards

4,096-token context, each engine's defaults, no speculation; prompts of about 50, 580 and 3,600 tokens.

Card, model	Measure	LexiEngine	llama.cpp 1.0.0
RX 7900 XTX, Qwythos 9B Q4_K_M	Generation	124 / 117 / 114 tok/s	108 / 106 / 104
	Prompt processing, 580 / 3,619 tokens	2,449 / 2,885 tok/s	1,701 / 2,719
RX 7900 XTX, Gemma 4 E4B Q4_K_M	Generation	139 / 134 / 131 tok/s	138 / 132 / 130
	Prompt processing, 556 / 3,595 tokens	3,782 / 5,688 tok/s	3,370 / 5,923
	Peak video memory	5.4 GB	3.2 GB
RX 7900 XTX, Gemma 4 31B Q4_K_M	Generation	35.4 / 34.7 / 33.7 tok/s	37.2 / 36.2 / 34.8
	Prompt processing, 556 / 3,595 tokens	561 / 684 tok/s	469 / 594
	Peak video memory	21.1 GB	19.7 GB
RTX 2060, Qwythos 9B Q4_K_M	Generation	47.4 / 46.7 / 42.1 tok/s	49.5 / 49.6 / 48.6
	Prompt processing, 580 / 3,619 tokens	373 / 345 tok/s	1,019 / 1,213
RTX 2060, Gemma 4 E4B Q4_K_M	Generation	67.9 / 64.1 / 63.0 tok/s	69.3 / 69.7 / 67.0
	Prompt processing, 556 / 3,595 tokens	1,161 / 1,241 tok/s	2,493 / 2,853

Where llama.cpp 1.0.0 is ahead: generation on Gemma 4 31B (3% to 5% at default settings), a 3,595-token prompt on Gemma 4 E4B (4%), and the RTX 2060, where LexiEngine's prompt processing on a 9B model is about a third of llama.cpp's and this version changed nothing. On Gemma 4 E4B generation is now level and a 556-token prompt is read 12% faster; in the previous version llama.cpp read Gemma prompts about twice as fast.

With the 27B's draft head on and the same settings on both engines at a 4,096-token context, LexiEngine generates at 82.3 / 71.8 / 58.9 tok/s against llama.cpp 1.0.0's 67.7 / 60.9 / 60.5, and its text with the draft head on is identical to its text with it off.

Method to reproduce a fair measurement

1. Record provenance first. Save model identity and content hash, GGUF quantization, engine build, backend and driver, kernel, CPU/GPU, power cap, free memory, KV-cache type, context, batch/slot count, sampling, MTP state and installed settings. If any differ, treat it as a different experiment.
2. Keep the machine idle and let thermal conditions stabilize. Close other GPU or CPU loads; note power and thermal policy. Do not mix benchmarks during maintenance, updates or background model downloads.
3. Use identical requests and generated-token limits. Test short, approximately 3.6k, 21k and — if memory supports it — 68k-context inputs. Use a new random prompt prefix for each request to prevent cache reuse; label approximate lengths with their actual token counts. Align draft-head speculation on both sides before comparison.
4. Repeat at least three times per routine test and record each run. Use the median, not the best result. For a one-run long-context measurement, label the result single-run instead of implying statistical confidence.
5. Separate metrics: (a) time to first token in seconds, (b) prompt processing in tok/s, (c) generation in tok/s excluding the first-token delay. Never compare incompatible denominators.
6. Assess noise and stability. Track per-run values, warm/cold behavior, variance, memory/power state and failure count. Calculate noise bands/confidence intervals from replicates where possible; do not claim statistical significance with three or fewer observations by default. Reproduce any changed result before accepting an optimization.
7. Check output quality with matched prompts, held-out evaluations, perplexity/KLD where supported, and deterministic sampling where possible. Speed gained by changing output or truncating work is not a like-for-like win.
8. Export a complete evidence record and compare through Bench and Evaluate, including device/context depth curves and quality gates; never substitute self-reported peaks for representative median trials.

Bench/Panel workflow

• Open Panel Benchmarks and Evals to inspect run results, context-depth curves, comparisons, histograms and evaluation history.
• Review Optimizer for noise-aware search, Fit Lab for quantization and KLD checks, and Power and Hardware for power cap/thermal conditions.
• The engine exposes lexi-engine bench, topology, lexify-plan, quantize and capture-activations operations; check the installed build's help before selecting flags. A command existing in the engine does not imply a corresponding Panel form is available on every kernel.

Controls and interpretation

Control / measurement	What it does
Bench / Evaluate	Runs or reviews performance and held-out quality checks; compare only matching model and hardware conditions.
Optimizer	Searches parameters with measured noise margins; candidate improvements require repeatability.
Power cap	Limits device power; altering it invalidates direct comparison with results recorded under another cap.
MTP on/off	Enables/disables draft-head token proposals; changing it changes the workload and measurement.
KV-cache / context / slots	Changes memory use and compute depth; keep all three fixed when comparing engines.
TTFT / prefill / decode rates	Measure distinct phases; do not combine or conflate their units.

See Hardware, Security and Release Notes.