Skip to content

docs(FEATURES): Vulkan is at parity with llama.cpp, not behind it (#310) - #311

Merged
localai-bot merged 1 commit into
mainfrom
row/DOCS-VULKAN-STATE
Aug 10, 2026
Merged

docs(FEATURES): Vulkan is at parity with llama.cpp, not behind it (#310)#311
localai-bot merged 1 commit into
mainfrom
row/DOCS-VULKAN-STATE

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

docs(FEATURES): Vulkan is at parity with llama.cpp, not behind it (#310)

docs/FEATURES.md said Qwen3.6-27B decodes 4.24 tok/s vs llama.cpp's 4.35.
That is the pre-#191 number: it claimed we were BEHIND when the binding
measurement is parity MET, and it contradicted the three other public surfaces,
all of which were reviewed and landed saying 4.36:

docs/STATUS.md decode 4.36/4.35 MET
docs/BENCHMARKS.md decode 4.36 vs 4.35, MET (7 clean legs)
README.md decodes 4.36 vs llama.cpp Vulkan 4.35 tok/s

Corrected, and it keeps saying the pass is NARROW: the binding figure is the
median of 7 clean legs at TPOT 228.66-230.25 ms, and the 0.69% leg spread IS the
noise floor. The detail stays in BENCHMARKS and the record rather than being
restated here.

Also adds the load-phase result the paragraph was missing beside its memory one
(ENG-LOAD-DIRECT-UPLOAD, #150): a load is 1.54x faster warm, alongside the
already-recorded 100.8 -> 53.4 GiB peak RSS.

WHY THE PARAGRAPH IS SHORTER THAN THE SUM OF ITS PARTS. The page budgets prose
at 20 paragraphs and 700 chars each, and my first correction failed both. The
paragraph-count failure was the interesting one: _prose_paragraphs treats any
line starting with * as a list item, and on main the wrap happened to put
**prefill 21.5x** at the start of a line -- so the checker had been reading ONE
paragraph as TWO, and the page was inside its 700-char budget only by accident.
Rewrapping honestly merged them and exposed 891 real chars. Rather than restore
the accidental split or spend the count, the paragraph was tightened to 677
(6/6 prompts / 96/96 tokens vs the vLLM 0.25.0 oracle -> 6/6 prompts vs the vLLM 0.25.0 oracle; all nine of that model's ops -> every op of that model),
which is what the checker asks for: the numbers belong in BENCHMARKS, and this
page explains what works.

DELIBERATELY NOT DONE: moving Vulkan out of STATUS's "Not supported yet"
section. Running a 27B at llama.cpp parity is a real milestone, but 25 native
ops of 112 with quant/MoE/MLA at zero is not "supported", and the entry already
states precisely what does and does not work.

Gates: check-public-doc-tables, check-doc-checkpoint and check-agent-record all
green. No source file changes.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude-Code:claude-opus-5 [Claude Code]

`docs/FEATURES.md` said Qwen3.6-27B decodes **4.24 tok/s vs llama.cpp's 4.35**.
That is the pre-#191 number: it claimed we were BEHIND when the binding
measurement is parity MET, and it contradicted the three other public surfaces,
all of which were reviewed and landed saying 4.36:

  docs/STATUS.md      decode 4.36/4.35 MET
  docs/BENCHMARKS.md  decode 4.36 vs 4.35, MET (7 clean legs)
  README.md           decodes 4.36 vs llama.cpp Vulkan 4.35 tok/s

Corrected, and it keeps saying the pass is NARROW: the binding figure is the
median of 7 clean legs at TPOT 228.66-230.25 ms, and the 0.69% leg spread IS the
noise floor. The detail stays in BENCHMARKS and the record rather than being
restated here.

Also adds the load-phase result the paragraph was missing beside its memory one
(`ENG-LOAD-DIRECT-UPLOAD`, #150): a load is 1.54x faster warm, alongside the
already-recorded 100.8 -> 53.4 GiB peak RSS.

WHY THE PARAGRAPH IS SHORTER THAN THE SUM OF ITS PARTS. The page budgets prose
at 20 paragraphs and 700 chars each, and my first correction failed both. The
paragraph-count failure was the interesting one: `_prose_paragraphs` treats any
line starting with `*` as a list item, and on main the wrap happened to put
`**prefill 21.5x**` at the start of a line -- so the checker had been reading ONE
paragraph as TWO, and the page was inside its 700-char budget only by accident.
Rewrapping honestly merged them and exposed 891 real chars. Rather than restore
the accidental split or spend the count, the paragraph was tightened to 677
(`6/6 prompts / 96/96 tokens vs the vLLM 0.25.0 oracle` -> `6/6 prompts vs the
vLLM 0.25.0 oracle`; `all nine of that model's ops` -> `every op of that model`),
which is what the checker asks for: the numbers belong in BENCHMARKS, and this
page explains what works.

DELIBERATELY NOT DONE: moving Vulkan out of STATUS's "Not supported yet"
section. Running a 27B at llama.cpp parity is a real milestone, but 25 native
ops of 112 with quant/MoE/MLA at zero is not "supported", and the entry already
states precisely what does and does not work.

Gates: check-public-doc-tables, check-doc-checkpoint and check-agent-record all
green. No source file changes.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude-Code:claude-opus-5 [Claude Code]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants