Skip to content

metal: PTQ1_0 exact-float trit extraction (+30-38% decode) and the missing per-expert mat-vec - #157

Merged
khosravipasha merged 2 commits into
prismfrom
perf/ptq1_0-metal-fp-extract
Sep 5, 2026
Merged

metal: PTQ1_0 exact-float trit extraction (+30-38% decode) and the missing per-expert mat-vec#157
khosravipasha merged 2 commits into
prismfrom
perf/ptq1_0-metal-fp-extract

Conversation

@bri-prism

Copy link
Copy Markdown

Two changes to the Metal PTQ1_0 mat-vec: a decode speedup, and a missing kernel whose absence segfaults any PTQ1_0 mixture-of-experts model on Metal today.

Decode: extract trits in exact float

The mat-vec pulled each trit out of its byte with the base-3 remainder recurrence, w = v*3; t = w >> 8; v = w & 0xFF. That is four integer ops and a convert per trit, on an ISA that cannot co-issue integer and floating-point work, and the kernel's own comment records that it is instruction-bound rather than bandwidth-bound. So the integer work was the ceiling.

A packed byte is a base-3 fraction of 256. With u = b/256, trit n is g_{n+1} - 3*g_n for g_k = floor(3^k * u), and every 3^k * b is below 2^24, so the floors are exact: this matches the recurrence bit for bit over all 256 bytes and all five positions. Summing t_n * y_n over a byte then collapses to sum_k g_k*(y_{k-1} - 3*y_k) + g_5*y_4, whose coefficients depend only on the activations, so the caller stages those in place of the raw y and reuses them across every row exactly as it did before. The inner loop is one floor and one fma per trit with no integer arithmetic. The qh trit uses the two-floor single-trit form, which replaces a lane-divergent loop that stepped the recurrence up to three times.

Measured on an M5 Pro, llama-bench -p 512 -n 128 -r 3, two interleaved passes, two dense models roughly five times apart in size:

decode before decode after prefill
smaller dense 221 287 tok/s, +29.5% unchanged
larger dense 59.3 81.7 tok/s, +37.8% unchanged

For scale, on the smaller model the same-binary TQ2_0 decode is 290, so PTQ1_0 now decodes at 99 percent of TQ2_0 at 1.75 versus 2.06 bits per weight. Prefill is untouched because the mat-mul path does not use this routine.

Output is not bit-identical to the previous kernel and should not be expected to be: staging coefficients reassociates the fp32 sum. The per-element values are exact; only summation order moves.

Correctness bug: the per-expert kernel did not exist

supports_op accepts PTQ1_0 for MUL_MAT_ID and the dispatcher has an nsg/nr0 entry for it, but kernel_mul_mv_id_ptq1_0_f32 was never instantiated. The first per-expert PTQ1_0 mat-vec on Metal fails the library lookup (kernel not found in any metal library) and the process segfaults. On current prism-v7 that is any PTQ1_0 MoE model.

It also hid itself in the test suite. test-backend-ops -o MUL_MAT_ID on Metal dies on its first PTQ1_0 case, which happens to come right after the pq2_0 case in the type list, so the run printed 24 clean cases and stopped. Nothing after that point, for any type, was being exercised, and the truncated output looks like a complete pass unless you count the cases or check the exit code. With the instantiation added the same run completes: 1022 MUL_MAT_ID cases, 75 of them PTQ1_0, zero failures, 3/3 backends.

The instantiation reuses the existing impl, so it inherits the extraction change above.

Verification

The identity was checked against the recurrence for all 256 bytes and all positions before the kernel was touched, and the collapse checked exact against the raw trit sum. test-backend-ops on Metal: MUL_MAT 45/45, GET_ROWS 4/4, MUL_MAT_ID 75/75 PTQ1_0 cases, zero failures. The A/B used two build directories with the embedded shader confirmed distinct by strings immediately before each measurement, since the shader is baked into libggml-metal.dylib and a stale or rebuilt arm silently invalidates the comparison.

Heads-up for the reviewer

There is uncommitted PTQ1_0 Metal work in another session's worktree on feat/tq1_0-g128 (a qh-only simdgroup broadcast, per its own notes). It touches the same ptq1_0_dot_reg and will need reconciling with this if both land.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The oversized explanatory comment conflicts with the repository's explicit concise-comment convention.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Optimizes Metal PTQ1_0 decoding and restores per-expert matrix-vector support.

Changes:

  • Replaces integer trit extraction with exact floating-point extraction.
  • Adds the missing PTQ1_0 MUL_MAT_ID kernel instantiation.
File summaries
File Description
ggml/src/ggml-metal/kernels/mul_mv.metal Optimizes PTQ1_0 decoding and enables per-expert dispatch.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +525 to +536
// Dot against coefficients already staged in registers by the caller and reused across
// all nr0 rows. A packed byte is a base-3 fraction of 256: with u = b/256, trit n is
// t_n = g_{n+1} - 3*g_n, g_k = floor(3^k * u)
// and 3^k*b <= 61965 is exact in fp32, so the floors are exact and this matches the
// integer recurrence bit for bit over all 256 bytes and all five positions. Summing
// t_n*y_n over a byte's trits then collapses to
// sum_{k=1..4} g_k*(y_{k-1} - 3*y_k) + g_5*y_4
// whose coefficients depend only on the activations, so the caller stages those in
// place of the raw y. The inner loop is one floor and one fma per trit and never
// leaves the float pipe, which matters because this ISA cannot co-issue integer and
// floating-point work; the recurrence spent four integer ops and a convert per trit.
// sumy is subtracted once for the -1 offset, exactly as before.
…urrence

The mat-vec pulled each trit out of its byte with the base-3 remainder
recurrence, w = v*3, t = w >> 8, v = w & 0xFF: four integer ops and a convert
per trit, on an ISA that cannot co-issue integer and floating-point work. The
kernel was already known to be instruction-bound rather than bandwidth-bound.

A packed byte is a base-3 fraction of 256. With u = b/256, trit n is
g_{n+1} - 3*g_n for g_k = floor(3^k*u), and every 3^k*b is below 2^24, so the
floors are exact and this matches the recurrence bit for bit over all 256
bytes and all five positions. Summing t_n*y_n over a byte's trits collapses to
sum_k g_k*(y_{k-1} - 3*y_k) + g_5*y_4, whose coefficients depend only on the
activations, so the caller stages those in place of the raw y and reuses them
across every row. The inner loop is one floor and one fma per trit with no
integer arithmetic. The qh trit uses the two-floor single-trit form, replacing
a lane-divergent loop that stepped the recurrence up to three times.

Verified against the recurrence for all bytes before the kernel was touched.
test-backend-ops MUL_MAT: 45 PTQ1_0 cases pass, zero failures. Output is not
bit-identical: the coefficient staging reassociates the fp32 sum.

Measured on an M5 Pro, llama-bench r=3, two interleaved passes, dense models:
  small  decode 221 -> 287 tok/s (+29.5%)
  large  decode  59 ->  82 tok/s (+37.8%)
Prefill unchanged; the mat-mul path does not use this routine.
…eady claims

supports_op accepts PTQ1_0 for MUL_MAT_ID and the dispatcher carries an nsg/nr0
entry for it, but the kernel_mul_mv_id_ptq1_0_f32 pipeline was never
instantiated, so the first per-expert PTQ1_0 mat-vec on Metal fails the library
lookup and the process segfaults. test-backend-ops hit this on its first PTQ1_0
MUL_MAT_ID case and died there, which truncated the Metal MUL_MAT_ID run to 24
cases while still printing a clean tail; nothing after that case, for any type,
was being exercised.

Add the instantiation next to the other low-bit per-expert kernels. It reuses
kernel_mul_mv_ptq1_0_f32_impl, so it inherits the exact-float extraction from
the previous commit. The full Metal MUL_MAT_ID run now completes: 1022 cases,
75 of them PTQ1_0, zero failures.
@khosravipasha
khosravipasha force-pushed the perf/ptq1_0-metal-fp-extract branch from e297bd2 to 11bf2a9 Compare September 5, 2026 20:57
@khosravipasha
khosravipasha changed the base branch from prism-v7 to prism September 5, 2026 20:57
@khosravipasha

Copy link
Copy Markdown
Collaborator

Rebased onto prism and retargeted; the two commits cherry-picked with no conflicts and no code changes. Going forward please open PRs against prism directly: prism-v7 is frozen and kept for history only, nothing lands there anymore.

Measured on an M5 Max against a clean prism baseline (d0ce941, separate worktree per arm, kernel_mul_mv_id_ptq1_0 confirmed absent in the baseline dylib and present in the PR's). PTQ1_0 packs of the four sizes, llama-bench -fa 1, r=3:

size tg128 before tg128 after pp512
2B 159 186 (+17%) unchanged
4B 120 154 (+28%) unchanged
9B 91 111 (+23%) unchanged (2107 both, two interleaved passes)
27B 31 37 (+19%) unchanged within thermal noise

PTQ1_0 decode is now at or above the PQ2_0 pack of the same size on the 2B, 4B and 9B.

Correctness: MUL_MAT ptq1_0 45/45, GET_ROWS 4/4, MUL_MAT_ID 949/949 with the suite running to completion (on prism it segfaults at the first PTQ1_0 MUL_MAT_ID case, which is the missing instantiation this fixes). KLD against the F16 golden is unchanged on the 9B and 2B; since that path runs at batch 512 and never hits the mat-vec, single-token perplexity (-b 1 -ub 1) was also compared and is identical to four decimals on both builds.

Merging now so it makes the next release. Left for follow-up PRs, not blocking:

  • The uncommitted PTQ1_0 Metal work on feat/tq1_0-g128 that touches ptq1_0_dot_reg needs reconciling with this before it lands.
  • From the metal, vulkan: FWHT kernels for wide block widths (to prism) #155 review: ggml_metal_fwht_supported_size should also gate on max_theadgroup_memory_size (the 8192 kernel needs 32 KiB) and ggml_metal_op_fwht should not set a null pipeline.
  • test-backend-ops -o CPY on Metal still segfaults on the permuted f32 -> pq2_0 copy case, unrelated to this PR.
  • The f16 FWHT kernels on Metal have no reachable test case (the signed test multiplies f16 by f32, which Metal rejects before the FWHT).

@khosravipasha
khosravipasha merged commit 7113f64 into prism Sep 5, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants