Skip to content

v-code01/roofline

Repository files navigation

roofline: is token decode memory-bandwidth-bound, and does batching escape it?

Token generation reads every model weight once per token to do a single multiply-add each - two FLOPs per weight. That is an arithmetic intensity of 2*params / weight_bytes, which for a 4-bit model is only 2-3 FLOP/byte. A roofline says an operation that low-intensity is limited by memory bandwidth, not compute. This measures the bandwidth ceiling (a multi-threaded STREAM triad in C++), the decode rate at several batch sizes (from the server), and places decode on the roofline. Two model sizes (1.5B and 0.5B, Q4_K_M).

Pre-registration

Three predictions were committed to git (PREREG.md) before the measurement: (1) single-token decode is memory-bandwidth-bound (intensity far below the ridge); (2) a single stream already uses a large fraction (>40%) of peak bandwidth; (3) batching amortizes the weight read and raises aggregate throughput but stays memory-bound. All three held.

Result

Measured memory-bandwidth ceiling (STREAM triad): 221 GB/s multi-thread, 124 GB/s single-thread. Roofline ridge at a nominal 2000 GFLOP/s compute peak: AI = 9.0 FLOP/byte.

 model    AI(1 tok)   batch   agg_tok/s   GFLOP/s   per-token-read GB/s   % of peak BW
 1.5B      3.12         1        151        466            149              67%
                        2        143        442            141              64%
                        4        207        639            205              92%
 0.5B      1.99         1        245        240            120              54%
                        2        403        395            198              90%
                        4        554        542            272             123%
  1. Single-token decode is memory-bandwidth-bound. Both models have an arithmetic intensity of 2-3 FLOP/byte, far below the ~9 ridge, so they sit deep on the memory-bound (diagonal) part of the roofline. A model would need ~9 FLOP/byte to become compute-bound; decode has a third of that.

  2. A single stream already runs near the memory ceiling. One decode stream implies 54-67% of the measured peak bandwidth - decode is not idling on memory, it is close to saturating it with a single request.

  3. Batching amortizes the weight read - and the numbers prove it. Aggregate throughput rises with batch size (0.5B 245 -> 554 tok/s, 1.5B 151 -> 207). At batch 4 the 0.5B's per-token-read bandwidth computes to 123% of the physical ceiling - impossible if each token re-read the weights, so the weights must be read once per step and shared across the batch. That sharing raises the effective arithmetic intensity and walks the operating point right along the roofline

    • toward, but not reaching, the compute ceiling. The gain saturates as the bandwidth ceiling is hit.

The one-line finding: decode is a memory-bandwidth problem, not a compute problem. A single stream already uses over half the memory bandwidth; batching helps only by amortizing weight reads against that same wall, and the smaller model - fewer bytes per token - both starts further from the ceiling and batches better. This is the mechanism behind the batching tax and the flat single-stream throughput seen at the request level.

Reproduce

./reproduce.sh 8081 8082        # PORT_15B PORT_05B
./scripts/gate.sh               # C++ -Werror build, ruff, mypy --strict, pytest, verify

tools/verify.py is an independent recompute (its own arithmetic-intensity and batch-scaling derivation, no shared code) that re-asserts the memory-bound conclusion, the near-ceiling single stream, and the amortization evidence from the raw stream and decode rows.

Limitations and falsifiers

  • One backend (llama.cpp on Apple GPU), one machine, Q4_K_M, two model sizes, batch up to 4. The bandwidth ceiling is a CPU STREAM triad; the GPU decode achieves an effective bandwidth in the same range, which is why STREAM is a valid scale reference, but it is not a direct GPU-bandwidth measurement.
  • Parameter counts are the nominal published values; weight bytes are the exact on-disk sizes. The memory-bound conclusion is robust to the compute-peak assumption because the intensity (2-3) is far below the ridge for any plausible peak.
  • Falsifier: if the arithmetic intensity were above the ridge, decode would be compute-bound; it is 2-3 vs a ridge of 9.
  • Falsifier: if batching gave no aggregate gain, the amortization claim would be wrong; it gives 1.4x-2.3x and drives the per-token-read bandwidth past the physical ceiling.

MIT licensed. The oracle is exact arithmetic plus monotonic-clock timing; no model-quality judgement.

About

No description, website, or topics provided.

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors