Repository navigation
Capping CPU clock on a hot laptop: same MoE token speed, 16 °C cooler (GPU/CPU expert split) #32234
Replies: 2 comments
|
Follow-up with hardware counters, and a correction. Correction: in this setup token generation is not RAM-bandwidth bound. I measured DRAM traffic during tg with LFM2.5-24B-A2B Q4_K_M, build 11502 (08246a2), cap 3.3 GHz on all cores except cpu4–7, Dynamic Boost on. Counters are the median of the last 10 s of a tg-only run.
What the RAM can deliver (pinned STREAM-style test, 3 repeats each): 4 P-core threads 36.8 GB/s triad / 38.5 read, 6 P-core threads 37.5 / 39.4, plus E-cores 31–34 / 37–38, all 20 threads 32.1 / 38.4. The llama.cpp build mattered more than CPU clock. Same flags, CPU uncapped, interleaved runs:
I haven't isolated whether that's a code change between builds, or the local native build vs the prebuilt runtime-selected CPU backend. |
|
Perhaps use |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
On a hot laptop running a MoE model with experts split between GPU and CPU, token generation barely depends on CPU clock (corrected: it is not RAM-bound either, see the follow-up comment with hardware counters). Capping the CPU's max frequency at 3.0 GHz cost ~1% tg and dropped CPU temperature from 100 °C (throttling) to 84 °C. One catch: the cap has to cover the cores in llama's
-Cmask.Setup
llama-bench -fitt 128 -fitc 131072 -fa 1 -ctk q4_0 -ctv q4_0 -ub 2048 -t 6 -C 0x555 -p 2048 -n 256 -r 3/sys/devices/system/cpu/cpu*/cpufreq/scaling_max_freq(intel_pstate active). This is a frequency cap, not an undervolt.sensorsduring each runResults
-t 4 -C 0x505The "none" and "except cpu4–7" rows are alternating A/B runs. The other rows are single runs (each with
-r 3).Takeaways
-C, or the cap does nothing for inference.uptimebefore trusting an A/B.Same card on the CUDA scoreboard: #15013 (comment 19601067).
All reactions