[CPU] The impact of changing CPU frequency on test results

[remittor]

When I run tests without locking CPU core frequency changes, I get different results every time. But once I completely lock the frequency at a single level, the test results become stable.

I'd like to know how the test server is currently configured.

ChatGPT advises me to do this:

# disable CPU boost
echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/boost
echo 0 | sudo tee /sys/devices/system/cpu/cpufreq/policy*/boost

# lock CPU freq
cpupower frequency-set -g performance
cpupower frequency-set -u 2.7GHz
cpupower frequency-set -d 2.7GHz

# activate passive mode
echo passive | sudo tee /sys/devices/system/cpu/amd_pstate/status
cpupower frequency-set -g userspace
cpupower frequency-set -f 2700000

@MDA2AV , and how do the framework and utility gcannon currently share CPU resources?

[MDA2AV]

CPU: AMD Ryzen Threadripper PRO 3995WX
Governor always set to performance
AMD PState: active mode
Boost: enabled (boost = 1)
Freq range: 1.74 - 4.31 GHz

We don't lock to a fixed frequency - we use performance governor with boostn enabled which lets the CPU run at max clock, this is intentional, we want to measure peak throughput, the performance governor keeps cores at max frequency rather than letting them scale up or down.

We do get stable results across runs with variance typically <2%

Other factors:

• kernel cache drop between runs (echo 3 > /proc/sys/vm/drop_caches)
• docker daemon restart between frameworks

For CPU sharing between gcannon and the framework: both run on the host with --network host, the baseline is the only that limits the framework container to 64 CPU threads to ensure same CPU usage among all frameworks, gcannon will never saturate the remaining 64. All other tests the gcannon load is much smaller and has a lot less impact, this is also one of the reason we use gcannon, it is much more performant than wrk or any load generator, gcannon is pure C io_uring built specifically for HttpArena loads. For the remaining tests the framework container has access to the 128 cpu threads, of course it will compete with gcannon but the OS will handle this well, we also show for each test the framework container CPU usage for good measure.

Things to consider - having a machine for load generator and other for framework container to avoid CPU sharing - this is not feasible, the networking would be the bottleneck, we have tests that hit 400Gbit/s, NICs for this would be unfeasible, also we would be introducing TCP network stack overhead.

NUMA - One single NUMA node for whole CPU