CPU Load value greater than 100% on MacOS

[b1nhack]

MacOS Sonoma 14.2.1
macchina 6.1.8
libmacchina 6.3.5

Maybe add a judgment to recalculate the value if it's greater than 100%.

[grtcdr]

Can anyone with Apple hardware confirm this? If so, please let us know by contributing to the conversation.

[Criomby]

Haven't experienced any issues yet.
Will test again while doing some heavy lifting.

macOS Sonoma 14.3.1
macchina 6.1.8
libmacchina 6.3.5

[b1nhack]

It's a very low-probability problem. I've only triggered it once by accident.

[mikemadden42]

When running large builds with cargo, I do see this issue with the load over 100%. Let me know if I can help debug the issue.

$ macchina --version
macchina     6.2.0
libmacchina  7.2.1

CPU Load    -  106%

CPU Load    -  284%

CPU Load    -  457%

CPU Load    -  574%

[grtcdr]

@123marvin123, do you also run into this issue?

[123marvin123]

I haven't seen it happen on my system. I checked the code and I don't seem to understand how this could occur, since we divide the cpu load reported by macOS by the available CPU cores.

It looks like "loadavg" does not really report CPU usage (see https://www.howtogeek.com/194642/understanding-the-load-average-on-linux-and-other-unix-like-systems/). For CPU usage, one would expect the value to be between 0 and 100%.

[grtcdr]

Thanks for your input Marvin,

Is it possible that the processor count could be misrepresented? I'm not well-versed in ARM or Apple hardware specificities but I had the suspicion that this may be due to how processor cores are being counted, it looks like get_num_cpus() prefers ¹ libc::_SC_NPROCESSORS_CONF for ARM processors and _ONLN for x86_64. Can you confirm the reasoning provided in the comments of the linked snippet?

If ARM does indeed power down individual processors affecting the processor count, then Lawrence's answer on StackOverflow ² which hints at the use of sched_getaffinity() (may be under a different name in macOS) in dynamic settings looks like the most robust choice for ARM targets.

Footnotes

1. https://github.com/seanmonstar/num_cpus/blob/7c03fc930cc47a9b94e8ca66ca44ef1a454c8f51/src/lib.rs#L365-L370 ↩

2. https://stackoverflow.com/a/47116857/9184674 ↩

[123marvin123]

The "num_cpus" crate returns the correct number of cores on my system.

According to the man pages for the "getloadavg" function, the function is reporting the number of processes in the run queue:

"The getloadavg() function returns the number of processes in the system
run queue averaged over various periods of time."

So a value greater than 100% makes sense if the system is "overloaded".

If we want the actual load of the CPU, we could use (on macOS)

host_statistics(..., HOST_CPU_LOAD_INFO, ..., ...)

https://developer.apple.com/documentation/kernel/1502546-host_statistics

[grtcdr]

I passed over the "overload" hypothesis while researching this problem, it seems plausible but I can't actually confirm it without doing some tests. Would you be able to give host_statistics a shot in libmacchina? If so, are the others willing to give us a hand in testing the changes?

[b1nhack]

Hope I can help.

[grtcdr]

If you don't mind getting your hands dirty, I would suggest hacking up a special implementation of libmacchina::shared::cpu_cores() for the macos module, but the best course of action would be to submit a pull request directly to num_cpus and work with the maintainer so that downstream packages that rely on it, libmacchina included, can benefit from the improvement.

[123marvin123]

I passed over the "overload" hypothesis while researching this problem, it seems plausible but I can't actually confirm it without doing some tests. Would you be able to give host_statistics a shot in libmacchina? If so, are the others willing to give us a hand in testing the changes?

Unfortunately, it looks like this approach won't work for an application like macchina, because we would have to calculate the CPU load during some interval (e.g. 1 second). I think the "load average" is the best we can get without increasing execution time drastically.

I will attach my code for querying CPU usage from the mach kernel for archiving purposes:

use std::mem;
use std::thread;
use std::time::Duration;

extern crate libc;

#[repr(C)]
#[derive(Debug, Default, Clone, Copy)]
struct host_cpu_load_info_data_t {
    cpu_ticks: [u32; 4],
}

fn get_cpu_load() -> Result<host_cpu_load_info_data_t, i32> {
    let mut cpu_load = host_cpu_load_info_data_t::default();
    let mut count = mem::size_of::<host_cpu_load_info_data_t>() as u32;

    let result = unsafe {
        libc::host_statistics64(
            libc::mach_host_self(),
            libc::HOST_CPU_LOAD_INFO,
            &mut cpu_load as *mut _ as *mut i32,
            &mut count,
        )
    };

    if result != libc::KERN_SUCCESS {
        Err(result)
    } else {
        Ok(cpu_load)
    }
}

fn main() {
    let mut prev_cpu_load = get_cpu_load().expect("Failed to get initial CPU load");

    loop {
        thread::sleep(Duration::from_secs(1));

        let cpu_load = get_cpu_load().expect("Failed to get CPU load");

        let cpu_delta = cpu_load.cpu_ticks.iter().zip(prev_cpu_load.cpu_ticks.iter())
            .map(|(a, b)| a - b)
            .collect::<Vec<_>>();

        let total_delta = cpu_delta.iter().sum::<u32>() as f64;

        let user_perc = cpu_delta[libc::CPU_STATE_USER as usize] as f64 / total_delta * 100.0;
        let sys_perc = cpu_delta[libc::CPU_STATE_SYSTEM as usize] as f64 / total_delta * 100.0;
        let idle_perc = cpu_delta[libc::CPU_STATE_IDLE as usize] as f64 / total_delta * 100.0;
        let nice_perc = cpu_delta[libc::CPU_STATE_NICE as usize] as f64 / total_delta * 100.0;
        let total_perc = user_perc + sys_perc + nice_perc;

        println!(
            "CPU usage: {:.2}% (user: {:.2}%, sys: {:.2}%, idle: {:.2}%, nice: {:.2}%)",
            total_perc, user_perc, sys_perc, idle_perc, nice_perc
        );

        prev_cpu_load = cpu_load;
    }
}

Output:

CPU usage: 4.12% (user: 2.75%, sys: 1.38%, idle: 95.88%, nice: 0.00%)
CPU usage: 5.99% (user: 3.75%, sys: 2.25%, idle: 94.01%, nice: 0.00%)
CPU usage: 6.18% (user: 3.66%, sys: 2.52%, idle: 93.82%, nice: 0.00%)
CPU usage: 7.61% (user: 5.24%, sys: 2.37%, idle: 92.39%, nice: 0.00%)
CPU usage: 10.39% (user: 7.26%, sys: 3.13%, idle: 89.61%, nice: 0.00%)
CPU usage: 11.64% (user: 7.63%, sys: 4.01%, idle: 88.36%, nice: 0.00%)
CPU usage: 16.94% (user: 12.90%, sys: 4.05%, idle: 83.06%, nice: 0.00%)
CPU usage: 5.53% (user: 3.89%, sys: 1.63%, idle: 94.47%, nice: 0.00%)
CPU usage: 9.28% (user: 6.52%, sys: 2.76%, idle: 90.72%, nice: 0.00%)
CPU usage: 5.01% (user: 3.63%, sys: 1.38%, idle: 94.99%, nice: 0.00%)
CPU usage: 6.12% (user: 4.00%, sys: 2.12%, idle: 93.88%, nice: 0.00%)
CPU usage: 6.76% (user: 5.26%, sys: 1.50%, idle: 93.24%, nice: 0.00%)
CPU usage: 7.49% (user: 4.87%, sys: 2.62%, idle: 92.51%, nice: 0.00%)
CPU usage: 11.29% (user: 7.90%, sys: 3.39%, idle: 88.71%, nice: 0.00%)
CPU usage: 7.65% (user: 5.27%, sys: 2.38%, idle: 92.35%, nice: 0.00%)
CPU usage: 8.81% (user: 5.66%, sys: 3.14%, idle: 91.19%, nice: 0.00%)
CPU usage: 13.88% (user: 9.64%, sys: 4.24%, idle: 86.12%, nice: 0.00%)
CPU usage: 81.93% (user: 64.38%, sys: 17.56%, idle: 18.07%, nice: 0.00%)
CPU usage: 55.32% (user: 31.52%, sys: 23.80%, idle: 44.68%, nice: 0.00%)
CPU usage: 11.45% (user: 6.54%, sys: 4.91%, idle: 88.55%, nice: 0.00%)
CPU usage: 13.98% (user: 9.70%, sys: 4.28%, idle: 86.02%, nice: 0.00%)
CPU usage: 7.61% (user: 5.11%, sys: 2.49%, idle: 92.39%, nice: 0.00%)
CPU usage: 18.16% (user: 14.50%, sys: 3.66%, idle: 81.84%, nice: 0.00%)
CPU usage: 20.30% (user: 16.17%, sys: 4.14%, idle: 79.70%, nice: 0.00%)
CPU usage: 18.51% (user: 12.47%, sys: 6.05%, idle: 81.49%, nice: 0.00%)
CPU usage: 25.22% (user: 20.68%, sys: 4.54%, idle: 74.78%, nice: 0.00%)
CPU usage: 10.89% (user: 8.14%, sys: 2.75%, idle: 89.11%, nice: 0.00%)
CPU usage: 8.15% (user: 5.26%, sys: 2.88%, idle: 91.85%, nice: 0.00%)
CPU usage: 8.03% (user: 5.27%, sys: 2.76%, idle: 91.97%, nice: 0.00%)
CPU usage: 8.67% (user: 5.78%, sys: 2.89%, idle: 91.33%, nice: 0.00%)
CPU usage: 12.41% (user: 8.02%, sys: 4.39%, idle: 87.59%, nice: 0.00%)
CPU usage: 8.42% (user: 5.78%, sys: 2.64%, idle: 91.58%, nice: 0.00%)
CPU usage: 20.73% (user: 14.82%, sys: 5.90%, idle: 79.27%, nice: 0.00%)
CPU usage: 21.43% (user: 17.92%, sys: 3.51%, idle: 78.57%, nice: 0.00%)
CPU usage: 55.02% (user: 43.96%, sys: 11.05%, idle: 44.98%, nice: 0.00%)
CPU usage: 73.75% (user: 57.62%, sys: 16.13%, idle: 26.25%, nice: 0.00%)
CPU usage: 92.28% (user: 78.38%, sys: 13.90%, idle: 7.72%, nice: 0.00%)
CPU usage: 99.48% (user: 78.58%, sys: 20.90%, idle: 0.52%, nice: 0.00%)
CPU usage: 89.17% (user: 76.82%, sys: 12.36%, idle: 10.83%, nice: 0.00%)
CPU usage: 91.80% (user: 73.44%, sys: 18.36%, idle: 8.20%, nice: 0.00%)
CPU usage: 90.03% (user: 74.68%, sys: 15.35%, idle: 9.97%, nice: 0.00%)
CPU usage: 96.24% (user: 80.44%, sys: 15.80%, idle: 3.76%, nice: 0.00%)
CPU usage: 85.84% (user: 73.60%, sys: 12.24%, idle: 14.16%, nice: 0.00%)
CPU usage: 89.02% (user: 79.82%, sys: 9.20%, idle: 10.98%, nice: 0.00%)
CPU usage: 92.54% (user: 83.19%, sys: 9.36%, idle: 7.46%, nice: 0.00%)
CPU usage: 78.61% (user: 70.38%, sys: 8.23%, idle: 21.39%, nice: 0.00%)
CPU usage: 95.95% (user: 60.97%, sys: 34.99%, idle: 4.05%, nice: 0.00%)
CPU usage: 98.07% (user: 48.40%, sys: 49.68%, idle: 1.93%, nice: 0.00%)
CPU usage: 71.39% (user: 46.36%, sys: 25.03%, idle: 28.61%, nice: 0.00%)
CPU usage: 71.17% (user: 55.99%, sys: 15.18%, idle: 28.83%, nice: 0.00%)
CPU usage: 88.60% (user: 59.41%, sys: 29.19%, idle: 11.40%, nice: 0.00%)
CPU usage: 92.39% (user: 63.35%, sys: 29.03%, idle: 7.61%, nice: 0.00%)
CPU usage: 92.36% (user: 70.60%, sys: 21.76%, idle: 7.64%, nice: 0.00%)
CPU usage: 78.33% (user: 62.44%, sys: 15.90%, idle: 21.67%, nice: 0.00%)
CPU usage: 81.19% (user: 73.57%, sys: 7.62%, idle: 18.81%, nice: 0.00%)
CPU usage: 59.75% (user: 51.01%, sys: 8.73%, idle: 40.25%, nice: 0.00%)
CPU usage: 78.02% (user: 68.87%, sys: 9.15%, idle: 21.98%, nice: 0.00%)
CPU usage: 99.49% (user: 87.44%, sys: 12.06%, idle: 0.51%, nice: 0.00%)
CPU usage: 98.46% (user: 80.44%, sys: 18.02%, idle: 1.54%, nice: 0.00%)
CPU usage: 82.26% (user: 64.78%, sys: 17.48%, idle: 17.74%, nice: 0.00%)
CPU usage: 67.01% (user: 53.96%, sys: 13.04%, idle: 32.99%, nice: 0.00%)
CPU usage: 87.45% (user: 74.26%, sys: 13.19%, idle: 12.55%, nice: 0.00%)
CPU usage: 99.61% (user: 89.03%, sys: 10.58%, idle: 0.39%, nice: 0.00%)
CPU usage: 82.36% (user: 72.21%, sys: 10.15%, idle: 17.64%, nice: 0.00%)
CPU usage: 92.99% (user: 84.20%, sys: 8.79%, idle: 7.01%, nice: 0.00%)
CPU usage: 93.41% (user: 81.37%, sys: 12.04%, idle: 6.59%, nice: 0.00%)
CPU usage: 82.61% (user: 70.72%, sys: 11.89%, idle: 17.39%, nice: 0.00%)
CPU usage: 68.45% (user: 60.56%, sys: 7.89%, idle: 31.55%, nice: 0.00%)
CPU usage: 42.78% (user: 36.02%, sys: 6.77%, idle: 57.22%, nice: 0.00%)
CPU usage: 40.15% (user: 32.20%, sys: 7.95%, idle: 59.85%, nice: 0.00%)
CPU usage: 33.12% (user: 27.90%, sys: 5.22%, idle: 66.88%, nice: 0.00%)
CPU usage: 40.53% (user: 33.55%, sys: 6.99%, idle: 59.47%, nice: 0.00%)
CPU usage: 37.67% (user: 33.12%, sys: 4.55%, idle: 62.33%, nice: 0.00%)
CPU usage: 35.19% (user: 29.87%, sys: 5.32%, idle: 64.81%, nice: 0.00%)