On Linux, try to use /proc/loadavg to get instaneous load (#2218)

[entrope]

The /proc/loadavg file includes the traditional 1-, 5- and 15-minute loads plus an instantaneous count of processes that are ready to run. This lets us respond faster to new processes being launched by other processes.

[entrope]

This is draft because I am not sure whether having a static variable is considered bad form. The Windows function CalculateProcessorLoad() already uses static variables, so perhaps it is okay here?

[satmandu]

Is there any chance of this getting merged?

[Flowdalic]

This changes the API of ninja's double GetLoadAverage() on Linux to return the currently running number of kernel-level threads (tasks) instead of the damped 1-minute average.

Frankly, besides me being skeptical that this function should return a different value depending on the operating system, using the instantaneous value also has drawbacks. The major one is that it, essentially being the value returned by Linux's nr_running(), does not account for uninterruptible tasks, i.e., tasks that are likely waiting for an I/O operation to complete, while those are included in the loadaverage (on Linux). You can read more about it in Brendan Gregg's excellent blog post: https://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html

This poses the risk of overloading the system as those I/O-waiting tasks are likely to become runnable again.

And yes, if they become not runnable again soon, then this would mean that no new ninja jobs are started, even though there are spare CPU resources. However, in this case, the system is likely already I/O-bound and starting to compilation jobs would be counterproductive anyways.

In summary, I am not sure if this change is sensible.

[entrope]

The underlying problem this was meant to address (#2218) is actually the opposite: The current approach takes too long to notice when lots of processes are creating, leading to CPU oversubscription and potentially VM (and thus swap -> IO) oversubscription. Do you propose a different way to solve the same problem?

[Flowdalic]

Do you propose a different way to solve the same problem?

Yes, see #1949, which got recently merged.