Rework CPU counting #656

cgzones · 2021-06-12T18:31:46Z

Currently htop does not support offline CPUs and hot-swapping, e.g. via
echo 0 > /sys/devices/system/cpu/cpu2/online

Split the current single cpuCount variable into activeCPUs and existingCPUs.

Supersedes: #650
Related: #580

/cc @mwahlroos @sthen

@natoscott please take a look at the PCP code

TODO:

solaris/SolarisProcessList.c

natoscott · 2021-06-15T03:25:49Z

ProcessList.h

@@ -81,12 +81,15 @@ typedef struct ProcessList_ {
   memory_t usedSwap;
   memory_t cachedSwap;

-   unsigned int cpuCount;
+   unsigned int activeCPUs;
+   unsigned int existingCPUs;


No big deal but I find the naming slightly unclear - AIUI 'active' means 'online' and 'existing' means the CPU exists but is either online or offline? An alternative might be to use 'onlineCPUs' and something like 'actualCPUs' or 'realCPUs' here?

I'd prefer actualCPUs over realCPUs to not clash with virtualization stuff …

natoscott · 2021-06-15T03:50:38Z

linux/LinuxProcessList.c

@@ -158,30 +158,85 @@ static void LinuxProcessList_initNetlinkSocket(LinuxProcessList* this) {

 #endif

-static void LinuxProcessList_updateCPUcount(ProcessList* super, FILE* stream) {
+static void LinuxProcessList_updateCPUcount(ProcessList* super) {


Hmm. This sysfs-walk looks like a relatively expensive approach - going from what is currently a handful of syscalls at most, per sample, to potentially many thousands of additional syscalls with each sample on large CPU count machines.

Does an offline CPU appear at all in /proc/stat output? If not, we should be able to do this operation using a combination of a/ one-time call to sysconf(_SC_NPROCESSORS_CONF) at startup and, b/ CPU state-management based on presence (or zero values?) in /proc/stat (while reading stats) at sample time... i.e. if cpu22 of 32 is missing in /proc/stat mark it as !online ... and update activeCPU count accordingly - would that work?

Reporter of the issue that probably got this started here. I have no other authority on the matter. :)

Another alternative in case of Linux would be to parse the list of online CPU nodes at /sys/devices/system/cpu/online on update. That would be single read from sysfs rather than one read per CPU.

The sysfs files are listed in the Linux sysfs documentation. The format for the CPU-listing files seems to be e.g. "0-3,5,7", which in case of the aggregate online file would mean that CPU nodes 0 to 3, and additionally 5 and 7 are online.

That would give explicit and authoritative information on which CPUs are currently being scheduled. It might of course be simpler to just assume that the ones not appearing in /proc/stat are offline. That seems to be the case on my machine but I have no broader knowledge on how the file lists the CPUs.

The original proposal used sysconf(_SC_NPROCESSORS_CONF), which is implemented as a sysfs-walk, see https://code.woboq.org/userspace/glibc/sysdeps/unix/sysv/linux/getsysstats.c.html#236.
By doing the walk on our own we can search specific for the data we need.
Maybe at RC stage we can take a more serious look on performance.

natoscott · 2021-06-15T03:53:35Z

linux/LinuxProcessList.c

      } else {
         unsigned int cpuid;
         (void) sscanf(buffer, "cpu%4u %16llu %16llu %16llu %16llu %16llu %16llu %16llu %16llu %16llu %16llu", &cpuid, &usertime, &nicetime, &systemtime, &idletime, &ioWait, &irq, &softIrq, &steal, &guest, &guestnice);
-         assert(cpuid == i - 1);
+         adjCpuId = cpuid + 1;


Ah, this is what I was looking for in that previous comment. :) So an alternate approach to finding offline CPUs might be to mark all offline at start of this function, and then mark each individually as 'online' as we parse the file and find a CPU (and update counts as we go if we need total online vs actual).

Currently htop does not support offline CPUs and hot-swapping, e.g. via echo 0 > /sys/devices/system/cpu/cpu2/online Split the current single cpuCount variable into activeCPUs and existingCPUs. Supersedes: htop-dev#650 Related: htop-dev#580

Bogus uptime measurements can result in wrap-arounds, leading to negative garbage values printed.

Kernel threads do not have an executable and the check can result in garbage values as unprivileged user.

Wait until it has been decided what kind of task the entry actually is.

sched_getaffinity() and sched_setaffinity() are also available on BSDs. Remove the Linux restraint.

Wait until it has been decided what kind of task the entry actually is.

Example hot-swapping: psradm -F -f 2

openbsd/OpenBSDProcessList.c:176:56: error: no member named 'ki_pid' in 'struct kinfo_proc'; did you mean 'p_pid'? const int mib[] = { CTL_KERN, KERN_PROC_CWD, kproc->ki_pid }; ^~~~~~ p_pid /usr/include/sys/sysctl.h:375:10: note: 'p_pid' declared here int32_t p_pid; /* PID_T: Process identifier. */ ^ openbsd/OpenBSDProcessList.c:458:33: error: comparison of integers of different signs: 'int' and 'unsigned int' [-Werror,-Wsign-compare] if (opl->cpus[i].cpuIndex == id) ~~~~~~~~~~~~~~~~~~~~~ ^ ~~

fasterit · 2021-08-02T13:20:38Z

Works on MacOSX 11.3.1 (Intel CPU)
https://discussions.apple.com/thread/250559322 on how to offline CPUs / Hypertreading on MacOS

Related: htop-dev#656

cgzones force-pushed the cpu_count branch 4 times, most recently from a3b2ac6 to 5408fd7 Compare June 12, 2021 21:07

BenBE added the enhancement Extension or improvement to existing feature label Jun 13, 2021

cgzones force-pushed the cpu_count branch 12 times, most recently from 37fd370 to 5ff6d27 Compare June 13, 2021 12:41

cgzones marked this pull request as ready for review June 13, 2021 12:42

cgzones force-pushed the cpu_count branch from 5ff6d27 to 99ebd37 Compare June 13, 2021 12:48

fraggerfox mentioned this pull request Jun 13, 2021

Add NetBSD support in htop(1) #603

Merged

natoscott reviewed Jun 15, 2021

View reviewed changes

solaris/SolarisProcessList.c Show resolved Hide resolved

natoscott reviewed Jun 15, 2021

View reviewed changes

cgzones added 6 commits July 18, 2021 07:44

Rework CPU counting

41af31b

Currently htop does not support offline CPUs and hot-swapping, e.g. via echo 0 > /sys/devices/system/cpu/cpu2/online Split the current single cpuCount variable into activeCPUs and existingCPUs. Supersedes: htop-dev#650 Related: htop-dev#580

Add ProcessList_isCPUonline

11d2206

UptimeMeter: treat all non-positive values as error

fbe3a21

Bogus uptime measurements can result in wrap-arounds, leading to negative garbage values printed.

FreeBSD: skip exe check for kernel thread

af0b67c

Kernel threads do not have an executable and the check can result in garbage values as unprivileged user.

FreeBSD: calculate whether to show entry last

3451b6c

Wait until it has been decided what kind of task the entry actually is.

FreeBSD: mark the original kernel thread with pid 0 as such

adcedf8

cgzones added 6 commits July 18, 2021 07:50

Enable affinity support for non-Linux

1fb0c72

sched_getaffinity() and sched_setaffinity() are also available on BSDs. Remove the Linux restraint.

DragonFlyBSD: drop void TODO

b148a4b

DragonFlyBSD: calculate whether to show entry last

f47e88f

Wait until it has been decided what kind of task the entry actually is.

Solaris: support offline CPUs and hot-swapping

90cc16e

Example hot-swapping: psradm -F -f 2

OpenBSD: support offline CPUs and hot-swapping

edf236f

BenBE force-pushed the cpu_count branch from 99ebd37 to edf236f Compare July 18, 2021 06:02

BenBE added this to the 3.1.0 milestone Jul 18, 2021

fasterit merged commit ed82ce6 into htop-dev:master Aug 2, 2021

fraggerfox mentioned this pull request Aug 3, 2021

NetBSD: Rework CPU counting. #718

Merged

cgzones deleted the cpu_count branch August 8, 2021 13:18

cgzones mentioned this pull request Aug 8, 2021

Linux: handle offline CPUs when getting frequencies #650

Closed

natoscott added a commit to natoscott/htop that referenced this pull request Aug 9, 2021

PCP: implement a missing piece for offline and hotplug CPUs

ee83126

Related: htop-dev#656

natoscott mentioned this pull request Aug 9, 2021

PCP: implement a missing piece for offline and hotplug CPUs #726

Merged

cgzones mentioned this pull request Aug 10, 2021

Linux: incorrect CPU frequencies when some CPUs are offline #648

Closed

mator mentioned this pull request Aug 21, 2021

[sparc64] limit cpu shown on active (online) cpus only? #757

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework CPU counting #656

Rework CPU counting #656

cgzones commented Jun 12, 2021 •

edited by fasterit

natoscott Jun 15, 2021

BenBE Jun 15, 2021

natoscott Jun 15, 2021

mwahlroos Jun 15, 2021

cgzones Aug 8, 2021

natoscott Jun 15, 2021

fasterit commented Aug 2, 2021

Rework CPU counting #656

Rework CPU counting #656

Conversation

cgzones commented Jun 12, 2021 • edited by fasterit

natoscott Jun 15, 2021

Choose a reason for hiding this comment

BenBE Jun 15, 2021

Choose a reason for hiding this comment

natoscott Jun 15, 2021

Choose a reason for hiding this comment

mwahlroos Jun 15, 2021

Choose a reason for hiding this comment

cgzones Aug 8, 2021

Choose a reason for hiding this comment

natoscott Jun 15, 2021

Choose a reason for hiding this comment

fasterit commented Aug 2, 2021

cgzones commented Jun 12, 2021 •

edited by fasterit