Fix Processes() calls with many cores

[atoulme]

Hello @shirou,

This is reprisal of the work of #1497. Given that we're not to touch the interfaces of gopsutil and cannot touch the mechanism by which we capture boot time, I have ventured to make boot time faster by making the /proc/stat file read more efficient.

Here are the details of the implementation, tested on an Ampere developer platform with 80 cores (the number of cores is particularly impactful as it makes /proc/stat longer to read).

on branch master:

goos: linux
goarch: arm64
pkg: github.com/shirou/gopsutil/v3/process
BenchmarkProcesses
BenchmarkProcesses-80    	       4	 252089573 ns/op
PASS

Process finished with the exit code 0
PASS

on branch read_file_faster:

goos: linux
goarch: arm64
pkg: github.com/shirou/gopsutil/v3/process
BenchmarkProcesses
BenchmarkProcesses-80    	       6	 203633386 ns/op
PASS

Benchstat output across all benchmarks of the process package:

goos: linux
goarch: arm64
pkg: github.com/shirou/gopsutil/v3/process
                              │   old.txt   │               new.txt               │
                              │   sec/op    │   sec/op     vs base                │
_fillFromCommWithContext-80     9.546µ ± 4%   9.679µ ± 3%   +1.40% (p=0.045 n=10)
_fillFromStatusWithContext-80   25.41µ ± 3%   26.01µ ± 5%        ~ (p=0.247 n=10)
NewProcess-80                   140.4µ ± 4%   128.2µ ± 2%   -8.66% (p=0.000 n=10)
ProcessName-80                  3.099n ± 0%   2.904n ± 0%   -6.29% (p=0.000 n=10)
ProcessPpid-80                  131.1µ ± 3%   118.8µ ± 2%   -9.36% (p=0.000 n=10)
Processes-80                    255.8m ± 2%   215.6m ± 3%  -15.70% (p=0.000 n=10)

Typical /proc/stat file:

cpu  768944 10288 1072079 11045715258 3486 0 5444 0 0 0
cpu0 8786 70 6308 137988080 26 0 67 0 0 0
cpu1 9068 30 6191 138072341 59 0 11 0 0 0
cpu2 65109 159 171353 137907618 83 0 8 0 0 0
cpu3 24084 118 40891 137992666 88 0 8 0 0 0
cpu4 43038 444 109352 137978323 83 0 4 0 0 0
cpu5 21472 193 40224 138025036 82 0 0 0 0 0
cpu6 42410 118 123254 137981039 92 0 3 0 0 0
cpu7 18037 232 33534 138039186 101 0 5 0 0 0
cpu8 31662 186 72772 138013182 98 0 4 0 0 0
cpu9 13657 181 17251 138023478 91 0 221 0 0 0
cpu10 31512 159 74950 138015049 40 0 0 0 0 0
cpu11 12116 472 16357 138037994 71 0 166 0 0 0
cpu12 21121 85 43973 138039462 63 0 4 0 0 0
cpu13 10251 27 10354 138021006 59 0 0 0 0 0
cpu14 18905 280 40738 138048357 50 0 5 0 0 0
cpu15 10326 360 10793 138065576 47 0 3 0 0 0
cpu16 11191 100 13201 138062883 52 0 0 0 0 0
cpu17 8028 107 7053 138066687 54 0 0 0 0 0
cpu18 7964 102 7473 138082226 43 0 6 0 0 0
cpu19 7326 158 5075 138084645 27 0 1 0 0 0
cpu20 7577 4 6287 138079436 41 0 0 0 0 0
cpu21 7813 12 6644 138067886 84 0 131 0 0 0
cpu22 6736 117 5078 138085923 19 0 2 0 0 0
cpu23 7123 128 5150 138086066 21 0 3 0 0 0
cpu24 9399 13 6703 138082336 24 0 39 0 0 0
cpu25 8955 45 6350 138083373 26 0 0 0 0 0
cpu26 6441 83 4730 138088400 31 0 2 0 0 0
cpu27 6157 25 4148 138089297 27 0 6 0 0 0
cpu28 6401 9 4586 138087528 44 0 1 0 0 0
cpu29 6391 17 4265 138087849 46 0 21 0 0 0
cpu30 6286 461 3308 138090520 50 0 0 0 0 0
cpu31 6094 104 3068 138091314 86 0 0 0 0 0
cpu32 6022 108 3822 138089008 19 0 1 0 0 0
cpu33 5909 73 3729 138088729 11 0 1 0 0 0
cpu34 6820 294 3541 138088142 6 0 1 0 0 0
cpu35 6616 129 3347 138090230 8 0 0 0 0 0
cpu36 6206 107 4706 138083252 9 0 1 0 0 0
cpu37 6584 28 2864 138085803 23 0 1 0 0 0
cpu38 6220 131 3001 138091331 37 0 2 0 0 0
cpu39 5486 217 2772 138092221 38 0 2 0 0 0
cpu40 5842 31 2824 138093761 14 0 0 0 0 0
cpu41 6105 125 2923 138093217 17 0 0 0 0 0
cpu42 5204 44 3539 138091542 12 0 1 0 0 0
cpu43 5839 32 3417 138091082 10 0 4 0 0 0
cpu44 5615 19 3980 138092487 22 0 4 0 0 0
cpu45 6026 424 4100 138091306 19 0 0 0 0 0
cpu46 5527 209 3688 138094125 12 0 0 0 0 0
cpu47 5336 109 3033 138094396 12 0 3 0 0 0
cpu48 5494 103 2653 138096147 2 0 1 0 0 0
cpu49 5218 30 2393 138097187 5 0 0 0 0 0
cpu50 5484 239 2130 138096460 8 0 0 0 0 0
cpu51 5043 34 2446 138096773 15 0 0 0 0 0
cpu52 5498 27 2971 138095154 8 0 12 0 0 0
cpu53 5181 43 2634 138087718 8 0 2 0 0 0
cpu54 4988 36 2976 138095457 4 0 0 0 0 0
cpu55 5081 21 2344 138095338 7 0 0 0 0 0
cpu56 4636 273 1897 138096693 12 0 2 0 0 0
cpu57 4884 316 1935 138095963 15 0 3 0 0 0
cpu58 4870 229 1745 138097331 6 0 1 0 0 0
cpu59 4924 28 1746 138097999 7 0 5 0 0 0
cpu60 4461 129 1955 138096954 361 0 0 0 0 0
cpu61 5030 35 1930 138097414 460 0 0 0 0 0
cpu62 4179 23 2157 138097375 9 0 5 0 0 0
cpu63 4242 134 2235 138096928 10 0 8 0 0 0
cpu64 6682 254 2108 138094052 4 0 3 0 0 0
cpu65 4354 115 1743 138097071 3 0 0 0 0 0
cpu66 4162 9 2496 138095759 7 0 0 0 0 0
cpu67 3863 28 1960 138097697 6 0 5 0 0 0
cpu68 13484 17 17176 137901816 109 0 4614 0 0 0
cpu69 12515 17 14273 137980618 100 0 4 0 0 0
cpu70 5085 74 3738 138092386 10 0 0 0 0 0
cpu71 4792 14 3574 138093476 13 0 0 0 0 0
cpu72 4943 167 2880 138090318 12 0 2 0 0 0
cpu73 5333 17 4367 138089338 19 0 0 0 0 0
cpu74 4909 5 2995 138076914 30 0 2 0 0 0
cpu75 4817 256 3160 138072268 30 0 0 0 0 0
cpu76 4146 233 2160 138096352 8 0 1 0 0 0
cpu77 4598 474 1860 138096155 10 0 3 0 0 0
cpu78 4733 34 2352 138094484 29 0 4 0 0 0
cpu79 4488 164 2359 138094227 44 0 2 0 0 0
intr 1136940775 0 1020438 49587864 0 0 0 210556659 0 0 0 0 0 867076724 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 609 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 51690 0 0 0 0 0 0 0 1 5604085 970576 914985 910669 70 33120 33768 24240 21358 14764 17850 12776 9795 8765 6098 7307 8012 15687 13640 7797 11437
ctxt 1918526774
btime 1691617230
processes 501797
procs_running 39
procs_blocked 0
softirq 272142382 51064 32640104 523 8408436 152655 0 4696 152439652 131 78445121

[shirou]

Could you check and run benchmark #1515 ?

[shirou]

I have tested on AWS c5a.16xlarge (CPU=64), and I got this result. It seems definite improvement. Thank you for your contribution!

goos: linux
goarch: amd64
pkg: github.com/shirou/gopsutil/v3/process
cpu: AMD EPYC 7R32
             │  base.txt   │             after.txt              │
             │   sec/op    │   sec/op     vs base               │
Processes-64   37.35m ± 1%   34.81m ± 1%  -6.79% (p=0.000 n=10)

             │   base.txt   │              after.txt               │
             │     B/op     │     B/op      vs base                │
Processes-64   9.250Mi ± 0%   7.009Mi ± 0%  -24.22% (p=0.000 n=10)

             │  base.txt   │              after.txt              │
             │  allocs/op  │  allocs/op   vs base                │
Processes-64   64.57k ± 0%   57.64k ± 0%  -10.73% (p=0.000 n=10)

(Memo: I have tested use ReadBytes instead of ReadString, but nothing changed)