document how htop calculates used/free memory values

[jackyf]

Hi!

A Debian user reported that there is a significant (5-10%) discrepancy between "free + caches/buffers" as reported by 'free' and the numbers reported by htop.

It would be nice for man page to contain a sentence about how memory usage is calculated.

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=795776

[hishamhm]

Hello,

On 17.08.2015 23:45, OmegaPhil wrote:
> omega@omega1:~/.config/xfce4/panel$ free
>               total        used        free      shared  buff/cache
> available
> Mem:      32998636     9515852      466464      263632    23016320
> 22807536
> Swap:      10485756           0    10485756
> 
> ====================================================================
> 
> free is '/usr/bin/free':
> [...]

I see now, an output format of 'free' changed in sid, I get similar now after I have installed a newer version.

> Atm htop reports 12550/32225MB used.

Yes, I see that numbers don't match exactly on your system.

I forwarded your report upstream.

Note that 32998636 / 1024 = 32225: htop reports in megabytes, /usr/bin/free reports in kilobytes.

Apparently /usr/bin/free is summing buffers and cache, so I don't get the separate numbers, but in my version of /usr/bin/free, where I have the "-/+ buffers/cache" column, the first value X that shows on "X/Y MB" in the memory meter is the same as the first value that appears in the "-/+ buffers/cache" line, divided by 1024 (in other words, used - (buffers + cache) from the first line).

I don't know how this new /usr/bin/free is calculating its numbers, but htop's calculation is straightforward, using numbers from /proc/meminfo. It's basically (MemTotal - MemFree - (Buffers + Cached)) / 1024.

[jackyf]

Hello,

On 20.08.2015 00:10, Hisham Muhammad wrote:

I don't know how this new /usr/bin/free is calculating its numbers, but htop's calculation is straightforward, using
numbers from /proc/meminfo. It's basically |(MemTotal - MemFree - (Buffers + Cached)) / 1024|.

Makes sense. Would be still nice to put the (variation of) last sentence to the man page.

-- t. Ievgen

[hishamhm]

htop calculates various things, not sure if I want to all replicate calculations in the manpage... (e.g. then there are CPU calculations, etc.) I'd rather keep it more abstract.

[OmegaPhil]

htop is used as a serious tool, so not documenting how it actually behaves is hardly a good thing. Memory is not 'abstract', like CPU capacity its a fixed resource, users, sysadmins and programmers rely on getting true information from tools like yours to determine whether they need more hardware, programs are behaving themselves etc etc.

You are basically saying you need to be a programmer (view the source) or have the luck to know of this ticket in order to actually understand what is going on, which is definitely wrong.

Separately, I've managed to get free fixed in terms of how it reports used memory - you can see how the key free tool works out cached memory here - currently you're not dealing with the kernel slab at all in LinuxProcessList_scanMemoryInfo - the kernel slab is mainly a cache, i.e. not all of it is reclaimable - so the correct stat to use for this is SReclaimable rather than Slab (see kernel documentation on the proc filesystem under meminfo).

Currently on this machine with 32GB of RAM, SReclaimable is 3629616KB, so htop is inflating the used RAM amount by 3.46GB! I've had it much worse too (all filesystems get rsync'd daily, which is probably why it shows up so badly on my machines).

I noticed that there has already been a pull request on this, but it was oddly closed - the change is wrong as it includes SUnreclaim as cache, but its at least along the correct lines?

[OmegaPhil]

For anyone else following, the problem has already been reported over a year ago with a pull request too at #67