Skip to content

Commit

Permalink
proc_sysinfo: Properly account for kb_main_shared
Browse files Browse the repository at this point in the history
Linux "free" output has been improved and adapted over time.  Already at
some point in 2014-2016, the format changed in an incompatible way.[1]

[1] https://askubuntu.com/questions/770108/what-do-the-changes-in-free-output-from-14-04-to-16-04-mean

Ideally I would recommend "free" to users to view several fields,
in particular:

1. "available" (requires not-ancient kernel)

2. swap usage

3. "shared".  Often hundreds of megabytes on modern desktops.  Excessive
   "shared" memory can help show/analyze a problem.  This will detect if
   tmpfs is filling up the system RAM.

   Sometimes this detects a memory leak that does not show up in process
   resident memory.  It has sometimes showed graphics allocations which are
   not being freed properly, including but not limited to GEM buffers.

Sadly, two of the six columns were misleading.

1. "cache" - and hence "buff/cache" - includes "shared".  But "shared" is
   not a reclaimable cache!  It must be swapped instead.  Linux handles it
   on the same LRU lists as anonymous memory.

   I am not able to explain why Cached included Shmem originally.
   I would love if Linux could change it!  But...  This "quirk" has also
   been enshrined in memory.stat - "cached" in cgroup-v1, and "file" in
   cgroup-v2.  These statistics are documented & at the top of the list.
   I would not dare think to change them.

   The true knowledge about Cached in /proc/meminfo has now been spread by
   several articles since at least 2017.  And if anyone had to compensate
   for this e.g. in their alert systems, the kernel ABI rules say they
   should not touch it.  I fear it is too late  now :'-(.  They should just
   admit it, and fix all the meminfo docs to admit the quirk.

   Let us fix the "cache" column in "free".  I think it will be nicer.  The
   "free" docs did not admit the "quirk" either.

   On the bright side, at least when there was a problem, and you run
   "free -h", the excessive "shared" figure stood out.  It was right next
    to the cached" figure that pretends there is no problem.

   I cannot imagine anyone mourning this quirk.

2. "used"

   In a previous version of "free", there were two separate lines, with the
   second line shown as "+- buffers/cache".  In this version, "used" is
   always show as "+- buffers/cache".  ...That's right, the entire value
   of "cached" was subtracted from "used", so that "used" no longer
   includes "shared".

   Now this might just be a personal irritant of mine.  But, when I have to
   look at "free", I spend far too long re-discovering why the "used" value
   is *so* different from "total - available".  I am not expecting it
   would match exactly: "total - available" should be higher, by up to 3
   times the "low water mark"[2].  The margin from the "watermarks" is
   usually relatively small - only a few percentage points out of total
   system memory.

   [2] https://unix.stackexchange.com/questions/261247/how-can-i-get-the-amount-of-available-memory-portably-across-distributions

   Including "shared" in the "used" figure will avoid the specific neurosis
   I have :-).

   Making used ~= total - available will significantly simplify the
   output of "free".  It will usually appear consistent with the single
   usage level shown in gnome-system-monitor.

   Community support is aware the "available" metric is the intended,
   modern, and reliable thing to look at.[3]

   [3] See e.g. https://www.linuxatemyram.com/

   I believe these arguments sufficient, to justify changing the "used"
   value even if we did not change "cache".  But given that we are
   changing the "cache" value, it is *essential* that we update the value
   previously known as "used +- buffers/cache".

   You can see this in the diff.  I only modify the line calculating
   kb_main_cached.  kb_main_used is already derived from kb_main_cached,
   so no further change is required.
  • Loading branch information
sourcejedi committed Jun 16, 2019
1 parent 2e7f387 commit f4b8938
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
4 changes: 2 additions & 2 deletions free.1
Expand Up @@ -19,7 +19,7 @@ columns are:
Total installed memory (MemTotal and SwapTotal in /proc/meminfo)
.TP
\fBused\fR
Used memory (calculated as \fBtotal\fR - \fBfree\fR - \fBbuffers\fR - \fBcache\fR)
Used memory (calculated as \fBtotal\fR - \fBfree\fR - \fBbuffers\fR - \fBcache\fR + \fBshared\fR)
.TP
\fBfree\fR
Unused memory (MemFree and SwapFree in /proc/meminfo)
Expand All @@ -31,7 +31,7 @@ Memory used (mostly) by tmpfs (Shmem in /proc/meminfo)
Memory used by kernel buffers (Buffers in /proc/meminfo)
.TP
\fBcache\fR
Memory used by the page cache and slabs (Cached and SReclaimable in /proc/meminfo)
Memory used by the page cache and slabs (Cached - Shmem + SReclaimable in /proc/meminfo)
.TP
\fBbuff/cache\fR
Sum of \fBbuffers\fR and \fBcache\fR
Expand Down
2 changes: 1 addition & 1 deletion proc/sysinfo.c
Expand Up @@ -778,7 +778,7 @@ void meminfo(void){
if(kb_inactive==~0UL){
kb_inactive = kb_inact_dirty + kb_inact_clean + kb_inact_laundry;
}
kb_main_cached = kb_page_cache + kb_slab_reclaimable;
kb_main_cached = kb_page_cache - kb_main_shared + kb_slab_reclaimable;
kb_swap_used = kb_swap_total - kb_swap_free;

/* if kb_main_available is greater than kb_main_total or our calculation of
Expand Down

0 comments on commit f4b8938

Please sign in to comment.