cadvisor memory legend

[cowboy-cod3r]

When looking at the total memory usage of a running container, the legend shows "hot" and "total". Is there any documentation that explains the definition "hot" and "total" and how that is calculated?

[vishh]

"hot" refers to the working set, which is held in RAM, and "total" refers
to all the memory used by a process, a part of which may reside on a swap
device.

+1 to your suggestion of documenting it. AFAIK this isn't documented
anywhere.

On Wed, Apr 8, 2015 at 1:45 PM, Sean Humbarger notifications@github.com
wrote:

When looking at the total memory usage of a running container, the legend
shows "hot" and "total". Is there any documentation that explains the
definition "hot" and "total" and how that is calculated?

—
Reply to this email directly or view it on GitHub
#638.

[rjnagal]

Hot is the working set - pages that has been recently touched as calculated
by the kernel.
Total includes hot + cold memory - where cold are the pages that have not
been touched in a while and can be reclaimed if there was global memory
pressure.

I don't think the total on cadvisor page includes swap, but we should
probably report that too.

On Wed, Apr 8, 2015 at 3:17 PM, Vish Kannan notifications@github.com
wrote:

"hot" refers to the working set, which is held in RAM, and "total" refers
to all the memory used by a process, a part of which may reside on a swap
device.

+1 to your suggestion of documenting it. AFAIK this isn't documented
anywhere.

On Wed, Apr 8, 2015 at 1:45 PM, Sean Humbarger notifications@github.com
wrote:

When looking at the total memory usage of a running container, the legend
shows "hot" and "total". Is there any documentation that explains the
definition "hot" and "total" and how that is calculated?

—
Reply to this email directly or view it on GitHub
#638.

—
Reply to this email directly or view it on GitHub
#638 (comment).

[vmarmol]

We used to have a bug open for this, but we closed it with some API comments. We should document this in the UI as well.

[cowboy-cod3r]

Thanks for the info!

[artworkad]

What I find confusing when using cAdvisor the first time is that it says 16Gib of a total of 24 Gib memory is used.

Via free -m I found out that I actually have 22 Gib free memory and that 14 Gib are just cached by linux.

[vmarmol]

Memory accounting is interesting because different parts of the kernel do it differently. I am not sure what free -m uses. I'm curious what the hot vs. cold memory we were reporting in that case

[mboussaa]

@vishh : the memory usage aggregated by cadvisor is the total hot (working set) memory + cold pages + swap that's right?

[vishh]

AFAIK it does not account for swap as of now.

On Mon, Oct 26, 2015 at 4:08 AM, mboussaa notifications@github.com wrote:

@vishh https://github.com/vishh : the memory usage aggregated by
cadvisor is the total hot (working set) memory + cold pages + swap that's
right?

—
Reply to this email directly or view it on GitHub
#638 (comment).

[mboussaa]

Working set memory == RSS ?

[jimmidyson]

Working set = RSS + cache (no swap as @vishh said above). We have #975 open for reporting RSS by itself.

We delegate to libcontainer to retrieve stats. We could implement this ourselves but it's kind of nice to have consistency.

[mboussaa]

the hot or total memory is sent to backend storage like influxdb ?

[jimmidyson]

Working set & total are sent - see https://github.com/google/cadvisor/blob/master/storage/influxdb/influxdb.go#L119-L124

[mboussaa]

So memory_usage (total) = RAM + Swap ? because @vishh said that cAdvisor is not supporting memory+swap accounting as of now here #913

If hot is RSS + cache what would be the total memory ?

[jimmidyson]

So let me try again. Details in https://www.kernel.org/doc/Documentation/cgroups/memory.txt.

Total (memory.usage_in_bytes) = rss + cache
Working set = Total - inactive (not recently accessed memory = inactive_anon + inactive_file)

How's that?

[Vanuan]

Is working set shared between containers? Because if I sum up all container's working sets it's well above the total memory. And all containers' working sets are exactly the same.

[vishh]

working_set is not shared between containers. Are you running with swap
enabled?

On Wed, Feb 24, 2016 at 7:17 PM, John Yani notifications@github.com wrote:

Is working set shared between containers? Because if I sum up all
container's working sets it's well above the total memory. And all
containers' working sets are exactly the same.

—
Reply to this email directly or view it on GitHub
#638 (comment).

[Vanuan]

Ah, it's probably heapster's bug then. I use memory/working_set_bytes_gauge which is the same, no matter which container I choose.

[Vanuan]

Posted kubernetes-retired/heapster#1005

[cboettig]

It would be really nice if cadvisor didn't include cached memory as "used" in it's overview / summary graphs. This results in confusing / inconsistent display on most linux hosts, e.g. the "overview" will show 90% memory used while the table directly below will list no process in which memory use is high (since it is mostly in cache).

[iwankgb]

Closing very old issue.