docker top for host level metrics

[jchauncey]

So I was wondering what the community thought about the idea of docker top returning the top metrics for all running containers on a host? This would be a great way to debug issues where a docker host is running out of memory and you want to find the offending container.

Seems like the first place to start would be to have an api endpoint that returns the data then implement it for the CLI. The endpoint could just be /containers/top.

Thoughts?

[rboyer]

Isn't this largely solved (minus the cli-aspect) by cadvisor?

[jchauncey]

Kind of. Thats a lot of infrastructure to run just to debug a host though.
The information is already there when you say docker top . I
just want an aggregate of that view on the command line.
On Nov 1, 2014 11:53 AM, "R.B. Boyer" notifications@github.com wrote:

Isn't this largely solved (minus the cli-aspect) by cadvisor
https://github.com/google/cadvisor?

Reply to this email directly or view it on GitHub
#8875 (comment).

[prasincs]

I'm working on something similar -- in exploration phase so no proposals yet. I'm considering adding a WATCH verb to the parser that allows people to track any cgroups metric (for now, networking is harder).. There's nothing stopping you from running another container in privileged mode and read from the cgroup hierarchy but I think it's more empowering to developers to allow giving explicit metrics to watch for. The docker daemon can probably keep a sliding window for each metric without too much overhead.

Still shaping the idea here. Is this close to what you're thinking?

[jchauncey]

kind of. The problem we have right now is that we are using docker for development environments with a tool we help maintain called spoon. Sometimes these development environments can run out of control and its really difficult to figure out which container is causing hte problem when there are 40+ running with similar processes.

So what I would like to do is aggregate the top information from /containers//top and have an endpoint at /containers/top which does that.

This way I can easily see which container is using all the memory on a host or all the cpu.

[jchauncey]

So this does what I am asking...

docker ps | cut -d' ' -f 1 | grep -v CONTAINER | xargs -I{} /bin/sh -c 'docker top {} aux | tail -n +2'

[crosbymichael]

With #9984 merged you can get an aggregate view of how much cpu, memory, io, etc containers are running by doing:

docker stats insurgency1 insurgency2 insurgency3 minecraft-family redis

CONTAINER           CPU %               MEM USAGE/LIMIT     MEM %               NET I/O
insurgency1         3.62%               244.4 MB/2.099 GB   11.64%              0 B/0 B
insurgency2         4.65%               135.6 MB/2.099 GB   6.46%               0 B/0 B
insurgency3         3.65%               79.18 MB/2.099 GB   3.77%               0 B/0 B
minecraft-family    14.13%              408.6 MB/2.099 GB   19.47%              0 B/0 B
redis               0.17%               6.558 MB/67.11 MB   9.77%               648 B/648 B