CPU of cadvisor too high

[ramuskay]

When i deploy the stack with default values the cadvisor container had a CPU usage too high regards to regular containers. Maybe its' normal ?

Expected Behaviour

I would expect that CPU usage was like other containers not that much

Current Behaviour

Bellow the CPU taken by cadvisor container

The top of my server :

Possible Solution

I tried to follow the case : google/cadvisor#1774
But it didn't seems to work on my server

Steps to Reproduce (for bugs)

Provide a link to a live example, or steps to reproduce this bug. Include code to reproduce, if relevant:

1. git clone https://github.com/vegasbrianc/prometheus.git
2. cd prometheus
3. docker stack deploy -c docker-stack.yml prom

Your Environment

[root@xxxxx prometheus]# docker version
Client: Docker Engine - Community
 Version:           19.03.7
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        7141c199a2
 Built:             Wed Mar  4 01:24:10 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.7
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       7141c199a2
  Built:            Wed Mar  4 01:22:45 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

My OS is CentOS Linux release 8.1.1911 (Core)

[github-actions]

Awesome! Thanks for taking the time to open an issue. We will have a look and answer as soon as we can.' first issue

[immortaly007]

I'm experiencing the same issue (15% CPU usage here). What helped a bit was putting the scrape_interval for cadvisor to 15s in prometheus/prometheus.yml:

  - job_name: 'cadvisor'

    # Override the global default and scrape targets from this job every 5 seconds.
-    scrape_interval: 5s
+    scrape_interval: 15s

    dns_sd_configs:
    - names:
      - 'tasks.cadvisor'
      type: 'A'
      port: 8080

But the CPU usage is still pretty high (~7%) so I opened an issue with cadvisor:
google/cadvisor#2523

I hope they have tips and I will keep this ticket updated if they have interesting conclusions!

[aspro-at]

I'm experiencing exactly the same.

[frayos]

Thank you @immortaly007 !

• scrape_interval: 5s

• scrape_interval: 15s

The above helped !

[immortaly007]

Good to hear!
I managed to bring the CPU usage down further by modifying the "housekeeping interval" and disabling statistics that weren't needed by the default (and my current) grafana dashboard setup.

To do this, under the cadvisor part of the docker-compose.yml file, add:

services:
#...
  cadvisor:
#  ...
    command:
      - '--docker_only' 
      - '--disable_metrics=disk,diskIO,tcp,udp,percpu,sched,process'
      - '--housekeeping_interval=10s'

[aspro-at]

Good to hear!
I managed to bring the CPU usage down further by modifying the "housekeeping interval" and disabling statistics that weren't needed by the default (and my current) grafana dashboard setup.

To do this, under the cadvisor part of the docker-compose.yml file, add:

services:
#...
  cadvisor:
#  ...
    command:
      - '--docker_only' 
      - '--disable_metrics=disk,diskIO,tcp,udp,percpu,sched,process'
      - '--housekeeping_interval=10s'

The above helped! Down from 20 % to below 7 % cpu usage. Thanks.

[github-actions]

This issue has seen no activity and is marked as stale

[balajeek]

Good to hear!
I managed to bring the CPU usage down further by modifying the "housekeeping interval" and disabling statistics that weren't needed by the default (and my current) grafana dashboard setup.

To do this, under the cadvisor part of the docker-compose.yml file, add:

services:
#...
  cadvisor:
#  ...
    command:
      - '--docker_only' 
      - '--disable_metrics=disk,diskIO,tcp,udp,percpu,sched,process'
      - '--housekeeping_interval=10s'

I recently setup cadvisor and prometheus and noticed 36% usage, when googled i ended up here, so i put in the command as you mentioned, after that cadvisor won't run, log shows a lot or lines, some that says error are:

-alsologtostderr
log to standard error as well as files
-logtostderr
log to standard error instead of files
-stderrthreshold value
logs at or above this threshold go to stderr (default 2)

any idea?