munin eth0 traffic graphs missing, patch #896

Open
glaszig opened this Issue Aug 2, 2016 · 8 comments

Comments

Projects
None yet
8 participants
@glaszig

glaszig commented Aug 2, 2016

i'm running on a ubuntu 14.04 droplet on digitalocean.

munin did show missing graphs for eth0 traffic. a quick lock into /var/log/munin/munin-update.log revealed some issue with how the rrd configuration was being created.

2016/08/02 23:30:02 [INFO]: Starting munin-update
2016/08/02 23:30:02 [INFO] starting work in 31337 for mx.example.com/127.0.0.1:4949.
2016/08/02 23:30:07 [INFO] creating rrd-file for if_eth0->down: '/var/lib/munin/example.com/mx.example.com-if_eth0-down-d.rrd'
2016/08/02 23:30:07 [ERROR] Unable to create '/var/lib/munin/example.com/mx.example.com-if_eth0-down-d.rrd': min must be less than max in DS definition
2016/08/02 23:30:07 [ERROR] In RRD: Error updating /var/lib/munin/example.com/mx.example.com-if_eth0-down-d.rrd: opening '/var/lib/munin/example.com/mx.example.com-if_eth0-down-d.rrd': No such file or directory
2016/08/02 23:30:07 [INFO] creating rrd-file for if_eth0->up: '/var/lib/munin/example.com/mx.example.com-if_eth0-up-d.rrd'
2016/08/02 23:30:07 [ERROR] Unable to create '/var/lib/munin/example.com/mx.example.com-if_eth0-up-d.rrd': min must be less than max in DS definition
2016/08/02 23:30:07 [ERROR] In RRD: Error updating /var/lib/munin/example.com/mx.example.com-if_eth0-up-d.rrd: opening '/var/lib/munin/example.com/mx.example.com-if_eth0-up-d.rrd': No such file or directory
2016/08/02 23:30:07 [INFO]: Munin-update finished for node example.com;mx.example.com (5.01 sec)

i then ran the if_eth0 plugin manually which yielded a negative value for the DS MAX value of the rrd config - see down.max -1000000 and up.max -1000000

# /etc/munin/plugins/if_eth0 config
graph_order down up
graph_title eth0 traffic
graph_args --base 1000
graph_vlabel bits in (-) / out (+) per ${graph_period}
graph_category network
graph_info This graph shows the traffic of the eth0 network interface. Please note that the traffic is shown in bits per second, not bytes. IMPORTANT: On 32-bit systems the data source for this plugin uses 32-bit counters, which makes the plugin unreliable and unsuitable for most 100-Mb/s (or faster) interfaces, where traffic is expected to exceed 50 Mb/s over a 5 minute period.  This means that this plugin is unsuitable for most 32-bit production environments. To avoid this problem, use the ip_ plugin instead.  There should be no problems on 64-bit systems running 64-bit kernels.
down.label received
down.type DERIVE
down.graph no
down.cdef down,8,*
down.min 0
up.label bps
up.type DERIVE
up.negative down
up.cdef up,8,*
up.min 0
up.max -1000000
up.info Traffic of the eth0 interface. Maximum speed is -1 Mb/s.
down.max -1000000

then i went to munin on github and looked at the plugin's history just to find 2 commits around this issue:

applying munin-monitoring/munin@f982751 manually to the plugin on my machine resolved the issue. the following is the diff which might be helpful to others and maybe should be integrated with mailinabox by way of just applying the patch or maintaining a seperate ppa or somehow pulling in a version of munin newer than 2.0.19 (which was released in 2014).

--- if_.orig    2016-08-02 23:52:05.691224811 +0200
+++ if_ 2016-08-02 23:52:49.563223127 +0200
@@ -91,7 +91,7 @@
     # iwlist first)
     if [[ -r /sys/class/net/$INTERFACE/speed ]]; then
             SPEED=$(cat /sys/class/net/$INTERFACE/speed 2>/dev/null)
-            if [[ -n "$SPEED" ]]; then
+            if [ -n "$SPEED" -a "$SPEED" -gt "0" ]; then
                 echo $SPEED
                 return
             fi

if you trust the internet, you can apply this patch with this one-liner:

curl -Ls https://gist.github.com/glaszig/02cb28c47453906fb64b918c2c9d5189/raw/c92bc1b419a7649f82009f7081836e47442c83c0/if_.patch | patch -b /usr/share/munin/plugins/if_

(be root before doing so)

@monotek

This comment has been minimized.

Show comment
Hide comment
@monotek

monotek Dec 25, 2016

Same problem on Ubuntu 16.04
Patch worked.

monotek commented Dec 25, 2016

Same problem on Ubuntu 16.04
Patch worked.

@lexo13

This comment has been minimized.

Show comment
Hide comment
@lexo13

lexo13 Jan 11, 2017

Same problem on Ubuntu 16.04 for me.
Patch worked.

lexo13 commented Jan 11, 2017

Same problem on Ubuntu 16.04 for me.
Patch worked.

@dmuth

This comment has been minimized.

Show comment
Hide comment
@dmuth

dmuth Jan 14, 2017

I am sad to say that applying this patch did NOT work for me on a fresh Ubuntu 16.04 instance on Digital Ocean. I ended up putting:

SPEED=1000

on line 130 of /usr/share/munin/plugins/if_.

I don't know the Munin project (or its source) all that well--if anyone here has suggestions for how a change could be made to force Munin to treat Ethernet devices at 1 Gigabit to prevent this sort of breakage, I'm all for it--and I'll even help write the patch.

-- Doug

dmuth commented Jan 14, 2017

I am sad to say that applying this patch did NOT work for me on a fresh Ubuntu 16.04 instance on Digital Ocean. I ended up putting:

SPEED=1000

on line 130 of /usr/share/munin/plugins/if_.

I don't know the Munin project (or its source) all that well--if anyone here has suggestions for how a change could be made to force Munin to treat Ethernet devices at 1 Gigabit to prevent this sort of breakage, I'm all for it--and I'll even help write the patch.

-- Doug

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Jan 24, 2017

The patch fixed the issue on my Ubuntu 16.04 nodes.

ghost commented Jan 24, 2017

The patch fixed the issue on my Ubuntu 16.04 nodes.

@SadPyro

This comment has been minimized.

Show comment
Hide comment
@SadPyro

SadPyro Feb 2, 2017

Big thanks for the patch! Worked on my 14.04 and 16.04 Ubuntu nodes

SadPyro commented Feb 2, 2017

Big thanks for the patch! Worked on my 14.04 and 16.04 Ubuntu nodes

@foto-andreas

This comment has been minimized.

Show comment
Hide comment
@foto-andreas

foto-andreas May 14, 2017

Thank you for the patch! Ubuntu 16.04.

Thank you for the patch! Ubuntu 16.04.

@wouterbudding

This comment has been minimized.

Show comment
Hide comment
@wouterbudding

wouterbudding Jun 10, 2017

Also this patch did not help me directly, but pointed me in a good way. I added (after the patch) on line 97 the following:

            else
                echo "1000"
            return

and now finally one of my graphs is created! I think the reason the patch didn't work at first time is that the result of cat /sys/class/net/ens3/speed is " -1 " :(. Running a VPS with Leaseweb NL.

wouterbudding commented Jun 10, 2017

Also this patch did not help me directly, but pointed me in a good way. I added (after the patch) on line 97 the following:

            else
                echo "1000"
            return

and now finally one of my graphs is created! I think the reason the patch didn't work at first time is that the result of cat /sys/class/net/ens3/speed is " -1 " :(. Running a VPS with Leaseweb NL.

@pimlie

This comment has been minimized.

Show comment
Hide comment
@pimlie

pimlie Sep 3, 2017

This is a common problem for virtual network devices. In these cases you can also manually set the interface speed with ethtool (if your distro is recent enough), e.g.

ethtool -s eth0 speed 1000 duplex full

See these links:
https://serverfault.com/questions/738840/get-link-speed-of-an-virtio-net-network-adapter
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581132

pimlie commented Sep 3, 2017

This is a common problem for virtual network devices. In these cases you can also manually set the interface speed with ethtool (if your distro is recent enough), e.g.

ethtool -s eth0 speed 1000 duplex full

See these links:
https://serverfault.com/questions/738840/get-link-speed-of-an-virtio-net-network-adapter
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1581132

@kslr kslr referenced this issue in munin-monitoring/munin Jan 13, 2018

Closed

there is no speed on ens3 #902

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment