Adds metrics and tests for UDP receive and send buffer errors

[pgporada]

Context: https://jvns.ca/blog/2016/08/24/find-out-where-youre-dropping-packets/

[pgporada]

Per Circle CI output, I'm not entirely sure why the following test broke.

time="2019-11-07T21:02:21Z" level=debug msg="file not found when retrieving stats for pid {1 collector/fixtures/proc}: \"open collector/fixtures/proc/1/stat: no such file or directory\"" source="processes_linux.go:111"

[SuperQ]

The test is failing because you need to update collector/fixtures/e2e-output.txt, and collector/fixtures/e2e-64k-page-output.txt with the new metrics output for the end-to-end test.

[SuperQ]

LGTM, thanks. I will merge as soon as the end-to-end is fixed up.

[pgporada]

Thank you for the pointers! That seems to have done the trick.

[SuperQ]

It's not obvious in the end-to-end test, but please add the same lines to collector/fixtures/e2e-64k-page-output.txt.

[pgporada]

[SuperQ]

Hrm, looking at some of my metrics, it seems like there's some overlap and now you've changed it to do UdpLite.

The Udp6RcvbufErrors is covered by Udp6InErrors. But we should include Udp6SndbufErrors.

$ grep -i errors /proc/net/snmp6
Ip6InHdrErrors                  	0
Ip6InTooBigErrors               	0
Ip6InAddrErrors                 	0
Icmp6InErrors                   	7
Icmp6OutErrors                  	0
Icmp6InCsumErrors               	0
Udp6InErrors                    	1479
Udp6RcvbufErrors                	1479
Udp6SndbufErrors                	9
Udp6InCsumErrors                	0
UdpLite6InErrors                	0
UdpLite6RcvbufErrors            	0
UdpLite6SndbufErrors            	0
UdpLite6InCsumErrors            	0

[SuperQ]

I wonder if Udp6RcvbufErrors + Udp6InCsumErrors == Udp6InErrors.

[pgporada]

Is there a reason to not gather UdpLite metrics? I see that I had messed up the regex instead of making it better. Switching it to Udp(Lite)?6?_(InDatagrams|OutDatagrams|NoPorts|RcvbufErrors|SndbufErrors) would allow collection of

Udp6InDatagrams
Udp6OutDatagrams
Udp6RcvbufErrors
Udp6SndbufErrors
UdpLite6RcvbufErrors
UdpLite6SndbufErrors
UdpInDatagrams
UdpOutDatagrams
UdpRcvbufErrors
UdpSndbufErrors
UdpLiteRcvbufErrors
UdpLiteSndbufErrors

This post sheds some light on UdpRcvbufErrors vs UdpInErrors netdata/netdata#4086 (comment)

[SuperQ]

We're trying to avoid ingesting too many metrics, so we prefer to take the highest top level covering metric for some of these cases. If X is Y+Z, we prefer to just have X and leave it up to the user to figure out if it's Y or Z for as many cases as possible.

[pgporada]

That makes sense, I'll remove the UdpLite portion of the regex.

[SuperQ]

I'd like to find some more docs or some kernel source that explains these things better. I'm not saying no to this PR, but I want to do some more detailed investigation before we finish things.

[pgporada]

Here are the cases where sndbuferrors is incremented, but inerrors is not.

• https://github.com/torvalds/linux/blob/master/net/ipv4/udp.c#L893-L897
• https://github.com/torvalds/linux/blob/master/net/ipv4/udp.c#L1229-L1232

Cases where rcvbuferrors is incremented, and inerrors is incremented

• https://github.com/torvalds/linux/blob/master/net/ipv4/udp.c#L1972-L1976
• https://github.com/torvalds/linux/blob/master/net/ipv4/udp.c#L2174-L2181

There are also several occurrences where inerrors is incremented that are unrelated to sndbuferrors and rcvbuferrors. While I am not a C coder or experienced at all in reading kernel code, I do still think having this metric will be useful to some network admins myself included.

[discordianfish]

@SuperQ Leaving this to you, I'm fine either way.

[SuperQ]

Sorry for the long delay in review. This fell out of my TODO list.

[pgporada]

Thank you!