Segmentation fault while freeing memory

[puetzp]

Hey,

I am encountering a problem using the program to query a device. The program terminates with a seg fault.
Maybe the following debugging output helps you to track down the issue:

root@<snip>:/var/log # gdb --args /usr/lib/nagios/plugins/contrib/check_interfaces '-h' '<snip>' '--user' '<snip>' '--auth-proto' '<snip>' '--auth-phrase' '<snip>' '--priv-proto' '<snip>' '--priv-phrase' '<snip>' '-x' '27' '-r' ''
<...>
Reading symbols from /usr/lib/nagios/plugins/contrib/check_interfaces...(no debugging symbols found)...done.
(gdb) run
Starting program: /usr/lib/nagios/plugins/contrib/check_interfaces -h <snip> --user <snip> --auth-proto <snip> --auth-phrase <snip> --priv-proto <snip> --priv-phrase <snip> -x 27 -r <snip>
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6ebeabf in malloc_consolidate (av=av@entry=0x7ffff71eb620 <main_arena>) at malloc.c:4165
4165    malloc.c: Datei oder Verzeichnis nicht gefunden.
(gdb) bt
#0  0x00007ffff6ebeabf in malloc_consolidate (av=av@entry=0x7ffff71eb620 <main_arena>) at malloc.c:4165
#1  0x00007ffff6ebf641 in _int_free (av=0x7ffff71eb620 <main_arena>, p=<optimized out>, have_lock=0) at malloc.c:4057
#2  0x00007ffff792b168 in snmp_free_varbind () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30
#3  0x00007ffff792b1b4 in snmp_free_pdu () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30
#4  0x000000000040217a in ?? ()
#5  0x00007ffff6e67b45 in __libc_start_main (main=0x401700, argc=17, argv=0x7fffffffe558, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe548)
    at libc-start.c:287
#6  0x0000000000403ae5 in ?? ()

[dnsmichi]

Which platform, version and plugin binary version is that? Can you please add a bt full output as well? This sounds more like an issue with net-snmp itself on this platform. Which net-snmp version is used?

[puetzp]

Here is the info:

(gdb) bt full
#0  0x00007ffff6ec2634 in __GI___libc_free (mem=0x611620) at malloc.c:2945
        ar_ptr = <optimized out>
        p = 0x611610
        hook = <optimized out>
#1  0x00007ffff792b1fd in snmp_free_pdu () from /usr/lib/x86_64-linux-gnu/libnetsnmp.so.30
No symbol table info available.
#2  0x000000000040217a in ?? ()
No symbol table info available.
#3  0x00007ffff6e67b45 in __libc_start_main (main=0x401700, argc=17, argv=0x7fffffffe548, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe538)
    at libc-start.c:287
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 4445349801132744119, 4209340, 140737488348480, 0, 0, -4445349800241603145, -4445365250230526537}, mask_was_saved = 0}}, priv = {
            pad = {0x0, 0x0, 0x7fffffffe5d8, 0x7ffff7ffe1b0}, data = {prev = 0x0, cleanup = 0x0, canceltype = -6696}}}
        not_first_call = <optimized out>
#4  0x0000000000403ae5 in ?? ()
No symbol table info available.

Dist info:

Distributor ID: Debian
Description:    Debian GNU/Linux 8.11 (jessie)
Release:        8.11
Codename:       jessie

libs:

<snip>:~ # apt-cache policy libsnmp-base
libsnmp-base:
  Installiert:           5.7.2.1+dfsg-1+deb8u2
<...>
remote@<snip>:~ # apt-cache policy libsnmp-dev
libsnmp-dev:
  Installiert:           5.7.2.1+dfsg-1+deb8u2
<...>
remote@<snip>:~ # apt-cache policy libsnmp30:amd64
libsnmp30:
  Installiert:           5.7.2.1+dfsg-1+deb8u2
<...>

I'm using the latest version of the plugin (1.4).

[dnsmichi]

@mxhash can you take a deeper look please? I haven't been involved in development here, nor understand the specific code parts where this could be triggered. I've just updated the README for getting things to compile easier.

[puetzp]

I agree that net-snmp seems to be the issue. Executing the plugin in a Debian Stretch environment with libsnmp v5.6.3 does not fix the problem. So it might not have been fixed in a newer version yet.
I should probably open an issue in net-snmp.

[phil-or]

I think I have the same problem. While executing the check, a Segmentation Fault returned.
But this error occurs just on a few hosts.

[root@]# gdb --args '/usr/lib64/nagios/plugins/check_interfaces' '--aliases' '--auth-phrase' 'xx' '--auth-proto' 'xx' '--hostname' 'xx' '--if-names' '--priv-phrase' 'xx' '--priv-proto' 'xx' '--regex' '' '--timeout' '30000' '--user' 'xx'
(gdb) run
Starting program: /usr/lib64/nagios/plugins/check_interfaces --aliases --auth-phrase xx --auth-proto xx --hostname xx --if-names --priv-phrase xx --priv-proto xx --regex xx --timeout 30000 --user xx
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6bb011e in _int_free () from /lib64/libc.so.6

Version:
check_interfaces 1.4
CentOS 7

Sometimes the error occurs, sometimes not...

[mxhash]

I give some tests a try. Any steps to reproduce the issue?

[phil-or]

We only have problems with checks on Nexus7700. But sometimes it works, sometimes not....

[mxhash]

Ah ok ;-) bad to test for me... Does this problem also occur when querying with snmp utils?

[puetzp]

Thank you for following up on this issue!
I queried all OIDs manually using a mix of snmpwalk/get/bulkget and the error did not occur whereas the program would produce a seg fault ...
Tested on a Cisco ASA.

[phil-or]

I also did a snmpwalk over all OIDs of this device and didn't get an error

[mxhash]

Thank you guys. I'll give it a try.