Double free in netsnmp_handler_free when snmpd exits

[jridky]

When using an extend, a double-free occurs when snmpd service shuts down.

Notice: net-snmp-5.7.2 was used for backtrace and gdb output, but current master version has the same issue.

Steps to Reproduce:

1. Patch /etc/snmp/snmpd.conf as shown below
   -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
   view all included .1 80
   ...
   access notConfigGroup "" any noauth exact all all none
   ...
   extend .1.3.6.1.4.1.2021.8 mpstat /usr/bin/mpstat -P ALL
   -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

2. Start snmpd
   -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
   # /usr/sbin/snmpd -f -LS0-6d
   -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

3. Execute a snmpwalk command triggering mpstat (can be installed, or not, same result)
   -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
   # snmpwalk -v 1 -c public localhost .1.3.6.1.4.1.2021.8
   -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

4. Stop snmpd
   -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
   # pkill -TERM snmpd
   -------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

Actual results:

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
*** Error in `/usr/sbin/snmpd': free(): invalid pointer: 0xXXX ***
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

Additional info:

The backtrace shows it crashes while freeing a pointer.

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
761 void
762 netsnmp_handler_registration_free(netsnmp_handler_registration *reginfo)
763 {
764 if (reginfo != NULL) {
765 netsnmp_handler_free(reginfo->handler); <---- HERE
766 SNMP_FREE(reginfo->handlerName);
767 SNMP_FREE(reginfo->contextName);
768 SNMP_FREE(reginfo->rootoid);
769 reginfo->rootoid_len = 0;
770 SNMP_FREE(reginfo);
771 }
772 }
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

During shutdown, we have the following:

1. Extend gets unregistered, which frees the "ereg_head" global list

(gdb) bt
# 0 _unregister_extend (eptr=0x555555883a50) at agent/extend.c:260
# 1 shutdown_extend () at agent/extend.c:316
# 2 0x00007ffff77d0915 in _shutdown_mib_modules (majorID=, minorID=,
serve=, client=) at ../agent/mibgroup/mib_module_shutdown.h:34
# 3 0x00007ffff721354f in snmp_call_callbacks (major=major@entry=0, minor=minor@entry=2,
caller_arg=caller_arg@entry=0x0) at callback.c:363
# 4 0x00007ffff71e62f7 in snmp_shutdown (type=) at snmp_api.c:910
# 5 0x00005555555579d4 in main (argc=, argv=) at snmpd.c:1135

2. Cache tree gets freed, leading again to freeing the already freed extend

(gdb) bt
# 0 0x00007ffff574d387 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
# 1 0x00007ffff574ea78 in __GI_abort () at abort.c:90
# 2 0x00007ffff578fed7 in __libc_message (do_abort=do_abort@entry=2,
fmt=fmt@entry=0x7ffff58a2350 "*** Error in `%s': %s: 0x%s ***\n")
at ../sysdeps/unix/sysv/linux/libc_fatal.c:196
# 3 0x00007ffff5798299 in malloc_printerr (ar_ptr=0x7ffff5ade760 <main_arena>, ptr=,
str=0x7ffff589fb50 "free(): invalid pointer", action=3) at malloc.c:4967
# 4 _int_free (av=0x7ffff5ade760 <main_arena>, p=, have_lock=0) at malloc.c:3843
# 5 0x00007ffff7b95b02 in netsnmp_handler_registration_free (reginfo=0x5555559ba7b0)
at agent_handler.c:765
# 6 0x00007ffff7b9886e in netsnmp_subtree_free (a=0x5555559ba6c0) at agent_registry.c:474
# 7 0x00007ffff7b9915c in clear_subtree (sub=) at agent_registry.c:961
# 8 0x00007ffff7b991ac in clear_context () at agent_registry.c:427
# 9 0x00007ffff7ba4f1e in shutdown_agent () at snmp_vars.c:370
# 10 0x00005555555579de in main (argc=, argv=) at snmpd.c:1137

In frame 6, we have the subtree reference the extend handler (reginfo), which has already been freed in step 1, causing the crash:

21 typedef struct netsnmp_subtree_s {
:
41 netsnmp_handler_registration reginfo; / new API */
:
45 } netsnmp_subtree;

original report: https://bugzilla.redhat.com/show_bug.cgi?id=1823841

[rmetrich]

Another scenario is while performing a reload:

1. Have the extend in /etc/snmp/snmpd.conf

   extend .1.3.6.1.4.1.2021.8 mpstat /usr/bin/mpstat -P ALL
   

2. Start snmpd

   # /usr/sbin/snmpd -f -Le0-6d
   

3. Remove the extend

   # extend .1.3.6.1.4.1.2021.8 mpstat /usr/bin/mpstat -P ALL
   

4. Reload snmpd

   # pkill -HUP snmpd
   

[bvanassche]

My attempt to reproduce this crash is available at https://github.com/net-snmp/net-snmp/blob/master/testing/fulltests/default/T065agentextend_oid_simple. That test passes against the v5.8 and master branches.

[jridky]

./configure --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --disable-static --enable-shared '--with-cflags=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_RPM_4_4_COMPAT' '--with-ldflags=-Wl,-z,relro -Wl,-z,now' --with-sys-location=Unknown --with-logfile=/var/log/snmpd.log --with-persistent-directory=/var/lib/net-snmp '--with-mib-modules=host agentx smux ucd-snmp/diskio tcp-mib udp-mib mibII/mta_sendmail ip-mib/ipv4InterfaceTable ip-mib/ipv6InterfaceTable ip-mib/ipAddressPrefixTable/ipAddressPrefixTable ip-mib/ipDefaultRouterTable/ipDefaultRouterTable ip-mib/ipv6ScopeZoneIndexTable ip-mib/ipIfStatsTable sctp-mib rmon-mib etherlike-mib ucd-snmp/lmsensorsMib' --with-libwrap=yes --sysconfdir=/etc --enable-ipv6 --enable-ucd-snmp-compatibility --with-openssl --with-pic --enable-embedded-perl --enable-as-needed --with-perl-modules=INSTALLDIRS=vendor --enable-mfd-rewrites --enable-local-smux --with-temp-file-pattern=/var/run/net-snmp/snmp-tmp-XXXXXX '--with-transports=DTLSUDP TLSTCP' --with-security-modules=tsm --with-mysql --with-systemd --with-sys-contact=root@localhost

[bvanassche]

If anyone can provide me a way to reproduce this with the master or v5.8 branch I will have a closer look at this issue.