collectd-5.6.2 memory leak #2303

marcin-github · 2017-05-29T14:26:47Z

Version of collectd: 5.6.2
Operating system / distribution: Gentoo

Expected behavior

RES memory shouldn't grow up to 2GB

Actual behavior

After a week or two of working RES memory usage grows and grows. Today I saw that collectd used almost 2GB of ram (on RES)/

This is what I got when I run collectd under valgrind:
# valgrind --leak-check=full --trace-children=yes --log-file=/tmp/coll-val.log --leak-resolution=med --show-leak-kinds=definite /usr/sbin/collectd -C /etc/collectd.conf -P /run/collectd/coll
ectd.pid -f

==10052== Memcheck, a memory error detector
==10052== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==10052== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==10052== Command: /usr/sbin/collectd -C /etc/collectd.conf -P /run/collectd/collectd.pid -f
==10052== Parent PID: 12265
==10052==
==10052== Syscall param prctl(arg4) contains uninitialised byte(s)
==10052== at 0x4D31A0A: prctl (syscall-template.S:84)
==10052== by 0x12522A: check_capability (common.c:1532)
==10052== by 0x4218C5E: dns_init (dns.c:311)
==10052== by 0x118ECE: plugin_init_all (plugin.c:1581)
==10052== by 0x10EE58: do_init (collectd.c:297)
==10052== by 0x10EE58: main (collectd.c:656)
==10052==
==10052==
==10052== HEAP SUMMARY:
==10052== in use at exit: 1,069,105 bytes in 8,923 blocks
==10052== total heap usage: 376,369 allocs, 367,446 frees, 35,566,813 bytes allocated
==10052==
==10052== 8 bytes in 1 blocks are definitely lost in loss record 12 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x4CC5288: strdup (strdup.c:42)
==10052== by 0x12907B: yyparse (parser.y:110)
==10052== by 0x1258E3: oconfig_parse_fh (oconfig.c:62)
==10052== by 0x1258E3: oconfig_parse_file (oconfig.c:89)
==10052== by 0x111276: cf_read_file (configfile.c:604)
==10052== by 0x111276: cf_read_generic (configfile.c:774)
==10052== by 0x11219B: cf_read (configfile.c:1006)
==10052== by 0x10EB18: main (collectd.c:523)
==10052==
==10052== 466 bytes in 43 blocks are definitely lost in loss record 368 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xB81CC6F: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0xE9FC9CF: ???
==10052==
==10052== 466 bytes in 43 blocks are definitely lost in loss record 369 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xB01BC6F: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0x538245F: ???
==10052==
==10052== 466 bytes in 43 blocks are definitely lost in loss record 370 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xB81CC6F: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0x545D5FF: ???
==10052==
==10052== 484 bytes in 47 blocks are definitely lost in loss record 371 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xA019C6F: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0x5328ABF: ???
==10052==
==10052== 502 bytes in 51 blocks are definitely lost in loss record 372 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xB01BC6F: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0x5328ABF: ???
==10052==
==10052== 520 bytes in 55 blocks are definitely lost in loss record 373 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xB81CC6F: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0x5328ABF: ???
==10052==
==10052== 864 bytes in 1 blocks are definitely lost in loss record 379 of 407
==10052== at 0x402DD7A: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x10FEC9: cf_ci_append_children.isra.2 (configfile.c:505)
==10052== by 0x111216: cf_read_generic (configfile.c:789)
==10052== by 0x11138B: cf_include_all (configfile.c:552)
==10052== by 0x11138B: cf_read_file (configfile.c:610)
==10052== by 0x11138B: cf_read_generic (configfile.c:774)
==10052== by 0x11219B: cf_read (configfile.c:1006)
==10052== by 0x10EB18: main (collectd.c:523)
==10052==
==10052== 1,007 bytes in 109 blocks are definitely lost in loss record 380 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xA019DBF: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0xE0CC97F: ???
==10052==
==10052== 1,007 bytes in 109 blocks are definitely lost in loss record 381 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xB81CDBF: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0x545D5FF: ???
==10052==
==10052== 2,014 bytes in 218 blocks are definitely lost in loss record 390 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xB81CDBF: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0xD797D1F: ???
==10052==
==10052== 2,014 bytes in 218 blocks are definitely lost in loss record 391 of 407
==10052== at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==10052== by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==10052== by 0xB01BDBF: ???
==10052== by 0xF16AC631E29C2BFF: ???
==10052== by 0x538245F: ???
==10052==
==10052== LEAK SUMMARY:
==10052== definitely lost: 9,818 bytes in 938 blocks
==10052== indirectly lost: 0 bytes in 0 blocks
==10052== possibly lost: 288 bytes in 1 blocks
==10052== still reachable: 1,058,999 bytes in 7,984 blocks
==10052== suppressed: 0 bytes in 0 blocks
==10052== Reachable blocks (those to which a pointer was found) are not shown.
==10052== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==10052==
==10052== For counts of detected and suppressed errors, rerun with: -v
==10052== Use --track-origins=yes to see where uninitialised values come from
==10052== ERROR SUMMARY: 14 errors from 14 contexts (suppressed: 0 from 0)

Steps to reproduce

step 1
step 2
step 3

marcin-github · 2017-05-29T14:32:29Z

xml comes from bind plugin. bind plugin is configured as below:

<Plugin "bind">
   URL "http://localhost:8053/";
   ParseTime       false
   OpCodes         true
   QTypes          true
   ServerStats     true
   ZoneMaintStats  true
   ResolverStats   false
   MemoryStats     true
    <View "external">
        QTypes        true
        ResolverStats true
        CacheRRSets   true
        Zone "foo.bar"
    </View>
</Plugin>

I commented out loading bind plugin. After this valgrind didn't mention libxml2:

==20537== Memcheck, a memory error detector
==20537== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==20537== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==20537== Command: /usr/sbin/collectd -C /etc/collectd.conf -P /run/collectd/collectd.pid -f
==20537== Parent PID: 12265
==20537== 
==20537== Syscall param prctl(arg4) contains uninitialised byte(s)
==20537==    at 0x4D31A0A: prctl (syscall-template.S:84)
==20537==    by 0x12522A: check_capability (common.c:1532)
==20537==    by 0x418CC5E: dns_init (dns.c:311)
==20537==    by 0x118ECE: plugin_init_all (plugin.c:1581)
==20537==    by 0x10EE58: do_init (collectd.c:297)
==20537==    by 0x10EE58: main (collectd.c:656)
==20537== 
==20537== 
==20537== HEAP SUMMARY:
==20537==     in use at exit: 527,490 bytes in 3,716 blocks
==20537==   total heap usage: 27,804 allocs, 24,088 frees, 7,069,157 bytes allocated
==20537== 
==20537== 8 bytes in 1 blocks are definitely lost in loss record 10 of 77
==20537==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==20537==    by 0x4CC5288: strdup (strdup.c:42)
==20537==    by 0x12907B: yyparse (parser.y:110)
==20537==    by 0x1258E3: oconfig_parse_fh (oconfig.c:62)
==20537==    by 0x1258E3: oconfig_parse_file (oconfig.c:89)
==20537==    by 0x111276: cf_read_file (configfile.c:604)
==20537==    by 0x111276: cf_read_generic (configfile.c:774)
==20537==    by 0x11219B: cf_read (configfile.c:1006)
==20537==    by 0x10EB18: main (collectd.c:523)
==20537== 
==20537== 816 bytes in 1 blocks are definitely lost in loss record 61 of 77
==20537==    at 0x402DD7A: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==20537==    by 0x10FEC9: cf_ci_append_children.isra.2 (configfile.c:505)
==20537==    by 0x111216: cf_read_generic (configfile.c:789)
==20537==    by 0x11138B: cf_include_all (configfile.c:552)
==20537==    by 0x11138B: cf_read_file (configfile.c:610)
==20537==    by 0x11138B: cf_read_generic (configfile.c:774)
==20537==    by 0x11219B: cf_read (configfile.c:1006)
==20537==    by 0x10EB18: main (collectd.c:523)
==20537== 
==20537== LEAK SUMMARY:
==20537==    definitely lost: 824 bytes in 2 blocks
==20537==    indirectly lost: 0 bytes in 0 blocks
==20537==      possibly lost: 288 bytes in 1 blocks
==20537==    still reachable: 526,378 bytes in 3,713 blocks
==20537==         suppressed: 0 bytes in 0 blocks
==20537== Reachable blocks (those to which a pointer was found) are not shown.
==20537== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==20537== 
==20537== For counts of detected and suppressed errors, rerun with: -v
==20537== Use --track-origins=yes to see where uninitialised values come from
==20537== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0)

rubenk · 2017-05-29T15:30:00Z

Can you reproduce this on master?

marcin-github · 2017-05-30T08:14:38Z

I compiled master version. I run it only under valgrind:

==30859== Memcheck, a memory error detector
==30859== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==30859== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==30859== Command: /usr/sbin/collectd -C /etc/collectd.conf -P /run/collectd/collectd.pid -f
==30859== Parent PID: 12265
==30859== 
==30859== 
==30859== HEAP SUMMARY:
==30859==     in use at exit: 1,115,947 bytes in 14,035 blocks
==30859==   total heap usage: 2,389,991 allocs, 2,375,956 frees, 196,062,708 bytes allocated
==30859== 
==30859== 8 bytes in 1 blocks are definitely lost in loss record 12 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x4EC9288: strdup (strdup.c:42)
==30859==    by 0x12783B: yyparse (parser.y:110)
==30859==    by 0x1240F3: oconfig_parse_fh (oconfig.c:62)
==30859==    by 0x1240F3: oconfig_parse_file (oconfig.c:89)
==30859==    by 0x110FB6: cf_read_file (configfile.c:604)
==30859==    by 0x110FB6: cf_read_generic (configfile.c:774)
==30859==    by 0x111EAB: cf_read (configfile.c:1005)
==30859==    by 0x10E728: main (collectd.c:522)
==30859== 
==30859== 53 bytes in 5 blocks are definitely lost in loss record 312 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xBBCEDBF: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xF332F0F: ???
==30859== 
==30859== 56 bytes in 8 blocks are definitely lost in loss record 313 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xB3CDDBF: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xDCAB27F: ???
==30859== 
==30859== 101 bytes in 10 blocks are definitely lost in loss record 354 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xB3CDDBF: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x532E9CF: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 375 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xABCCC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xD26C1AF: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 376 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0x9BCAC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xE5F133F: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 377 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xABCCC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x54A83EF: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 378 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0x9BCAC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xDD0A90F: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 379 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xABCCC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x534BD5F: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 380 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0x9BCAC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x532052F: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 381 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xB3CDC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x535DA1F: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 382 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xA3CBC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x532052F: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 383 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xB3CDC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xDCAB27F: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 384 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xABCCC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xDD0596F: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 385 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xB3CDC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x532E9CF: ???
==30859== 
==30859== 466 bytes in 43 blocks are definitely lost in loss record 386 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0x9BCAC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xDA75E9F: ???
==30859== 
==30859== 538 bytes in 59 blocks are definitely lost in loss record 387 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xA3CBC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x534BD5F: ???
==30859== 
==30859== 719 bytes in 75 blocks are definitely lost in loss record 391 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xBBCEDBF: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xD491CBF: ???
==30859== 
==30859== 849 bytes in 95 blocks are definitely lost in loss record 393 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xA3CBDBF: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xEC98EBF: ???
==30859== 
==30859== 873 (864 direct, 9 indirect) bytes in 1 blocks are definitely lost in loss record 394 of 448
==30859==    at 0x402DD7A: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x10FA79: cf_ci_append_children.isra.2 (configfile.c:505)
==30859==    by 0x110F56: cf_read_generic (configfile.c:789)
==30859==    by 0x1110CB: cf_include_all (configfile.c:552)
==30859==    by 0x1110CB: cf_read_file (configfile.c:610)
==30859==    by 0x1110CB: cf_read_generic (configfile.c:774)
==30859==    by 0x111EAB: cf_read (configfile.c:1005)
==30859==    by 0x10E728: main (collectd.c:522)
==30859== 
==30859== 919 bytes in 107 blocks are definitely lost in loss record 395 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xBBCEDBF: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x521F4FF: ???
==30859== 
==30859== 932 bytes in 86 blocks are definitely lost in loss record 396 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xA3CBC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xEC98EBF: ???
==30859== 
==30859== 932 bytes in 86 blocks are definitely lost in loss record 397 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xB3CDC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x533B16F: ???
==30859== 
==30859== 932 bytes in 86 blocks are definitely lost in loss record 398 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0x9BCAC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xE95050F: ???
==30859== 
==30859== 932 bytes in 86 blocks are definitely lost in loss record 399 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0x9BCAC6F: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x52AD01F: ???
==30859== 
==30859== 981 bytes in 106 blocks are definitely lost in loss record 400 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xBBCEDBF: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0xE92035F: ???
==30859== 
==30859== 1,000 bytes in 108 blocks are definitely lost in loss record 402 of 448
==30859==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==30859==    by 0x58C9875: xmlStrndup (in /usr/lib64/libxml2.so.2.9.4)
==30859==    by 0xA3CBDBF: ???
==30859==    by 0x4A528ED55EA9CAFF: ???
==30859==    by 0x532052F: ???
==30859== 
[...]
==30859== 
==30859== LEAK SUMMARY:
==30859==    definitely lost: 59,012 bytes in 6,085 blocks
==30859==    indirectly lost: 9 bytes in 1 blocks
==30859==      possibly lost: 272 bytes in 1 blocks
==30859==    still reachable: 1,056,654 bytes in 7,948 blocks
==30859==         suppressed: 0 bytes in 0 blocks
==30859== Reachable blocks (those to which a pointer was found) are not shown.
==30859== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==30859== 
==30859== For counts of detected and suppressed errors, rerun with: -v
==30859== ERROR SUMMARY: 54 errors from 54 contexts (suppressed: 0 from 0)

Now I run collectd without valgrind, I'll know more about memory usage after a couple of days. As for now collectd uses 14MB of RES.

rubenk · 2017-05-30T15:07:39Z

Could you see if this helps?

diff --git a/src/bind.c b/src/bind.c
index 02bffd3d..9eeb7fec 100644
--- a/src/bind.c
+++ b/src/bind.c
@@ -522,8 +522,10 @@ static int bind_parse_generic_name_value(const char *xpath_expression, /* {{{ */
         status = bind_xml_read_gauge(doc, counter, &value.gauge);
       else
         status = bind_xml_read_derive(doc, counter, &value.derive);
-      if (status != 0)
+      if (status != 0) {
+        xmlFree(name);
         continue;
+      }

       status = (*list_callback)(name, value, current_time, user_data);
       if (status == 0)

rubenk · 2017-05-30T15:34:32Z

Scratch that, I found a few more. Could you give #2306 a try?

marcin-github · 2017-05-31T07:46:31Z

I run collectd for a 10 hours, RES has stable size. Valgrind says:

==20761== Memcheck, a memory error detector
==20761== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==20761== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==20761== Command: /usr/sbin/collectd -C /etc/collectd.conf -P /run/collectd/collectd.pid -f
==20761== Parent PID: 12265
==20761==
==20761==
==20761== HEAP SUMMARY:
==20761==     in use at exit: 1,062,026 bytes in 7,984 blocks
==20761==   total heap usage: 4,039,029 allocs, 4,031,045 frees, 327,768,207 bytes allocated
==20761==
==20761== 8 bytes in 1 blocks are definitely lost in loss record 12 of 383
==20761==    at 0x402ADD5: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==20761==    by 0x4EC9288: strdup (strdup.c:42)
==20761==    by 0x12783B: yyparse (parser.y:110)
==20761==    by 0x1240F3: oconfig_parse_fh (oconfig.c:62)
==20761==    by 0x1240F3: oconfig_parse_file (oconfig.c:89)
==20761==    by 0x110FB6: cf_read_file (configfile.c:604)
==20761==    by 0x110FB6: cf_read_generic (configfile.c:774)
==20761==    by 0x111EAB: cf_read (configfile.c:1005)
==20761==    by 0x10E728: main (collectd.c:522)
==20761==
==20761== 864 bytes in 1 blocks are definitely lost in loss record 361 of 383
==20761==    at 0x402DD7A: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==20761==    by 0x10FA79: cf_ci_append_children.isra.2 (configfile.c:505)
==20761==    by 0x110F56: cf_read_generic (configfile.c:789)
==20761==    by 0x1110CB: cf_include_all (configfile.c:552)
==20761==    by 0x1110CB: cf_read_file (configfile.c:610)
==20761==    by 0x1110CB: cf_read_generic (configfile.c:774)
==20761==    by 0x111EAB: cf_read (configfile.c:1005)
==20761==    by 0x10E728: main (collectd.c:522)
==20761==
==20761== LEAK SUMMARY:
==20761==    definitely lost: 872 bytes in 2 blocks
==20761==    indirectly lost: 0 bytes in 0 blocks
==20761==      possibly lost: 272 bytes in 1 blocks
==20761==    still reachable: 1,060,882 bytes in 7,981 blocks
==20761==         suppressed: 0 bytes in 0 blocks
==20761== Reachable blocks (those to which a pointer was found) are not shown.
==20761== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==20761==
==20761== For counts of detected and suppressed errors, rerun with: -v
==20761== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)

For me it looks good, thank you for quick fix. Will be this fix backported to 5.6 branch?

rubenk · 2017-05-31T08:18:16Z

Nice, thanks for testing!

Sure, I'll backport this to 5.6 in a few days. Any reason you're not on 5.7 yet? It contains a lot more of those kinds of fixes ;)

marcin-github · 2017-05-31T08:21:06Z

I'm waiting for ebuild in Gentoo:)

rubenk · 2017-05-31T08:27:23Z

Just summon @Whissi, he will fix that ;)

See collectd/collectd#2303 Package-Manager: Portage-2.3.5, Repoman-2.3.2

Whissi · 2017-06-01T18:52:04Z

Thanks for the ping, @rubenk!

And I'll add collect-5.7 probably in the next week.

rubenk · 2017-06-01T20:07:51Z

Thanks Thomas!

marcin-github · 2017-06-02T08:56:02Z

Thanks Ruben & Thomas.

octo added the Pending feedback label May 29, 2017

octo added Pending contributor action and removed Pending feedback labels May 29, 2017

octo added Pending feedback and removed Pending contributor action labels May 30, 2017

octo added Pending contributor action and removed Pending feedback labels May 30, 2017

rubenk mentioned this issue May 30, 2017

Bind plugin: plug a few leaks #2306

Closed

octo added Pending feedback and removed Pending contributor action labels May 31, 2017

octo added Pending contributor action and removed Pending feedback labels May 31, 2017

rubenk closed this as completed in fd01cdd Jun 1, 2017

gentoo-bot pushed a commit to gentoo/gentoo that referenced this issue Jun 1, 2017

app-admin/collectd: Rev bump to fix a memory leak in bind plugin

d7dcb46

See collectd/collectd#2303 Package-Manager: Portage-2.3.5, Repoman-2.3.2

octo added the Bug A genuine bug label Jun 5, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

collectd-5.6.2 memory leak #2303

collectd-5.6.2 memory leak #2303

marcin-github commented May 29, 2017

marcin-github commented May 29, 2017

rubenk commented May 29, 2017

marcin-github commented May 30, 2017

rubenk commented May 30, 2017

rubenk commented May 30, 2017

marcin-github commented May 31, 2017

rubenk commented May 31, 2017

marcin-github commented May 31, 2017

rubenk commented May 31, 2017

Whissi commented Jun 1, 2017

rubenk commented Jun 1, 2017

marcin-github commented Jun 2, 2017

collectd-5.6.2 memory leak #2303

collectd-5.6.2 memory leak #2303

Comments

marcin-github commented May 29, 2017

Expected behavior

Actual behavior

Steps to reproduce

marcin-github commented May 29, 2017

rubenk commented May 29, 2017

marcin-github commented May 30, 2017

rubenk commented May 30, 2017

rubenk commented May 30, 2017

marcin-github commented May 31, 2017

rubenk commented May 31, 2017

marcin-github commented May 31, 2017

rubenk commented May 31, 2017

Whissi commented Jun 1, 2017

rubenk commented Jun 1, 2017

marcin-github commented Jun 2, 2017