Memory leak with gp_port_info_list_new()/gp_port_info_list_load() and GPPortInfoList

[define-private-public]

I'm using verison: 2.5.12-1

Here is my relevant code:

void MGPhoto2Camera::__scanForCameras() {
    GPContext *scanContext;
    CameraList *camList;
    GPPortInfoList *portList;
    CameraAbilitiesList *abilitiesList;
    CameraAbilities abilities;
    const char *name, *value;

    // Init
    scanContext = gp_context_new();
    gp_list_new(&camList);
    gp_port_info_list_new(&portList);           // Memory leak here, sort of...
    gp_abilities_list_new(&abilitiesList);

    // Query
    gp_port_info_list_load(portList);
    gp_abilities_list_load(abilitiesList, scanContext);
    gp_abilities_list_detect(abilitiesList, portList, camList, scanContext);

    // See how many we got
    int numCams = gp_list_count(camList);
    QVector<QString> names;
    QVector<QString> ports;
    for (int i = 0; i < numCams; i++) {
        // Pull out some info
        gp_list_get_name(camList, i, &name);
        gp_list_get_value(camList, i, &value);

        // Init data
        gp_abilities_list_get_abilities(abilitiesList, i, &abilities);

        // Store it in the list If it's a camera
        if (abilities.device_type == GP_DEVICE_STILL_CAMERA) {
            names.append(QString(name));
            ports.append(QString(value));
        }
    }

    // Cleanup GPhoto2 data
    gp_list_unref(camList);
    gp_port_info_list_free(portList);
    gp_abilities_list_free(abilitiesList);
    gp_context_unref(scanContext);

    if (!__usbPort.isEmpty() && !ports.contains(__usbPort)) {
        // Did we have a connect before, that's not there anymore?
        __log->warn("Camera {} on port {} has disappeared", __modelName, __usbPort);
        __usbPort.clear();
        __modelName.clear();
    } else if (__usbPort.isEmpty() && (!ports.isEmpty())) {
        // What if we don't have one, but a new one has been detected?  Set it!
        // Grab the first
        __usbPort = ports[0];
        __modelName = names[0];
        __log->info("Camera {} has been detected on port {}", __modelName, __usbPort);
    }
}

If I comment out that one like tagged with "memory leak," I no longer have any more memory leaks, but that kind of sucks since I can't query for connected cameras then. What's going on? Am I not using this library correct? Is the memory not being freed properly?

[define-private-public]

You've got a memory leak.

I tried using only the gp_camera_autodetect() function in a loop with uses some of these functions. The memory usage was slowly climbing. This time I tried installing the latest version from source. It was there.

[msmeissn]

I tried reproducing it, but failed.

running above code 100 times does not change leakage compared to 1 time.

(I used 2.5.16 and 2.5.12 ... )

It is weird, which libusb version is used?

[define-private-public]

ben@mybox:~$ apt-cache show libusb-dev 
Package: libusb-dev
Priority: optional
Section: libdevel
Installed-Size: 229
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Aurelien Jarno <aurel32@debian.org>
Architecture: amd64
Source: libusb
Version: 2:0.1.12-30
Depends: libusb-0.1-4 (= 2:0.1.12-30), libc6-dev | libc-dev
Filename: pool/main/libu/libusb/libusb-dev_0.1.12-30_amd64.deb
Size: 30278
MD5sum: e0c148274a9ddc89d077932fc4e1d525
SHA1: cbbf50b59c299a1665c83060faad811b3da253f5
SHA256: 2a7b32261bd29e1fc0bcfd75c56aef838d9502b292d045fa4c0344b8b9942c63
Description-en: userspace USB programming library development files
 Library for programming USB applications without the knowledge
 of Linux kernel internals.
 .
 This package contains what you need for compiling sources that
 use this library in your own code.
Description-md5: 07c28a16589d7bc7f4eda04968bb1f5a
Homepage: http://www.linux-usb.org/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
Supported: 9m

Looks like I'm out of date.

[define-private-public]

I'm still having issues with this. Can you tell me which version of libusb you are using? And what platform you're on? I'm on Ubuntu 18.04. I'd really like to get this issue resolved as soon as possible.

[msmeissn]

libusb 1.0 is curently the one to be used, libusb 0.1 is deprecated for a while now

[define-private-public]

Here's the one I have installed right now:

Package: libusb-1.0-0
Architecture: amd64
Version: 2:1.0.21-2
Multi-Arch: same
Priority: standard
Section: libs
Source: libusb-1.0
Origin: Ubuntu
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Aurelien Jarno <aurel32@debian.org>
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Installed-Size: 124
Depends: libc6 (>= 2.17), libudev1 (>= 183)
Filename: pool/main/libu/libusb-1.0/libusb-1.0-0_1.0.21-2_amd64.deb
Size: 43258
MD5sum: e63c2278ecf0cd157278ae48fd51cad9
SHA1: c414d932e61b6b1433c69c1c8fad3d9ed2e84f68
SHA256: 6cb1401f2078e314f6da0ae382349fb302cb395bb883ffaf1317449074847d66
Homepage: http://www.libusb.info
Description-en: userspace USB programming library
 Library for programming USB applications without the knowledge
 of Linux kernel internals.
 .
 This package contains what you need to run programs that use this
 library.
Description-md5: 2dcfdc1b1a0fdb8e8f86496cec6f9062
Task: standard
Supported: 5y

I'm still having the same problem. Is anyone else able to confirm this?

[define-private-public]

Okay, so I grabbed the latest copy of ubuntu (server) 18.04 and did a fresh install (with the latest updates too). installed both libusb (1.0.22) and libgphoto2 (2.5.18) from source, using the latest release tarballs (gotten from the GitHub release pages). The leak is still happening. Here is some better testing code:

#include <vector>
#include <string>
using namespace std;

#include <unistd.h>
#include "gphoto2/gphoto2.h"

void scan() {
    GPContext *scanContext;
    CameraList *camList;
    GPPortInfoList *portList;
    CameraAbilitiesList *abilitiesList;
    CameraAbilities abilities;
    const char *name, *value;

    // Init
    scanContext = gp_context_new();
    gp_list_new(&camList);
    gp_port_info_list_new(&portList);
    gp_abilities_list_new(&abilitiesList);

/*
    // Query
    gp_port_info_list_load(portList);           // I think the memory leak is coming from here
    gp_abilities_list_load(abilitiesList, scanContext);
    gp_abilities_list_detect(abilitiesList, portList, camList, scanContext);

    // See how many we got
    int numCams = gp_list_count(camList);
    vector<string> names;
    vector<string> ports;
    for (int i = 0; i < numCams; i++) {
        // Pull out some info
        gp_list_get_name(camList, i, &name);
        gp_list_get_value(camList, i, &value);

        // Init data
        gp_abilities_list_get_abilities(abilitiesList, i, &abilities);

        // Store it in the list If it's a camera
        if (abilities.device_type == GP_DEVICE_STILL_CAMERA) {
            names.push_back(string(name));
            ports.push_back(string(value));
        }
    }
*/

    // Cleanup GPhoto2 data
    gp_list_unref(camList);
    gp_port_info_list_free(portList);
    gp_abilities_list_free(abilitiesList);
    gp_context_unref(scanContext);
}

int main() {
    // Run forever
    while (true) {
        scan();
        usleep(1000 * 10); // 1/100th of a second
    }

    return 0;
}

If you run the above code, there will be no memory leaks and the program will keep on chugging along forever, or until you kill it. But if you uncomment the line where gp_port_info_list_load() is, then the leak will show up again.

I ran valgrind on this code (though without the infinite loop, block comment, and one pass of scan()), and it said that a memory leak was coming from gp_port_info_list_load().

[msmeissn]

one question, is a device being present while you run this loop?
i tried with libgphoto2 git, libusb 1.0.22

[define-private-public]

With my VM, I did not have a device plugged in. I can test it again with a device later, but I'm away from the VM right now.

[msmeissn]

I can confirm some leaks at least. lets see what I find

[define-private-public]

Hi. Have you been able to investigate this further? I have no problem submitting a PR to fix this, but I haven't investigated it any more deeper than the valgrind run I did.

[define-private-public]

I should paste in some valgrind output on my testing code.

==2369== Memcheck, a memory error detector
==2369== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==2369== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==2369== Command: ./a.out
==2369== 
==2369== 
==2369== HEAP SUMMARY:
==2369==     in use at exit: 11,561 bytes in 12 blocks
==2369==   total heap usage: 3,803 allocs, 3,791 frees, 1,090,366 bytes allocated
==2369== 
==2369== 4,096 bytes in 1 blocks are definitely lost in loss record 7 of 8
==2369==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2369==    by 0x6CF770B: ???
==2369==    by 0x6CFAAF0: ???
==2369==    by 0x6AE8C0C: ???
==2369==    by 0x6AE7452: ???
==2369==    by 0x6ADEC9C: ???
==2369==    by 0x68D475B: ???
==2369==    by 0x50C23FB: ??? (in /usr/lib/x86_64-linux-gnu/libgphoto2_port.so.12.0.0)
==2369==    by 0x56BF899: ??? (in /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.1)
==2369==    by 0x56BF23B: ??? (in /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.1)
==2369==    by 0x56BFE18: lt_dlforeachfile (in /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.1)
==2369==    by 0x50C2854: gp_port_info_list_load (in /usr/lib/x86_64-linux-gnu/libgphoto2_port.so.12.0.0)
==2369== 
==2369== 4,096 bytes in 1 blocks are definitely lost in loss record 8 of 8
==2369==    at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2369==    by 0x6CF770B: ???
==2369==    by 0x6CFADAB: ???
==2369==    by 0x6CFF0AA: ???
==2369==    by 0x6CFF510: ???
==2369==    by 0x6D0151B: ???
==2369==    by 0x6D018E4: ???
==2369==    by 0x6D02F0E: ???
==2369==    by 0x6AE8C2A: ???
==2369==    by 0x6AE7452: ???
==2369==    by 0x6ADEC9C: ???
==2369==    by 0x68D475B: ???
==2369== 
==2369== LEAK SUMMARY:
==2369==    definitely lost: 8,192 bytes in 2 blocks
==2369==    indirectly lost: 0 bytes in 0 blocks
==2369==      possibly lost: 0 bytes in 0 blocks
==2369==    still reachable: 3,369 bytes in 10 blocks
==2369==         suppressed: 0 bytes in 0 blocks
==2369== Reachable blocks (those to which a pointer was found) are not shown.
==2369== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==2369== 
==2369== For counts of detected and suppressed errors, rerun with: -v
==2369== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0)

My suspicion is that it's with something in this function:

libgphoto2/libgphoto2_port/libgphoto2_port/gphoto2-port-info-list.c

Line 194 in b97ab76

foreach_func (const char *filename, lt_ptr data)

[AntiSol]

Hi, @define-private-public
Did you ever find the cause or a workaround for this? I'm seeing a similar memory leak doing repeated calls to gp_camera_get_config

[define-private-public]

Hi,

I haven't looked at this in a bit (as I'm now using a somewhat hacky solution to get around this problem). I do think the cause it related somehow to how libtool might be being used. I honestly don't know. I'd really like to get this resolve though.

[trumpetlicks]

Im also interested in this issue, I have seen the revision numbers moving for both libgphoto and libusb, as well as libtool. Has this been fixed with a newer revision of any of these libraries?

[AntiSol]

in my case the issue turned out to be with the python bindings, and the maintainer of the python-gphoto2 repo helped and fixed the issue. HTH :)

[trumpetlicks]

True, but the code left above still makes me believe that there could be a c-level problem, as it is also C++. You are correct, I am using the python bindings, but if there is a lower level issue in the c interface, then I would rather it be fixed there, then have to work around.

[msmeissn]

there are likely smaller leaks in there. but currently they are hard to pinpoint :/

[trumpetlicks]

Totally understand, memory leaks are one of the most difficult to find :-) Perhaps then, another question. I am using an EOS type camera, and the python gphoto bound interface. What is the most proven method for initial bring-up of a camera?

[define-private-public]

Hi, been a while since I posted here. I'm still having this problem (with the latest release of libgphto2). I'm not the most familiar with C/C++ debugging tools. If I compiled a debug build of libgphoto2 (and accompanying libraries) would it fill in those ??? with other function calls?

[msmeissn]

I tied the C level code ... when you press ctrl-c it will show a bigger leak as the data is not freed.
I added a loop with variable iterations
with a 100 runs of the main logo and 50 and it stays at 5212 bytes of leak. these seem to be in the dynamic library loading, which is not really controllable by us.

I think we can close it, if you find other cases please report.

[define-private-public]

Wait, I'm a little confused. Is it a problem with a third party library?

[msmeissn]

I see it coming from the libtool library, it likely allocates some onetime things.

Currently I see no recurring leaks.