Using C API with Linux: possible memory leak?

[jpka-]

Hello. Please help me with this phenomenon. I use KFR C API with my C code [1]. It works well, but, I found that there is 73,728 bytes of memory "leak" (and below, i'll mean this same amount of "leaked" bytes). Reduce this case to minimal reproduceable one, i found this one-liner with empty C code, which shows that "leak" occur not at C code itself, but at linker phase [2]:

echo 'int main() { return 0; }' | gcc -lkfr_capi -ljack -o /tmp/test -xc - && valgrind /tmp/test

As you may note, it is impossible at this stage to tell if it's something wrong with KFR C API, as the command contains other library (JACK), and "leak" disappear without it (and vice versa [2]):

echo 'int main() { return 0; }' | gcc -lkfr_capi -o /tmp/test -xc - && valgrind /tmp/test

With further research, i was able to found these commands. It shows "leak" with -lkfr_capi alone, but, not with -ljack alone:

echo 'int main() { return 0; }' | gcc -fsanitize=leak -lkfr_capi -o /tmp/test -xc - && valgrind /tmp/test

echo 'int main() { return 0; }' | gcc -fsanitize=undefined -lkfr_capi -o /tmp/test -xc - && valgrind /tmp/test

Compiling with these sanitize switches also reveals memory "leak" with KFR's provided capi_test.cpp: (Btw, it should be .c, not .cpp?)

gcc -fsanitize=undefined -lkfr_capi capi_test.cpp && valgrind ./a.out

Expected results: No errors printed. Obtained results: still reachable: 73,728 bytes valgrind says.

So i can think that something happens with kfr_capi library, and it was reason for me to report here.

The native (C++) versions of KFR examples, are all clear.

The "leak"s are quoted, because i found no way so far to distinguish possible real leak from, either, possible oversensitive valgrind, and/or linker behaviour, and/or possible misuse of tools with my test command examples (am I missed some gcc switches? Due to network jam, can't access kfrlib.com for C API usage).

I have fresh Archlinux, 6.19.9-arch1-1 #1 SMP PREEMPT_DYNAMIC, Intel x86_64, two desktops tested. I've tested libkfr_capi.so either compiled with Archlinux's PKGBUILD one, and, manually with Ninja one, with same result. I am would be glad to know if this phenomenon is reproduceable on other machines.

Thanks!

[1] https://github.com/twonoise/jasmine-sa
[2] https://github.com/twonoise/jasmine-sa?tab=readme-ov-file#known-bugs

[ZX41R]

I would treat this as “not proven to be a leak” until Valgrind shows definitely lost or indirectly lost with a useful allocation stack.

still reachable means there is still a live root to the allocation when the process exits. That is common for shared-library constructors, audio backends, sanitizer runtimes, one-time FFT tables, TLS caches, and atexit cleanup paths. It is noisy in exactly the kind of test you are running: an empty program whose only job is to load DSOs and then exit.

Two things in your commands make the signal harder to read:

gcc -fsanitize=leak ... && valgrind ./a.out
gcc -fsanitize=undefined ... && valgrind ./a.out

That mixes Valgrind with sanitizer runtimes. The sanitizer runtime itself changes process startup/shutdown behavior and can allocate persistent state. I would run these as separate experiments:

valgrind --leak-check=full --show-leak-kinds=all --num-callers=40 /tmp/test

LSAN_OPTIONS=verbosity=1:report_objects=1 ./a.out

If Valgrind still only reports still reachable, I would not chase it as a real leak. If you want to identify the owner anyway, add --trace-children=no --track-origins=yes --num-callers=40 and look at the first non-libc frame in the allocation stack. Also test -Wl,--no-as-needed -lkfr_capi vs the default link line so you know whether the library is actually being loaded in the empty program on your distro.

For a C API leak test, the stronger reproducer is a loop that repeatedly creates and destroys the same KFR object/call path in one long-lived process, then watch RSS or LSan at shutdown. A fixed 73,728 bytes after loading libkfr_capi.so is much more likely one-time reachable state than per-use leakage.

[jpka-]

Thanks so much @ZX41R for explaining.

Sadly i am not expirienced in modern debugging, but try to learn using your suggestions.

I would run these as separate experiments:
If Valgrind still only reports still reachable

Yes, it is.

If you want to identify the owner anyway ... and look at the first non-libc frame

Here is the problem, i am can't detect something similar (or not) to "the first non-libc frame" at Valgrind's output. It says "73,728 bytes in 1 blocks..." following by trace which i can get any logical meaning from. (There is no 'libc' word at it though).

Also test -Wl,--no-as-needed -lkfr_capi

I am do not know the place where i can know if "whether the library is actually being loaded" using this switch; visually there is no any difference.

The ultimate problem is that "leak" occur, afaicu, not in the code (as it is empty) but at linker/loader stage, so all various debug tools/ways i can find via web search, are not applicable. (trying to find Valgrind alternative to cross-check both of these, is failed).

a loop that repeatedly creates and destroys the same KFR object/call

Looping is not reproduceable, as i am not create any KFR objects in empty code. Should i elaborate more complex case as such? As i thought that code should be shortest possible yet revealing the "leak", which is empty body.

A fixed 73,728 bytes after loading libkfr_capi.so

Yes, it is always exactly this amount.

I will research it more today.

Thanks!

[jpka-]

So i've conducted multiple random tests and found something.

If library is created like this: (Here i assume kfr dir is local git which has been cloned and built once as per docs)

cd kfr/build/build-release/
/usr/bin/clang -fPIC -O3 -DNDEBUG   -shared -Wl,-soname,libkfr_capi.so -o lib/libkfr_capi.so src/capi/CMakeFiles/kfr_capi.dir/capi.cpp.o  lib/libkfr_dft.a  lib/libkfr_dsp.a  -lkfr_audio  -nodefaultlibs  -Wl,-Bdynamic  -lm  -lc  -lstdc++  -lgcc  -s  -lstdc++  -lpthread  -lm
cp lib/* /usr/lib64/
echo 'int main() { return 0; }' | gcc -lkfr_capi -ljack -o /tmp/test -xc - && valgrind /tmp/test

...then "problem" is "solved": valgrind is all clear, with empty code, as well as with my ultimate program.

As default one is

/usr/bin/clang -fPIC -O3 -DNDEBUG   -shared -Wl,-soname,libkfr_capi.so -o lib/libkfr_capi.so src/capi/CMakeFiles/kfr_capi.dir/capi.cpp.o  lib/libkfr_dft.a  lib/libkfr_dsp.a  -lkfr_audio  -nodefaultlibs  -Wl,-Bdynamic  -lm  -lc  -Wl,-Bstatic  -lstdc++  -lgcc  -s  -lstdc++  -lpthread  -lm

one may note that difference is with -Wl,-Bstatic key. I have not enough skills to judge if my "fix" is correct, but, looks like, can be good starting point at least for research.

Btw, one may also note other things, like duplicated -l... keys or -lkfr_audio stated but not used with C API, afaicu.

In my case it also work as simple as

/usr/bin/clang -fPIC -O3 -DNDEBUG -shared -Wl,-soname,libkfr_capi.so -o lib/libkfr_capi.so src/capi/CMakeFiles/kfr_capi.dir/capi.cpp.o lib/libkfr_dft.a lib/libkfr_dsp.a

Please do not use these "fixes" on production machines, unless you (unlike me) know what you are doing.

Thank you for great software.