`test_generate_gir_with_address_sanitizer` failure #11754

benoit-pierre · 2023-05-01T15:55:39Z

Describe the bug

The test_generate_gir_with_address_sanitizer test fails on my machine.

Testsuite output

▸ ./run_unittests.py -v -ktest_generate_gir_with_address_sanitizer
[…]
[77/126] Generating gir/dep1/dep2/MesonDep2-1.0.gir with a custom command (wrapped by meson to set env)
FAILED: gir/dep1/dep2/MesonDep2-1.0.gir
env PKG_CONFIG_PATH=[…]/meson/tmpdbxb23ra/meson-uninstalled 'CC=sccache cc' /usr/bin/g-ir-scanner --quiet --no-libtool --namespace=MesonDep2 --nsversion=1.0 --warn-all --output gir/dep1/dep2/MesonDep2-1.0.gir '-I[…]/meson/test cases/frameworks/7 gnome/gir/dep1/dep2' -I[…]/meson/tmpdbxb23ra/gir/dep1/dep2 --filelist=[…]/meson/tmpdbxb23ra/gir/dep1/dep2/libdep2lib.so.p/MesonDep2_1.0_gir_filelist --include=GObject-2.0 --symbol-prefix=meson --identifier-prefix=Meson --cflags-begin -DMESON_TEST_1 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/sysprof-4 -I/usr/include/gobject-introspection-1.0 --cflags-end --add-include-path=/usr/share/gir-1.0 -lasan -L[…]/meson/tmpdbxb23ra/gir/dep1/dep2 --library dep2lib --extra-library=gobject-2.0 --extra-library=glib-2.0 --extra-library=girepository-1.0 --sources-top-dirs '[…]/meson/test cases/frameworks/7 gnome/' --sources-top-dirs […]/meson/tmpdbxb23ra/
/usr/lib/gobject-introspection/giscanner/scannermain.py:343: EncodingWarning: 'encoding' argument not specified
  with open(options.filelist, "r", encoding=None) as filelist_file:

=================================================================
==76350==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 280 byte(s) in 2 object(s) allocated from:
    #0 0x7f4fdcabf411 in __interceptor_calloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f4fdc8b3a31 in g_malloc0 (/usr/lib/libglib-2.0.so.0+0x5fa31)

Indirect leak of 4164 byte(s) in 2 object(s) allocated from:
    #0 0x7f4fdcabfa89 in __interceptor_malloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f4fdc8b2e19 in g_malloc (/usr/lib/libglib-2.0.so.0+0x5ee19)

Indirect leak of 96 byte(s) in 1 object(s) allocated from:
    #0 0x7f4fdcabf411 in __interceptor_calloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7f4fdc8b3a31 in g_malloc0 (/usr/lib/libglib-2.0.so.0+0x5fa31)

Indirect leak of 28 byte(s) in 1 object(s) allocated from:
    #0 0x7f4fdcabfa89 in __interceptor_malloc /usr/src/debug/gcc/gcc/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7f4fdc512330  (/usr/lib/libc.so.6+0x7a330)
    #2 0xcbedc2ef920422ff  (<unknown module>)

SUMMARY: AddressSanitizer: 4568 byte(s) leaked in 6 allocation(s).
Command '['[…]/meson/tmpdbxb23ra/tmp-introspectrfshifd7/MesonDep2-1.0', '--introspect-dump=[…]/meson/tmpdbxb23ra/tmp-introspectrfshifd7/functions.txt,[…]/meson/tmpdbxb23ra/tmp-introspectrfshifd7/dump.xml']' returned non-zero exit status 1.
[78/126] Compiling C object mkenums/enumprog4.p/meson-generated_.._enums4.c.o
[…]

System parameters

operating system: Arch Linux 64 bit
what Python version are you using e.g. 3.8.0
meson --version: 1.1.0-57-g7388737c7
ninja --version: 1.11.1

Workaround

Run the testsuite with: env 'LSAN_OPTIONS=exitcode=0' ./run_unittests.py -v -ktest_generate_gir_with_address_sanitizer.

Tentative patch

 mesonbuild/modules/gnome.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git i/mesonbuild/modules/gnome.py w/mesonbuild/modules/gnome.py
index 7d471e4ce..e8acb03a2 100644
--- i/mesonbuild/modules/gnome.py
+++ w/mesonbuild/modules/gnome.py
@@ -982,6 +982,8 @@ class GnomeModule(ExtensionModule):
         # g-ir-scanner uses Python's distutils to find the compiler, which uses 'CC'
         cc_exelist = state.environment.coredata.compilers.host['c'].get_exelist()
         run_env.set('CC', [quote_arg(x) for x in cc_exelist], ' ')
+        # Don't fail if the build is using `-Db_sanitize=address`.
+        run_env.set('LSAN_OPTIONS', 'exitcode=0', '')
         run_env.merge(kwargs['env'])
 
         return GirTarget(

The text was updated successfully, but these errors were encountered:

eli-schwartz · 2023-05-10T03:45:48Z

This also prevents CI from rebuilding our images. Seems likely to be an underlying glib problem...

@xclaesse any idea?

In this case, PEP 668 was created to allow a thing that Debian wanted, which is for `pip install foobar` to not break the system python. This despite the fact that the system python is fine, unless you use sudo pip which is discouraged for separate reasons, and it is in fact quite natural to install additional packages to the user site-packages. It isn't even the job of the operating system to decide whether the user site-packages is broken, whether the operating system gets the answer correct or not -- it is the job of the operating system to decide whether the operating system is broken, and that can be solved by e.g. enforcing a shebang policy for distribution-packaged software, which distros like Fedora do, and mandating not only that python shebangs do not contain `/usr/bin/env`, but that they *do* contain -s. Anyway, this entire kerfuffle is mostly just a bit of pointless interactive churn, but it bites pretty hard for our use case, which is a container image which is fortunately tested before deployment, so instead of failing to deploy because of theoretical conflicts with the base system (we specifically need base system integration...) we fail to deploy because 5 minutes into pulling apt updates at the very beginning, pip refuses point-blank to work. I especially do not know why it is the job of the operating system to throw errors intended for interactive users at people baking "appliance" containers who cannot "break" the system python anyway. Fix this by doing what Debian and Ubuntu should both have done from the beginning, and opting containers out of this questionable feature entirely. Note that CI images may still not actually complete their build/test cycle and be updated, because e.g. LLVM 16 issues tracked by mesonbuild#11642 or glib ASAN issues tracked by mesonbuild#11754.

eli-schwartz · 2023-05-28T23:56:22Z

@ebassi I bisected glib for this, and GNOME/glib@3b7af4d is the commit that introduced this asan failure.

benoit-pierre · 2023-05-29T00:05:08Z

So who knows how old the leak is...

benoit-pierre · 2023-05-29T00:06:33Z

I mean, did you bisect with G_SLICE=always-malloc?

eli-schwartz · 2023-05-29T00:28:51Z

Okay yes that demonstrates the issue as far back as 2.70 at least (I didn't spot check further, just verified that that bisected commit only uncovered a pre-existing issue).

ebassi · 2023-05-29T09:52:01Z

It would probably be easier if ASan printed out a backtrace, to see which object is getting allocated without getting freed.

@ebassi I bisected glib for this, and GNOME/glib@3b7af4d is the commit that introduced this asan failure.

That commit appears because the object allocation was switched from the slice allocator to the system one, so anything that was masked by GSlice being opaque to ASan is now visible.

xclaesse · 2023-07-18T14:27:54Z

I'm tried to reproduce this issue, but env 'LSAN_OPTIONS=exitcode=0' ./run_unittests.py -v -ktest_generate_gir_with_address_sanitizer works just fine on my Ubuntu. I assume I have to build glib with -Db_sanitize=leak, but then I run meson unit test inside glib's devenv and that test gets skipped... @eli-schwartz how are you reproducing?

xclaesse · 2023-07-18T14:29:44Z

but then I run meson unit test inside glib's devenv and that test gets skipped

Found that it gets skipped because:

>>> import gi
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/gi/__init__.py", line 40, in <module>
    from . import _gi
ImportError: /lib/x86_64-linux-gnu/liblsan.so.0: cannot allocate memory in static TLS block

I don't know what that means...

xclaesse · 2023-07-18T14:34:56Z

Oh actually I can reproduce with Ubuntu glib version, it's just the env 'LSAN_OPTIONS=exitcode=0' in the bug report that makes no sense, that hide the issue...

xclaesse · 2023-07-18T14:38:35Z

I got better backtraces with ASAN_OPTIONS=fast_unwind_on_malloc=0:

Direct leak of 152 byte(s) in 1 object(s) allocated from:
    #0 0x7fd3960de667 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7fd395eba4e0 in g_malloc0 ../glib/gmem.c:163
    #2 0x7fd395fd77dc in g_type_create_instance ../gobject/gtype.c:1979
    #3 0x7fd395fbad0f in g_object_new_internal ../gobject/gobject.c:2245
    #4 0x7fd395fbcaa2 in g_object_new_valist ../gobject/gobject.c:2584
    #5 0x7fd395fbd0fc in g_object_new ../gobject/gobject.c:2057
    #6 0x55a367a7b21a in dump_irepository (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x421a) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #7 0x55a367a7b5d1 in main (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x45d1) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #8 0x7fd395a23a8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #9 0x7fd395a23b48 in __libc_start_main_impl ../csu/libc-start.c:360
    #10 0x55a367a79864 in _start (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x2864) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)

Direct leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x7fd3960de667 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77
    #1 0x7fd395eba4e0 in g_malloc0 ../glib/gmem.c:163
    #2 0x7fd395fd77dc in g_type_create_instance ../gobject/gtype.c:1979
    #3 0x7fd395fbad0f in g_object_new_internal ../gobject/gobject.c:2245
    #4 0x7fd395fbc377 in g_object_new_with_properties ../gobject/gobject.c:2408
    #5 0x7fd395fbd120 in g_object_new ../gobject/gobject.c:2054
    #6 0x7fd395d8cf51 in _g_local_file_output_stream_replace ../gio/glocalfileoutputstream.c:1318
    #7 0x7fd395d81751 in g_local_file_replace ../gio/glocalfile.c:1425
    #8 0x55a367a7b146 in dump_irepository (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x4146) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #9 0x55a367a7b5d1 in main (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x45d1) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #10 0x7fd395a23a8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #11 0x7fd395a23b48 in __libc_start_main_impl ../csu/libc-start.c:360
    #12 0x55a367a79864 in _start (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x2864) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)

Indirect leak of 4096 byte(s) in 1 object(s) allocated from:
    #0 0x7fd3960decaf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7fd395eba488 in g_malloc ../glib/gmem.c:130
    #2 0x7fd395c98203 in g_buffered_input_stream_set_buffer_size ../gio/gbufferedinputstream.c:238
    #3 0x7fd395fba43c in object_set_property ../gobject/gobject.c:1811
    #4 0x7fd395fbadc7 in g_object_new_internal ../gobject/gobject.c:2290
    #5 0x7fd395fbcaa2 in g_object_new_valist ../gobject/gobject.c:2584
    #6 0x7fd395fbd0fc in g_object_new ../gobject/gobject.c:2057
    #7 0x55a367a7b21a in dump_irepository (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x421a) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #8 0x55a367a7b5d1 in main (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x45d1) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #9 0x7fd395a23a8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #10 0x7fd395a23b48 in __libc_start_main_impl ../csu/libc-start.c:360
    #11 0x55a367a79864 in _start (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x2864) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)

Indirect leak of 79 byte(s) in 1 object(s) allocated from:
    #0 0x7fd3960decaf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7fd395eba488 in g_malloc ../glib/gmem.c:130
    #2 0x7fd395ed5592 in g_strdup ../glib/gstrfuncs.c:363
    #3 0x7fd395d8cf88 in g_strdup_inline ../glib/gstrfuncs.h:321
    #4 0x7fd395d8cf88 in _g_local_file_output_stream_replace ../gio/glocalfileoutputstream.c:1324
    #5 0x7fd395d81751 in g_local_file_replace ../gio/glocalfile.c:1425
    #6 0x55a367a7b146 in dump_irepository (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x4146) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #7 0x55a367a7b5d1 in main (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x45d1) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #8 0x7fd395a23a8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #9 0x7fd395a23b48 in __libc_start_main_impl ../csu/libc-start.c:360
    #10 0x55a367a79864 in _start (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x2864) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)

Indirect leak of 28 byte(s) in 1 object(s) allocated from:
    #0 0x7fd3960decaf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x7fd395a82677 in __vasprintf_internal libio/vasprintf.c:116
    #2 0x7fd395f094a1 in vasprintf /usr/include/x86_64-linux-gnu/bits/stdio2.h:169
    #3 0x7fd395f094a1 in g_vasprintf ../glib/gprintf.c:340
    #4 0x7fd395ed5780 in g_strdup_vprintf ../glib/gstrfuncs.c:553
    #5 0x7fd395ed583c in g_strdup_printf ../glib/gstrfuncs.c:583
    #6 0x7fd395d8cb63 in _g_local_file_output_stream_really_close ../gio/glocalfileoutputstream.c:441
    #7 0x7fd395cdaa52 in g_output_stream_internal_close ../gio/goutputstream.c:875
    #8 0x7fd395cdd63d in g_output_stream_close ../gio/goutputstream.c:940
    #9 0x55a367a7b4f3 in dump_irepository (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x44f3) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #10 0x55a367a7b5d1 in main (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x45d1) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)
    #11 0x7fd395a23a8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #12 0x7fd395a23b48 in __libc_start_main_impl ../csu/libc-start.c:360
    #13 0x55a367a79864 in _start (/home/xclaesse/programmation/meson/tmpiy7u11bg/tmp-introspectg634vene/MesonDep3-1.0+0x2864) (BuildId: e4e5cb28033150d89915843755851e23d917b0c0)

xclaesse · 2023-07-18T15:10:34Z

Here is a fix of the leak in gobject-introspection: https://gitlab.gnome.org/GNOME/gobject-introspection/-/merge_requests/411.

Until that fix reaches our CI distro, we probably need to use the workaround proposed: LSAN_OPTIONS=exitcode=0.

Fixes: mesonbuild#11754

This makes Meson unit test fail: mesonbuild/meson#11754

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: mesonbuild#11754

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: #11754

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: #11754 (cherry picked from commit 9d88d0d)

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: mesonbuild#11754

3v1n0 · 2023-08-10T08:58:52Z

Thanks for the upstream fix (I was about to be annoyed enough to do it too :-D), however another trick to get the tests working is compiling with ASAN_OPTIONS=start_deactivated=true,detect_leaks=0 and then using meson test without that.

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: mesonbuild#11754

This makes Meson unit test fail: mesonbuild/meson#11754

eli-schwartz added the module:gnome label May 10, 2023

eli-schwartz added meson tests and removed module:gnome labels May 10, 2023

eli-schwartz mentioned this issue May 28, 2023

ciimage: fix pathological brokenness in Debian packaging of python #11825

Merged

xclaesse added a commit to xclaesse/meson that referenced this issue Jul 18, 2023

ci: Fix failing test_generate_gir_with_address_sanitizer test

ace7796

Fixes: mesonbuild#11754

xclaesse mentioned this issue Jul 18, 2023

ci: Fix failing test_generate_gir_with_address_sanitizer test #12004

Merged

xclaesse added a commit to xclaesse/meson that referenced this issue Jul 18, 2023

ci: Fix failing test_generate_gir_with_address_sanitizer test

62067ba

Fixes: mesonbuild#11754

gnomesysadmins pushed a commit to GNOME/gobject-introspection that referenced this issue Jul 18, 2023

gdump: Fix leaked io streams

bf96a92

This makes Meson unit test fail: mesonbuild/meson#11754

xclaesse added a commit to xclaesse/meson that referenced this issue Jul 18, 2023

ci: Skip test_generate_gir_with_address_sanitizer

eced1a5

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: mesonbuild#11754

eli-schwartz closed this as completed in #12004 Jul 18, 2023

eli-schwartz pushed a commit that referenced this issue Jul 18, 2023

ci: Skip test_generate_gir_with_address_sanitizer

9d88d0d

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: #11754

eli-schwartz pushed a commit that referenced this issue Aug 6, 2023

ci: Skip test_generate_gir_with_address_sanitizer

556702a

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: #11754 (cherry picked from commit 9d88d0d)

eli-schwartz pushed a commit that referenced this issue Aug 6, 2023

ci: Skip test_generate_gir_with_address_sanitizer

4540474

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: #11754 (cherry picked from commit 9d88d0d)

Dudemanguy pushed a commit to Dudemanguy/meson that referenced this issue Aug 7, 2023

ci: Skip test_generate_gir_with_address_sanitizer

0d77230

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: mesonbuild#11754

bruchar1 pushed a commit to bruchar1/meson that referenced this issue Aug 25, 2023

ci: Skip test_generate_gir_with_address_sanitizer

01d4f92

It fails with glib >= 2.76 and gobject-introspection <= 1.76.1. Fixes: mesonbuild#11754

gnomesysadmins pushed a commit to GNOME/glib that referenced this issue Oct 16, 2023

gdump: Fix leaked io streams

a194c42

This makes Meson unit test fail: mesonbuild/meson#11754

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`test_generate_gir_with_address_sanitizer` failure #11754

`test_generate_gir_with_address_sanitizer` failure #11754

benoit-pierre commented May 1, 2023

eli-schwartz commented May 10, 2023

eli-schwartz commented May 28, 2023

benoit-pierre commented May 29, 2023

benoit-pierre commented May 29, 2023

eli-schwartz commented May 29, 2023

ebassi commented May 29, 2023 •

edited

Loading

xclaesse commented Jul 18, 2023

xclaesse commented Jul 18, 2023

xclaesse commented Jul 18, 2023

xclaesse commented Jul 18, 2023

xclaesse commented Jul 18, 2023

3v1n0 commented Aug 10, 2023

test_generate_gir_with_address_sanitizer failure #11754

test_generate_gir_with_address_sanitizer failure #11754

Comments

benoit-pierre commented May 1, 2023

eli-schwartz commented May 10, 2023

eli-schwartz commented May 28, 2023

benoit-pierre commented May 29, 2023

benoit-pierre commented May 29, 2023

eli-schwartz commented May 29, 2023

ebassi commented May 29, 2023 • edited Loading

xclaesse commented Jul 18, 2023

xclaesse commented Jul 18, 2023

xclaesse commented Jul 18, 2023

xclaesse commented Jul 18, 2023

xclaesse commented Jul 18, 2023

3v1n0 commented Aug 10, 2023

`test_generate_gir_with_address_sanitizer` failure #11754

`test_generate_gir_with_address_sanitizer` failure #11754

ebassi commented May 29, 2023 •

edited

Loading