address sanitizer does not catch simple memory leak on aarch64/RHEL8

[tbaederr]

This simple test case from the llvm-toolchain-integration-test-suite does not detect the memory leak when run on RHEL 8.

clang is 14.0.6, glibc is 2.28-209.

The bug only reproduces on RHEL 8 and only on aarch64. RHEL 9 and Fedora are unaffected on all arches.

[llvmbot]

@llvm/issue-subscribers-backend-aarch64

[DavidSpickett]

(doh, confused asan and leaksan)

We (Linaro) will double check the status of the test on our bots and get back to you.

[DavidSpickett]

I completely missed that this isn't part of a test suite in llvm but an external repo. So no we don't run this on our bots but we do run the tests within llvm-project and see no issues there.

I wasn't able to get a RHEL8 container going but I did get Centos 8 (they are related, right?), which also fails to detect the leak.

This first bit is the Ubuntu container we run our bots on. The test works there.

$ /tmp/test.o

=================================================================
==1885363==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 7 byte(s) in 1 object(s) allocated from:
    #0 0xaaaae2ef20e8  (/tmp/test.o+0xcf0e8)
    #1 0xaaaae2f2be14  (/tmp/test.o+0x108e14)
    #2 0xffff854c2e0c  (/lib/aarch64-linux-gnu/libc.so.6+0x20e0c) (BuildId: b7b30ba2c0f618615a8d32d86813718bb2fdb567)
    #3 0xaaaae2e49254  (/tmp/test.o+0x26254)

SUMMARY: AddressSanitizer: 7 byte(s) leaked in 1 allocation(s).
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.4 LTS
Release:        20.04
Codename:       focal
$ ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.9) 2.31

Then the Centos 8 container.

[root@180a406fd42d build-llvm]# /tmp/test.o
<no output>
[root@180a406fd42d build-llvm]# ldd --version
ldd (GNU libc) 2.28
[root@180a406fd42d build-llvm]# cat /etc/centos-release
CentOS Linux release 8.5.2111

We also have Bionic which is glibc 2.27, I'll see what that does.

[DavidSpickett]

Focal works, it has glibc 2.27.

# /tmp/test.o

=================================================================
==25933==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 7 byte(s) in 1 object(s) allocated from:
    #0 0xaaaae4141a20  (/tmp/test.o+0x101a20)
    #1 0xaaaae418d144  (/tmp/test.o+0x14d144)
    #2 0xffff9cb9679c  (/lib/aarch64-linux-gnu/libc.so.6+0x2079c) (BuildId: 32bfa5c5cab041643e8a536549c837fa5938b975)
    #3 0xaaaae4065bf0  (/tmp/test.o+0x25bf0)

SUMMARY: AddressSanitizer: 7 byte(s) leaked in 1 allocation(s).
root@3d5dbd604d22:~/build-llvm# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.6 LTS
Release:        18.04
Codename:       bionic
# ldd --version
ldd (Ubuntu GLIBC 2.27-3ubuntu1.5) 2.27

Though I'm just guessing that the glibc version is important. It could be unrelated.

[panukulv]

Clang

[zatrazz]

I think this is an issue with RHEL8 clang itself, rather than glibc. I checked on a CENTOS8, but I assume kernel and runtime will be similar to RHEL8:

$ uname -a
Linux gcc185.osuosl.org 4.18.0-348.2.1.el8_5.aarch64 #1 SMP Tue Nov 16 14:41:36 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
$ ./bin/clang --version
clang version 16.0.0 (https://github.com/llvm/llvm-project.git ce4c7a987fa3f255fa49570da4be1b9739815369)
Target: aarch64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/azanella/llvm/llvm-build-stage1-lld/./bin
$ ./bin/clang -o bz -fsanitize=address -g bz.c
$ env ASAN_OPTIONS="log_path=stdout:exitcode=0" ./bz

=================================================================
==2252559==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 7 byte(s) in 1 object(s) allocated from:
    #0 0xaaaaad81263c  (/home/azanella/llvm/llvm-build-stage1-lld/bz+0xc263c)
    #1 0xaaaaad84b144  (/home/azanella/llvm/llvm-build-stage1-lld/bz+0xfb144)
    #2 0xffff9abc4380  (/lib64/libc.so.6+0x24380) (BuildId: ceec3eae82b0ac24e2835b515681c50133800fd9)
    #3 0xaaaaad7735bc  (/home/azanella/llvm/llvm-build-stage1-lld/bz+0x235bc)

SUMMARY: AddressSanitizer: 7 byte(s) leaked in 1 allocation(s).
$ clang --version
clang version 14.0.0 (Red Hat 14.0.0-1.module_el8.7.0+1142+5343df54)
Target: aarch64-redhat-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
$ clang -o bz -fsanitize=address -g bz.c
$ env ASAN_OPTIONS="log_path=stdout:exitcode=0" ./bz
$

[tbaederr]

Wait, so you both tried Centos8 and one of you could reproduce it but the other one couldn't?

[zatrazz]

Wait, so you both tried Centos8 and one of you could reproduce it but the other one couldn't?

It was not clear to me which clang was used by you or by @DavidSpickett. Debugging a little more, it seems to be the ASAN runtime and not the instrumentation, since linking an object built with CENTOS8 provided clang against the git as an runtime does work.

[DavidSpickett]

For my Centos 8 checks I used clang 12 as the host compiler then built the test using the clang built from main.

[root@180a406fd42d build-llvm]# clang --version
clang version 12.0.1 (Red Hat 12.0.1-4.module_el8.5.0+1025+93159d6c)

I'm not sure whether the compiler-rt build uses the host compiler or the just built clang though.

[zatrazz]

For my Centos 8 checks I used clang 12 as the host compiler then built the test using the clang built from main.

[root@180a406fd42d build-llvm]# clang --version
clang version 12.0.1 (Red Hat 12.0.1-4.module_el8.5.0+1025+93159d6c)

I'm not sure whether the compiler-rt build uses the host compiler or the just built clang though.

The asan runtime is usually linked statically by clang (libclang_rt.asan_static.a) and if you linked the tests using the build clang it will use the one built. I am not sure if you actually install if clang would use the system one, I will need to check it.