allocator_oom_test fails on AArch64

[rovka]

Bugzilla Link	33972
Resolution	FIXED
Resolved on	Aug 24, 2017 17:31
Version	5.0
OS	Linux
Blocks	#33196
CC	@zmodem,@kcc,@rengolin

Extended Description

While testing the 5.0.0 release, I get this failure:

04:11:01.083 FAIL: AddressSanitizer-aarch64-linux :: TestCases/Linux/allocator_oom_test.cc (2721 of 43683)
04:11:01.083 ******************** TEST 'AddressSanitizer-aarch64-linux :: TestCases/Linux/allocator_oom_test.cc' FAILED ********************
04:11:01.083 Script:
04:11:01.083 --
04:11:01.083 /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/./bin/clang --driver-mode=g++ -fsanitize=address -mno-omit-leaf-frame-pointer -fno-omit-frame-pointer -fno-optimize-sibling-calls -gline-tables-only -O0 /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc -o /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp
04:11:01.083 ulimit -v 22024290304
04:11:01.083 not /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp malloc 2>&1 | FileCheck /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc --check-prefixes=CHECK-MALLOC,CHECK-CRASH
04:11:01.083 env ASAN_OPTIONS=allocator_may_return_null=0 not /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp malloc 2>&1 | FileCheck /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc --check-prefixes=CHECK-MALLOC,CHECK-CRASH
04:11:01.083 env ASAN_OPTIONS=allocator_may_return_null=1 /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp malloc 2>&1 | FileCheck /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc --check-prefixes=CHECK-MALLOC,CHECK-NULL
04:11:01.083 env ASAN_OPTIONS=allocator_may_return_null=0 not /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp calloc 2>&1 | FileCheck /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc --check-prefixes=CHECK-CALLOC,CHECK-CRASH
04:11:01.083 env ASAN_OPTIONS=allocator_may_return_null=1 /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp calloc 2>&1 | FileCheck /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc --check-prefixes=CHECK-CALLOC,CHECK-NULL
04:11:01.083 env ASAN_OPTIONS=allocator_may_return_null=0 not /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp realloc 2>&1 | FileCheck /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc --check-prefixes=CHECK-REALLOC,CHECK-CRASH
04:11:01.083 env ASAN_OPTIONS=allocator_may_return_null=1 /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp realloc 2>&1 | FileCheck /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc --check-prefixes=CHECK-REALLOC,CHECK-NULL
04:11:01.083 env ASAN_OPTIONS=allocator_may_return_null=0 not /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp realloc-after-malloc 2>&1 | FileCheck /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc --check-prefixes=CHECK-MALLOC-REALLOC,CHECK-CRASH
04:11:01.083 env ASAN_OPTIONS=allocator_may_return_null=1 /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/Phase3/Release/llvmCore-5.0.0-rc1.obj/projects/compiler-rt/test/asan/AARCH64LinuxConfig/TestCases/Linux/Output/allocator_oom_test.cc.tmp realloc-after-malloc 2>&1 | FileCheck /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc --check-prefixes=CHECK-MALLOC-REALLOC,CHECK-NULL
04:11:01.083 --
04:11:01.083 Exit Code: 1
04:11:01.083
04:11:01.083 Command Output (stderr):
04:11:01.083 --
04:11:01.083 /home/tcwg-buildslave/workspace/tcwg-llvm-release/tcwg-apm_64-build/rc1/llvm.src/projects/compiler-rt/test/asan/TestCases/Linux/allocator_oom_test.cc:80:18: error: expected string not found in input
04:11:01.083 // CHECK-MALLOC: malloc:
04:11:01.083 ^
04:11:01.083 :1:1: note: scanning from here
04:11:01.083 ==20264==ERROR: AddressSanitizer failed to allocate 0x1bfe00000000 (30777735643136) bytes at address 41200000000 (errno: 12)
04:11:01.083 ^
04:11:01.083 :1:42: note: possible intended match here
04:11:01.083 ==20264==ERROR: AddressSanitizer failed to allocate 0x1bfe00000000 (30777735643136) bytes at address 41200000000 (errno: 12)

[zmodem]

The test is new for 5.0.0, added in r305569 by Alex.

Alex: it's already been disabled for s390 and android. Perhaps it needs disabling for ARM as well?

Diana, Renato: how come this isn't firing on some buildbot?

[llvmbot]

Before disabling, I'd like to understand the problem. Where can I repro it?

I do not see why aarch64 ASan would want to allocate that much memory for the shadow, curious to see your configuration.

[rengolin]

It turns out the buildbot did pick it up [1], but we were in a period of instability and we decided to use another machine for that builder, only that never came.

The error in the bot was identical to the one Diana picked up in the release, so there is no mistery, you can probably replicate that on any 48-bit VMA AArch64 machine out there (ex: Ubuntu Xenial onwards).

I'm not sure how long do we have until the final RC is out, but we have three options:

1. Fix it, see that the buildbot goes green and backport.
2. Disable on ARM too, because it clearly doesn't work, backport the patch.
3. Turn off the feature until it can be correctly reproduced on more than one arch.

I don't mind which one we end up doing, but whatever we do, has to be fast, ie. faster than the release, because we're moving the bot upstream back again sooner than that.

cheers,
--renato

[1] http://master2.llvm.validation.linaro.org/builders/clang-cmake-aarch64-full/builds/287/

[rengolin]

The test still fails:

http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/2391

Let's revert the patch for now and see how to best proceed after that.

[zmodem]

Trying disabling the test for arm & aarch64 here: r311674

[zmodem]

Trying disabling the test for arm & aarch64 here: r311674

And the bot seemed happy about that: http://lab.llvm.org:8011/builders/clang-cmake-aarch64-full/builds/2396

Merged to 5.0 in r311736.