Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigate leaks of data allocated in _Unwind_Find_FDE in ClangReplInterpreterExceptionTests. #67586

Closed
lhames opened this issue Sep 27, 2023 · 5 comments
Closed

Investigate leaks of data allocated in _Unwind_Find_FDE in ClangReplInterpreterExceptionTests. #67586

lhames opened this issue Sep 27, 2023 · 5 comments
Labels
orcjit

Comments

@lhames
Copy link
Contributor

lhames commented Sep 27, 2023

Raised in discussion on https://reviews.llvm.org/D159167.
Example failed build: https://lab.llvm.org/buildbot/#/builders/5/builds/36614

The change in behavior was caused by ca8d253, which added export_executable_symbols to ClangReplInterpreterExceptionTests, but this just caused the test to start running instead of being skipped -- it is not the root cause.

Config for build:

cmake -DLLVM_APPEND_VC_REV=OFF -GNinja -DCMAKE_BUILD_TYPE=Release -DLLVM_CCACHE_BUILD=ON -DLLVM_USE_LINKER=lld -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF -DCMAKE_C_COMPILER=/b/sanitizer-x86_64-linux-fast/build/llvm_build0/bin/clang -DCMAKE_CXX_COMPILER=/b/sanitizer-x86_64-linux-fast/build/llvm_build0/bin/clang++ '-DLLVM_ENABLE_PROJECTS='\''clang;lld;clang-tools-extra;mlir'\''' '-DLLVM_USE_SANITIZER=Address;Undefined' -DLLVM_ENABLE_LIBCXX=ON '-DCMAKE_C_FLAGS=-nostdinc++ -isystem /b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/include -isystem /b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/include/c++/v1 -fsanitize=address,undefined -Wl,--rpath=/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -L/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -w' '-DCMAKE_CXX_FLAGS=-nostdinc++ -isystem /b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/include -isystem /b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/include/c++/v1 -fsanitize=address,undefined -Wl,--rpath=/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -L/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -w' '-DCMAKE_EXE_LINKER_FLAGS=-Wl,--rpath=/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -L/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib' /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm

Log output from failed test:

FAIL: Clang-Unit :: Interpreter/ExceptionTests/./ClangReplInterpreterExceptionTests/0/1 (53743 of 77523)
******************** TEST 'Clang-Unit :: Interpreter/ExceptionTests/./ClangReplInterpreterExceptionTests/0/1' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/tools/clang/unittests/Interpreter/ExceptionTests/./ClangReplInterpreterExceptionTests-Clang-Unit-3345656-0-1.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=1 GTEST_SHARD_INDEX=0 /b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/tools/clang/unittests/Interpreter/ExceptionTests/./ClangReplInterpreterExceptionTests
--
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from InterpreterTest
[ RUN      ] InterpreterTest.CatchException
[       OK ] InterpreterTest.CatchException (183 ms)
[----------] 1 test from InterpreterTest (183 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (184 ms total)
[  PASSED  ] 1 test.
LLVMSymbolizer: error reading file: No such file or directory
=================================================================
==302471==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x5594a52ef23e in malloc /b/sanitizer-x86_64-linux-fast/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #1 0x7f841b469459 in _Unwind_Find_FDE (/lib/x86_64-linux-gnu/libgcc_s.so.1+0x1d459) (BuildId: 81899e6e627233f39aeb6104df0ab2e435622084)
    #2 0x7fff4b3fa2ef  ([stack]+0x1e2ef)
    #3 0x7f841b43bfff  (<unknown module>)
SUMMARY: AddressSanitizer: 48 byte(s) leaked in 1 allocation(s).
--
exit: 1
--
@lhames lhames added orcjit and removed new issue labels Sep 27, 2023
@llvmbot
Copy link
Collaborator

llvmbot commented Sep 27, 2023

@llvm/issue-subscribers-orcjit

Raised in discussion on https://reviews.llvm.org/D159167. Example failed build: https://lab.llvm.org/buildbot/#/builders/5/builds/36614

The change in behavior was caused by ca8d253, which added export_executable_symbols to ClangReplInterpreterExceptionTests, but this just caused the test to start running instead of being skipped -- it is not the root cause.

Config for build:

cmake -DLLVM_APPEND_VC_REV=OFF -GNinja -DCMAKE_BUILD_TYPE=Release -DLLVM_CCACHE_BUILD=ON -DLLVM_USE_LINKER=lld -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF -DCMAKE_C_COMPILER=/b/sanitizer-x86_64-linux-fast/build/llvm_build0/bin/clang -DCMAKE_CXX_COMPILER=/b/sanitizer-x86_64-linux-fast/build/llvm_build0/bin/clang++ '-DLLVM_ENABLE_PROJECTS='\''clang;lld;clang-tools-extra;mlir'\''' '-DLLVM_USE_SANITIZER=Address;Undefined' -DLLVM_ENABLE_LIBCXX=ON '-DCMAKE_C_FLAGS=-nostdinc++ -isystem /b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/include -isystem /b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/include/c++/v1 -fsanitize=address,undefined -Wl,--rpath=/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -L/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -w' '-DCMAKE_CXX_FLAGS=-nostdinc++ -isystem /b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/include -isystem /b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/include/c++/v1 -fsanitize=address,undefined -Wl,--rpath=/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -L/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -w' '-DCMAKE_EXE_LINKER_FLAGS=-Wl,--rpath=/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib -L/b/sanitizer-x86_64-linux-fast/build/libcxx_build_asan_ubsan/lib' /b/sanitizer-x86_64-linux-fast/build/llvm-project/llvm

Log output from failed test:

FAIL: Clang-Unit :: Interpreter/ExceptionTests/./ClangReplInterpreterExceptionTests/0/1 (53743 of 77523)
******************** TEST 'Clang-Unit :: Interpreter/ExceptionTests/./ClangReplInterpreterExceptionTests/0/1' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:/b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/tools/clang/unittests/Interpreter/ExceptionTests/./ClangReplInterpreterExceptionTests-Clang-Unit-3345656-0-1.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=1 GTEST_SHARD_INDEX=0 /b/sanitizer-x86_64-linux-fast/build/llvm_build_asan_ubsan/tools/clang/unittests/Interpreter/ExceptionTests/./ClangReplInterpreterExceptionTests
--
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from InterpreterTest
[ RUN      ] InterpreterTest.CatchException
[       OK ] InterpreterTest.CatchException (183 ms)
[----------] 1 test from InterpreterTest (183 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (184 ms total)
[  PASSED  ] 1 test.
LLVMSymbolizer: error reading file: No such file or directory
=================================================================
==302471==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 48 byte(s) in 1 object(s) allocated from:
    #<!-- -->0 0x5594a52ef23e in malloc /b/sanitizer-x86_64-linux-fast/build/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
    #<!-- -->1 0x7f841b469459 in _Unwind_Find_FDE (/lib/x86_64-linux-gnu/libgcc_s.so.1+0x1d459) (BuildId: 81899e6e627233f39aeb6104df0ab2e435622084)
    #<!-- -->2 0x7fff4b3fa2ef  ([stack]+0x1e2ef)
    #<!-- -->3 0x7f841b43bfff  (&lt;unknown module&gt;)
SUMMARY: AddressSanitizer: 48 byte(s) leaked in 1 allocation(s).
--
exit: 1
--
</details>

@lhames
Copy link
Contributor Author

lhames commented Sep 27, 2023

I wonder whether this is a leak in libgcc_s, or whether the JIT is just not deallocating the registered frames when it shuts down?

@vitalybuka If the leak is showing up in _Unwind_Find_FDE, does that imply that libgcc_s was compiled with ASan? Or is LeakSanitizer able to detect leaks even if the memory was allocated by a library compiled without ASan?

@vitalybuka
Copy link
Collaborator

Lsan is link time feature, it just replaces malloc, and the scan stack, heap, globals, registers for references to blocks which are not free. So it will detected _Unwind_Find_FDE leaks as well.

lhames added a commit that referenced this issue Sep 27, 2023
@lhames
[clang-repl] Disable LSan in InterpreterExceptionTest.
47625fe 
Disabling LSan for this test until we can track down the leak described in
#67586.
@zero9178
Copy link
Member

This is a bug in GCC 13.0 that was fixed in GCC 13.2
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=109685

@lhames
Copy link
Contributor Author

lhames commented Sep 27, 2023

@zero9178 That saves us a lot of debugging -- thank you very much!

I'll update the FIXME to say that we should re-enable Lsan once we can assume GCC 13.2 or higher.

@lhames lhames closed this as completed Sep 27, 2023
lhames added a commit that referenced this issue Sep 27, 2023
@lhames
[clang-repl] Update FIXME based on feedback from @zero9178.
31e01e9 
See #67586.
lhames added a commit that referenced this issue Sep 27, 2023
@lhames
Re-apply "[ORC][LLJIT] Move enable-debugger-support utility out of..."
61b0f12 
This re-applies e1a5bb5, which was reverted in e5f169f due to LSan
failures on some bots (see #67586).
The LSan failures were not caused by this patch (just exposed by it), so LSan
was disabled for the failing test in 47625fe. This should be safe to
re-land now.
legrosbuffle pushed a commit to legrosbuffle/llvm-project that referenced this issue Sep 29, 2023
@lhames @legrosbuffle
[clang-repl] Disable LSan in InterpreterExceptionTest.
98ea6c8 
Disabling LSan for this test until we can track down the leak described in
llvm#67586.
legrosbuffle pushed a commit to legrosbuffle/llvm-project that referenced this issue Sep 29, 2023
@lhames @legrosbuffle
[clang-repl] Update FIXME based on feedback from @zero9178.
942753e 
See llvm#67586.
legrosbuffle pushed a commit to legrosbuffle/llvm-project that referenced this issue Sep 29, 2023
@lhames @legrosbuffle
Re-apply "[ORC][LLJIT] Move enable-debugger-support utility out of..."
539db22 
This re-applies e1a5bb5, which was reverted in e5f169f due to LSan
failures on some bots (see llvm#67586).
The LSan failures were not caused by this patch (just exposed by it), so LSan
was disabled for the failing test in 47625fe. This should be safe to
re-land now.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
orcjit
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@lhames @vitalybuka @zero9178 @llvmbot