Workarounds for #837 (Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING.) #856
Comments
|
This problem is present in Linux 4.12, and fixed in 4.13rc5. Both the break and the fix are present in 4.4.84 LTS track. The breaking change is also present in Ubuntu kernel 4.4.0-93, and likely to be fixed in the following update of the kernel package. The bug is limited to x86_64 and AArch64 platforms. You are most likely seeing this bug if a position-independent executable is loaded between addresses 0x 0001 0000 0000 and 0x 0100 0000 0000. As for the workarounds, there is not much one can do. Load a different kernel, or don't build your binary as PIE (use -no-pie linker flag). |
Attempts to work around the following issue with -fsanitize=address:
[ RUN-TEST ] ccan/check_type/test/run
==28662==Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING.
==28662==ASan shadow was supposed to be located in the [0x00007fff7000-0x10007fff7fff] range.
==28662==This might be related to ELF_ET_DYN_BASE change in Linux 4.12.
==28662==See google/sanitizers#856 for possible workarounds.
==28662==Process memory map follows:
...
Signed-off-by: Andrew Jeffery <andrew@aj.id.au>
|
I'm having this problem with the kernel |
|
Any update on this? |
|
I too am having this problem with kernel 5.4.0-52-generic |
Did you use |
|
FYI, this error can happen when you run an ASan-enabled-binary under valgrind. :) |
|
AddressSanitizer is not supposed to work under Valgrind, this is WAI. |
* The `r132/ydb704` subtest failed with the below symptom when it was run with ASAN enabled. ``` # Test invoking YottaDB via valgrind does not produce a %YDB-E-YDBDISTUNVERIF ################################################################### # First test valgrind -q yottadb -version ==83955==Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING. ==83955==ASan shadow was supposed to be located in the [0x00007fff7000-0x10007fff7fff] range. ==83955==This might be related to ELF_ET_DYN_BASE change in Linux 4.12. ==83955==See google/sanitizers#856 for possible workarounds. ``` * My guess is that both ASAN (which also includes the leak sanitizer LSAN) and valgrind use shadow memory to keep track of memory leaks and so cannot co-exist. * Therefore we disable this subtest if the current build of YottaDB was done with ASAN enabled.
…when google/sanitizers#856 is closed * The `java/callins` subtest failed with the following symptom when ASAN is enabled. ``` ################### lbl0 ################### ==31240==Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING. ==31240==ASan shadow was supposed to be located in the [0x00007fff7000-0x10007fff7fff] range. ==31240==This might be related to ELF_ET_DYN_BASE change in Linux 4.12. ==31240==See google/sanitizers#856 for possible workarounds. ==31240==Process memory map follows: 0x00060c000000-0x00062b400000 0x00062b400000-0x0007ff800000 0x0007ff800000-0x0007ffb00000 0x0007ffb00000-0x0007ffb76000 /usr/lib/jvm/java-16-openjdk-amd64/lib/server/classes.jsa . . ==31240==End of process memory map. ==31240==ERROR: AddressSanitizer failed to deallocate 0xfb07fff8000 (17250736111616) bytes at address 0x004fffffffff ==31240==AddressSanitizer CHECK failed: ../../../../src/libsanitizer/sanitizer_common/sanitizer_posix.cpp:61 "(("unable to u nmap" && 0)) != (0)" (0x0, 0x0) <empty stack> ``` * google/sanitizers#856 is still open. Not sure if there is a Java interaction with ASAN. Therefore we disable this subtest for now. Once that upstream issue is fixed, we can see if re-enabling this test is possible.
|
I have the same problem, is there a solution? Linux version 5.4.0-42-generic (buildd@lgw01-amd64-038) (gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)) #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 |
|
@surechen Are you sure you are not using |
Hi, Thanks. In fact, I use Rust, and my executable dynamically loads a dynamic link library “xxxxxx.so” when running. I linked the librustc-nightly_rt.asan.a statically for this dynamic link library “xxxxxx.so”. My executable are compiled including the RUSTFLAGS "-zsanitizer =address". This is the build.rs for my dynamic link library “xxxxxx.so” println!("cargo:rustc-link-lib=static=rustc-nightly_rt.asan");
println!("cargo:rustc-link-arg=-Wl,--whole-archive");
|
|
sudo sysctl -w kernel.randomize_va_space=0 works for me as workaround with gcc |
google/sanitizers#856 Don't use -fsanitize=address on compile step.
|
I use UB+ASAN (UBSAN and ASAN combined) builds of MariaDB server often. I recently noticed an issue where these builds would fail to start with a SIGSEGV (and And it fixed the issue! Here is the full fault seen: |
I met same issue as your mentioned, and which caused stack-over-flow. |
|
@RingsC Did the |
After use that, ASan does not issue the output message, It works. |
|
ulimit -s only affects the main thread stack. This overflow is on a
secondary thread.
…On Tue, Apr 23, 2024 at 11:59 PM RingsC ***@***.***> wrote:
@RingsC <https://github.com/RingsC> Did the sudo sysctl
vm.mmap_rnd_bits=28 help in your case also?
After use that, ASan does not issue the output message, but the stack over
flow still exists.
I Alpha V.- (GPL).
868 AddressSanitizer:DEADLYSIGNAL
869 =================================================================
870 ==158745==ERROR: AddressSanitizer: stack-overflow on address 0x15551f7dd158 (pc 0x55555c411f6c bp 0x15551f854180 sp 0x15551f7dd158 T44)
871 #0 0x55555c411f6c in ShannonBase::ML::ML_regression::train() /home/lihao/workshop/ShannonBase/ml/ml_regression.cpp:65
872 #1 0x55555c408ab1 in ShannonBase::ML::Auto_ML::train() /home/lihao/workshop/ShannonBase/ml/auto_ml.cpp:195
873 #2 0x55555a2c5254 in Item_func_ml_train::val_int() /home/lihao/workshop/ShannonBase/sql/item_func.cc:10068
874 #3 0x55555a1a4bb1 in Item::save_in_field_inner(Field*, bool) /home/lihao/workshop/ShannonBase/sql/item.cc:6895
875 #4 0x55555a1b8ee4 in Item::save_in_field(Field*, bool) /home/lihao/workshop/ShannonBase/sql/item.cc:6765
876 #5 0x55555a69db1d in sp_eval_expr(THD*, Field*, Item**) /home/lihao/workshop/ShannonBase/sql/sp.cc:2591
877 #6 0x5555597ee90c in sp_rcontext::set_variable(THD*, Field*, Item**) /home/lihao/workshop/ShannonBase/sql/sp_rcontext.cc:409
878 #7 0x5555597cd53f in sp_rcontext::set_variable(THD*, unsigned int, Item**) /home/lihao/workshop/ShannonBase/sql/sp_rcontext.h:172
879 #8 0x55555a630eb2 in Query_dumpvar::send_data(THD*, mem_root_deque<Item*> const&) /home/lihao/workshop/ShannonBase/sql/query_result.cc:722
880 #9 0x555559c43dbc in Query_expression::ExecuteIteratorQuery(THD*) /home/lihao/workshop/ShannonBase/sql/sql_union.cc:1785
881 #10 0x555559c44184 in Query_expression::execute(THD*) /home/lihao/workshop/ShannonBase/sql/sql_union.cc:1823
882 #11 0x555559ae554c in Sql_cmd_dml::execute_inner(THD*) /home/lihao/workshop/ShannonBase/sql/sql_select.cc:1023
883 #12 0x555559b018e6 in Sql_cmd_dml::execute(THD*) /home/lihao/workshop/ShannonBase/sql/sql_select.cc:794
884 #13 0x5555599f2da8 in mysql_execute_command(THD*, bool) /home/lihao/workshop/ShannonBase/sql/sql_parse.cc:4797
885 #14 0x5555597dbd15 in sp_instr_stmt::exec_core(THD*, unsigned int*) /home/lihao/workshop/ShannonBase/sql/sp_instr.cc:992
886 #15 0x5555597e0530 in sp_lex_instr::reset_lex_and_exec_core(THD*, unsigned int*, bool) /home/lihao/workshop/ShannonBase/sql/sp_instr.cc:461
887 #16 0x5555597e0d4e in sp_lex_instr::validate_lex_and_execute_core(THD*, unsigned int*, bool) /home/lihao/workshop/ShannonBase/sql/sp_instr.cc:746
888 #17 0x5555597e228b in sp_instr_stmt::execute(THD*, unsigned int*) /home/lihao/workshop/ShannonBase/sql/sp_instr.cc:918
889 #18 0x5555597c2f7f in sp_head::execute(THD*, bool) /home/lihao/workshop/ShannonBase/sql/sp_head.cc:2233
890 #19 0x5555597c62e6 in sp_head::execute_procedure(THD*, mem_root_deque<Item*>*) /home/lihao/workshop/ShannonBase/sql/sp_head.cc:2953
891 #20 0x55555a6f9364 in Sql_cmd_call::execute_inner(THD*) /home/lihao/workshop/ShannonBase/sql/sql_call.cc:236
892 #21 0x555559b018e6 in Sql_cmd_dml::execute(THD*) /home/lihao/workshop/ShannonBase/sql/sql_select.cc:794
893 #22 0x5555599f2da8 in mysql_execute_command(THD*, bool) /home/lihao/workshop/ShannonBase/sql/sql_parse.cc:4797
894 #23 0x5555599f6414 in dispatch_sql_command(THD*, Parser_state*) /home/lihao/workshop/ShannonBase/sql/sql_parse.cc:5447
895 #24 0x5555599f8f61 in dispatch_command(THD*, COM_DATA const*, enum_server_command) /home/lihao/workshop/ShannonBase/sql/sql_parse.cc:2112
896 #25 0x5555599fc724 in do_command(THD*) /home/lihao/workshop/ShannonBase/sql/sql_parse.cc:1459
897 #26 0x555559ddc1a8 in handle_connection /home/lihao/workshop/ShannonBase/sql/conn_handler/connection_handler_per_thread.cc:303
898 #27 0x55555dda2f95 in pfs_spawn_thread /home/lihao/workshop/ShannonBase/storage/perfschema/pfs.cc:3043
899 #28 0x155553a94ac2 in start_thread nptl/pthread_create.c:442
900 #29 0x155553b2684f (/lib/x86_64-linux-gnu/libc.so.6+0x12684f)
901
902 SUMMARY: AddressSanitizer: stack-overflow /home/lihao/workshop/ShannonBase/ml/ml_regression.cpp:65 in ShannonBase::ML::ML_regression::train()
903 Thread T44 created by T0 here:
904 #0 0x155554a58685 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:216
905 #1 0x55555c3694ac in my_thread_create(my_thread_handle*, pthread_attr_t const*, void* (*)(void*), void*) /home/lihao/workshop/ShannonBase/mysys/my_thread.cc:80
906 #2 0x55555dda2e20 in pfs_spawn_thread_vc(unsigned int, unsigned int, my_thread_handle*, pthread_attr_t const*, void* (*)(void*), void*) /home/lihao/workshop/ShannonBase/storage/perfschema/pfs.cc:3089
907 #3 0x555559ddb039 in inline_mysql_thread_create /home/lihao/workshop/ShannonBase/include/mysql/psi/mysql_thread.h:139
908 #4 0x555559ddc5ed in Per_thread_connection_handler::add_connection(Channel_info*) /home/lihao/workshop/ShannonBase/sql/conn_handler/connection_handler_per_thread.cc:420
909 #5 0x55555a00efe8 in Connection_handler_manager::process_new_connection(Channel_info*) /home/lihao/workshop/ShannonBase/sql/conn_handler/connection_handler_manager.cc:260
910 #6 0x5555596da20e in Connection_acceptor<Mysqld_socket_listener>::connection_event_loop() (/home/lihao/workshop/shannon-bin/bin/mysqld+0x418620e)
911 #7 0x5555596d2fa2 in mysqld_main(int, char**) /home/lihao/workshop/ShannonBase/sql/mysqld.cc:8355
912 #8 0x5555596aa635 in main /home/lihao/workshop/ShannonBase/sql/main.cc:25
913 #9 0x155553a29d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
914
915 ==158745==ABORTING
I incorporate LightGBM, which is ML tools, as static lib into my code.
And when i try to invoke the functions. It causes the coredump, the call
stack as described above. It's wiered message, stack-overflow, i have
already used ulimit -s unlimted.
—
Reply to this email directly, view it on GitHub
<#856 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AADG4SSGS3MFUIHFO2HSKHTY65J6NAVCNFSM4DZ4EPTKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBXGQZDCMZWGI3Q>
.
You are receiving this because you were assigned.Message ID:
***@***.***>
|
En,yes. I have fixed it, thanks for your reply. |
There are some know issues with address sanitizer: google/sanitizers#856 and that results to issues with the testing code: 0x050f#44 and 0x050f#46 as an example.
|
I tried change ShadowMemory Base from 0x7fff8000 to 0xbfff8000, because my binary was too large (2.2G), the error was gone, but another SEGV was reported: ||
|
after change the hardcoded Shadow Base in gcc source code, the SEGV was gone. I think this is a feasible workaround so far. |
This commit introduces the same change done at 0x050f/libft-war-machine PR#47. This fixes some issues with the address sanitizers. For more info check google/sanitizers#856
Due to a kernel change asan may not work: #837
The kernel change is now reverted upstream, but a few kernel releases are not fully compatible with asan.
This issue will list all possible workarounds as well as affected kernel versions (CC: @kees)
The text was updated successfully, but these errors were encountered: