-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Address sanitizer (gcc version) is generating false positives #88529
Comments
Running the buildbot for #26595 The tests pass with an added assert to verify that the address in question is in bounds. All failures seem to happen after a fork, which seems to be a longstanding weakness of the address sanitizer. I'd like to merge #26595. I'd also like to keep the buildbots working. Dichotomy. |
Buildbots have the priority, please don't merge anything that breaks the buildbots. We should figure out how to silence the false positive if indeed is one, but I don't think it is. The address sanitizer is not speculative, it literally checks if someone has written out of bounds |
I can reproduce that failure of address sanitizer with GCC and clang separately, so it doesn't seem like a false positive. |
Also, I can reproduce locally without any fork whatsoever: ./python -m test test_lib2to3 SUMMARY: AddressSanitizer: heap-buffer-overflow Python/ceval.c:3549 in _PyEval_EvalFrameDefault |
Even valgrind sees the problem: ❯ valgrind ./python -m test test_lib2to3 |
I'm closing this as 3 different tools complain about the same thing, with a very simple reproducer. |
If I run the following on main I get 22 failures. So something is broken. test_lib2to3 does fork; at least it does when I run it under gdb. |
As I mentioned in https://bugs.python.org/issue43693, the way to run it is the following: $ export ASAN_OPTIONS=detect_leaks=0:allocator_may_return_null=1:handle_segv=0
$ ./configure --with-address-sanitizer --without-pymalloc
$ make -j -s
$ ./python -m test test_lib2to3 There are known failures with things like ctypes and libcript so the following tests are ignored: test_ctypes test_capi test_crypt test_decimal test_faulthandler test_interpreters ----------------- test_lib2to3 does one fork for this call: https://github.com/python/cpython/blob/main/Lib/lib2to3/tests/test_parser.py#L64 but that doesn't matter. Comment that and keeps falining |
I still get quite a few failures on the main branch. There isn't anything wrong with that line, but as I plan to change it anyway I guess it doesn't matter: I plan to get the main branch passing the tests with address sanitization (at least on my machine with my gcc :) Running the tests doesn't seem to take that long (at least not compared with refleak tests). |
That's a good idea,let me see how easy doing that would be |
Even if you don't trust the sanitizer, you can also use valgrind, it shows the same error: ❯ valgrind ./python -m test test_lib2to3 For running valgrind, make sure do disable pymalloc: export PYTHONMALLOC=malloc |
This is not a false positive. I break into gdb at the moment the sanitizer makes the report and inspecting the values that apparently are wrong. I did that by breaking into __sanitizer::ColorizeReports which is called for making the report. THis is what I did: $export CFLAGS='-g3 -O0' Breakpoint 1, __sanitizer::ColorizeReports () That ep->me_key is obviously corrupted. |
What commit are you running that on? |
commit e858ea1 (HEAD -> specialize-load-attr)
diff --git a/Python/ceval.c b/Python/ceval.c
index 06a02b40f9..784d0244e8 100644
--- a/Python/ceval.c
+++ b/Python/ceval.c
@@ -3517,6 +3517,7 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
DEOPT_IF(!PyModule_CheckExact(owner), LOAD_ATTR);
PyDictObject *dict = (PyDictObject *)((PyModuleObject *)owner)->md_dict;
DEOPT_IF(dict->ma_keys->dk_version != cache1->dk_version_or_hint, LOAD_ATTR);
+ assert(cache0->index < dict->ma_keys->dk_nentries);
PyDictKeyEntry *ep = DK_ENTRIES(dict->ma_keys) + cache0->index;
res = ep->me_value;
DEOPT_IF(res == NULL, LOAD_ATTR); |
This line seems to be responsible for most of the failures: Which does appear to be a true positive. |
From the gdb session looks to me that you are reading freed or invalid memory from the dictionary keys. Well, I'm quite sure you are doing that, but I don't know why they is happening. |
It looks like I've been a bit unfair to the address sanitizer. It does appear to produce incorrect locations sometimes, but that's not really a false positive and the reports are generally useful. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: