-
-
Notifications
You must be signed in to change notification settings - Fork 31.7k
Assertion failed in gc with __del__ and weakref #70804
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
$ python3.5-dbg crash.py
python3.5-dbg: ../Modules/gcmodule.c:364: update_refs: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.
$ python3.5-dbg crash-multithread.py
python3.5-dbg: ../Modules/gcmodule.c:364: update_refs: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed. ============ crash.py ========== import weakref
import gc
def callback(o):
gc.collect()
class C:
def __del__(self):
pass
def main():
c = C()
cref = weakref.ref(c, callback)
del c
main() ============ crash-multithread.py ========== import threading
import weakref
import gc
ei = threading.Event()
eo = threading.Event()
def gc_worker():
ei.wait()
gc.collect()
eo.set()
def callback(o):
ei.set()
eo.wait()
class C:
def __del__(self):
pass
def main():
t = threading.Thread(target=gc_worker)
t.start()
c = C()
cref = weakref.ref(c, callback)
del c
t.join()
main() |
The crash may be raised on both Windows OS and Linux. We got crashed under Ubuntu 14.04, while a similar issue was found https://bugs.python.org/issue23187#msg238139 last year under Windows OS. We actually raise such bug working on a program with asyncio and multithread(use asyncio.BaseEventLoop.run_in_executor with its default threadpool). |
Oh, I confirm that the bug occurs on Python 3.5.1+ (3.5 branch) and Python 3.6 (default branch). Call stack: _Py_Dealloc() => subtype_dealloc() => PyObject_ClearWeakRefs() => handle_callback() of weakref => ... => gc_collect => ... => assertion error Full gdb tracebacks: python: Modules/gcmodule.c:364: update_refs: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed. Program received signal SIGABRT, Aborted. (gdb) where (gdb) py-bt
Traceback (most recent call first):
Garbage-collecting
<built-in method collect of module object at remote 0x7ffff05e2a58>
File "crash.py", line 7, in callback
gc.collect()
File "crash.py", line 17, in main
del c
File "crash.py", line 20, in <module>
main() |
Ben Bangert reported to me that this crash caused instabilities in an app using asyncio (home-assistant/core#3453). This hack made his crashes go away: What are the chances of getting the crash fixed in 3.5.3 and 3.6b2? |
I don't think that the crash is a release blocker, but I just sent an email to python-dev to ask for reviews. |
I was unable to reproduce a crash but the patch looks straightforward and obvious. |
Hi asvetlov, debug mode of python interpreter should be enabled to reproduce this bug, cause assertion is disabled in release mode. One environment that can reproduce this bug: $ cat /etc/issue
Ubuntu 14.04.3 LTS \n \l
$ uname -r
3.13.0-32-generic
$ uname -m
x86_64
$ dpkg -s python3.4-dbg | grep Version
Version: 3.4.3-1ubuntu1~14.04.4
$ python3.4-dbg crash.py
python3.4-dbg: ../Modules/gcmodule.c:364: update_refs: Assertion `((gc)->gc.gc_refs >> (1)) != 0' failed.
Aborted |
New changeset c9b7272e2553 by Benjamin Peterson in branch '3.5': New changeset 520cb70ecb90 by Benjamin Peterson in branch '3.6': New changeset c1d0df056c19 by Benjamin Peterson in branch 'default': |
Would be nice if 3.4 could also be patched, not just 3.5 and 3.6, since python in EPEL currently is python34-3.4.3-7.el7.x86_64. The patch can be applied without conflict and resolves some serious cases of segfaults. |
Sorry but Python 3.4 is not more supported upstream: You should ask for a downstream backport. |
I created the issue: https://bugzilla.redhat.com/show_bug.cgi?id=1384957 |
I'll accept this as a backport for 3.4. I'm convinced that, although the sample programs don't crash in release builds, the behavior is wrong and could be exploited. Will someone have time to create the PR in the next day or two? (It's not that it's necessary, but it seems that Victor and Serhiy are very quick, which is nice. And adding it sooner will make it more comfortable to tag and release 3.4.7rc1 next weekend, which I think I'm going to do.) |
I'll do this. |
If it's also reproducible with 3.3, I'd take a 3.3 PR for it. |
I wait merging PR 2695 for cherry-picking it to 3.3. |
Sorry for the delay in merging your PR. I believe Ned is planning to tag the next 3.3 rc1 tomorrow (Sunday). If you see this, and you have time, it'd be great if you could make a PR for the 3.3 backport. Note that 3.3 is probably *not* getting converted to use "blurb". So if you do make the backport, please just edit Misc/NEWS directly. Thanks! |
Seems this isn't reproducible with 3.3. |
I'm tracking something very similar issue to this in bug: http://bugs.python.org/issue31061 Given its similarities, anyone have any ideas? Based on the second callstack I'm starting to think this is an issue with defaultdict |
so I just discovered that the object that has the zero refcount has the same tp_dealloc: (gdb) print *FROM_GC(gc)->ob_type This is for a GatheringFuture, something tells me perhaps there is more to this function that needs to be resolved? |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: