-
-
Notifications
You must be signed in to change notification settings - Fork 453
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix cython's deep C-stacks upon deallocation #13901
Comments
Attachment: test.sage.gz script to cause a C stack overflow. |
comment:1
Example output:
|
comment:3
About what you pointed out in #13896 and I've reproduced here where it now belongs, It seems to me the trashcan macros where designed to avoid the situation you produced here, that is objects pointing to objects pointing to... which could eventually blow up the stack. But then it also includes the fact that a given type can extend another one and so they had to use that trick to fool the trashcan mecanism. The problem to deal with here is that each layer of Cython type will have its own dealloc and without any knowledge on the depth of the type tower we cannot so easily trick the trashcan macros indeed... |
comment:4
This is now http://trac.cython.org/cython_trac/ticket/797 |
comment:5
Replying to @jpflori:
The problem there is that if the type gets trashcanned halfway up the tower, the pointer would be added to the trashcan we'd be returning all the way up to the leaf class dealloc instance. That would decref and free the memory block. Once the trashcan gets emptied, the pointer (now masquerading as an instance of one of the supers) gets processed again and again decreffed and freed. That's the double-free we're trying to avoid. For cython, the appropriate solution is probably something along the lines of
i.e., |
comment:6
Replying to @nbruin:
I think I meant the other way around, that is emulating for Cython classes what subtype_dealloc does to skip the subtype themselves using subtype_dealloc. Of course implementing that will be more complicated than in the subtype_dealloc, because we have to know we are at the top and the kind of types above us. Not sure in fact now that the esaiest way is not to first check the depth of the type tower and use exactly the same trick as CPython but with that depth instead of 1 when decr/incrementing artificially the trashcan depth counter... |
This comment has been minimized.
This comment has been minimized.
Changed upstream from Reported upstream. No feedback yet. to Reported upstream. Developers acknowledge bug. |
According to cython/cython#2842 there is now a cython directive that makes the trashcan available in cython. Some testing to see that it meets the requirements in sage required |
That cython PR is for their elusive 3.0 release. We still carry a backport patch for our cython 0.29.x: |
Oh, OK. I saw a "backport" mention at the bottom of the closed ticket, but now I see it's on an unrelated project. One has to be careful with reading github mentions carefully, apparently. |
Once you know about python's
trashcan
that is used in its dealloc methods and that cython does not make use of it, it's not hard to cause a crash in deallocation of a cython class:A long linked list of these will quickly exhaust the C-stack (set a ulimit on the stack if you have a lot of memory):
Once you interleave with a python container type (tuple, for instance), the trashcan starts to kick in. The following runs without problem.
This issue came up as a side issue on #13896. The trashcan is a rather complicated thing to get working right. In particular, cython must take precautions to ensure that once deallocation is started on an object, it isn't put into the trashcan in deallocs of base types.
This is now upstream http://trac.cython.org/cython_trac/ticket/797
Upstream: Reported upstream. Developers acknowledge bug.
CC: @simon-king-jena
Component: memleak
Issue created by migration from https://trac.sagemath.org/ticket/13901
The text was updated successfully, but these errors were encountered: