@cython.trashcan directive to enable the Python trashcan for deallocations #2842
Conversation
Cython/Utility/ExtensionTypes.c
Outdated
| @@ -74,6 +74,41 @@ static int __Pyx_PyType_Ready(PyTypeObject *t) { | |||
| return r; | |||
| } | |||
|
|
|||
| /////////////// TRASHCAN.proto /////////////// | |||
There was a problem hiding this comment.
No need to scream. ;-) Just name it PyTrashcan or PyTrashcanSupport.
There was a problem hiding this comment.
No need to scream. ;-) Just name it
PyTrashcanorPyTrashcanSupport.
I prefer to use TRASHCAN in the name because the upstream macros are also called Py_TRASHCAN_SAFE_BEGIN and Py_TRASHCAN_SAFE_END. It will be more confusing if Cython uses a different made-up name.
There was a problem hiding this comment.
I was referring to the name of the utility code snippet, not the C macros (where upper case is normal). There is no precedent for an all-upper-case utility code name.
Cython/Utility/ExtensionTypes.c
Outdated
| // Unlike the Py_TRASHCAN_SAFE_BEGIN/Py_TRASHCAN_SAFE_END macros, they | ||
| // allow dealing correctly with subclasses. | ||
| // | ||
| // This requires Python version >= 2.7.4 or >= 3.2.4 |
There was a problem hiding this comment.
We could increase the minimum version for Cy3.0 iff this feature is used, i.e. we can make it fail to compile with a C #error right here if PY_VERSION_HEX < 2.7.4.
But I'd rather turn the macros into no-ops for older versions. In 99% of the cases, that's still better than not working at all. I mean, you can currently get a crash with Py2.7.3 in rare situations, and you'd get a crash in the future for them. Nothing lost, really.
|
|
||
| // These macros are taken from https://github.com/python/cpython/pull/11841 | ||
| // Unlike the Py_TRASHCAN_SAFE_BEGIN/Py_TRASHCAN_SAFE_END macros, they | ||
| // allow dealing correctly with subclasses. |
There was a problem hiding this comment.
We should consider using the original macros in Py3.8+ if/when that contains the fixes.
I also wonder if PyPy's cpyext supports the trashcan at all, or if this is a CPython-only feature (for now?).
|
Concerning testing: it's hard to test that the trashcan is not used: the only observable effect is a stack overflow (typically leading to a segmentation fault). |
|
Actually, I spoke too quick: the order of deallocations between base class and subclass is different, so trashcan usage can be detected. |
|
PyPy doesn't use the trashcan (it supports the |
|
I think that I took care of all your comments. |
|
Perfect! I'll merge it right away to avoid bit-rot, because it is a clear improvement over not supporting the trashcan mechanism at all. If the CPython devs decide that they want thing different, we can adapt to it on our side later. |
|
Thanks! Should I take care of docs and changelog? |
|
Please do. :) |
|
I finally took the time to understand this, and not that it matters (since this is already merged), but this makes sense to me now and I agree that it's useful to be able to (re-)use this mechanism when implementing Cython types. Thanks! My thinking had been that it should be possible to re-implement something like this specifically for cypari, and while it could be done it would be much more of a PITA, when we can reuse the mechanism that's already there in the Python interpreter for exactly this purpose. |
|
IIUC it should be possible to use this also without the additional feature in Cython, but it's certainly cleaner this way. |
Fix #1354
This enables a
@cython.trashcandirective which can enable using the Python trashcan for deallocations. This is optional, disabled by default (but one could argue about making it enabled by default).Also see https://bugs.python.org/issue35983