tp_dealloc docs should mention error indicator may be set

[ezyang]

BPO	45210
Nosy	@vstinner, @ezyang
PRs	bpo-45210: Document that error indicator may be set in tp_dealloc #28358 bpo-45210: _Py_Dealloc() checks tp_dealloc exception #32357
Files	unicode_dealloc_bug.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-09-15.16:47:23.387>
labels = ['3.11', 'type-feature', 'docs']
title = 'tp_dealloc docs should mention error indicator may be set'
updated_at = <Date 2022-04-06.09:40:23.648>
user = 'https://github.com/ezyang'

bugs.python.org fields:

activity = <Date 2022-04-06.09:40:23.648>
actor = 'vstinner'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation']
creation = <Date 2021-09-15.16:47:23.387>
creator = 'ezyang'
dependencies = []
files = ['50724']
hgrepos = []
issue_num = 45210
keywords = ['patch']
message_count = 3.0
messages = ['401854', '401944', '416849']
nosy_count = 4.0
nosy_names = ['vstinner', 'docs@python', 'python-dev', 'ezyang']
pr_nums = ['28358', '32357']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue45210'
versions = ['Python 3.11']

[ezyang]

The fact that the error indicator may be set during tp_dealloc is somewhat well known (https://github.com/posborne/dbus-python/blob/fef4bccfc535c6c2819e3f15384600d7bc198bc5/_dbus_bindings/conn.c#L387) but it's not documented in the official manual. We should document it.

A simple suggested patch:

diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
index b17fb22b69..e7c9b13646 100644
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -668,6 +668,20 @@ and :c:type:`PyType_Type` effectively act as defaults.)
    :c:func:`PyObject_GC_Del` if the instance was allocated using
    :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar`.
 
+   If you may call functions that may set the error indicator, you must
+   use :c:func:`PyErr_Fetch` and :c:func:`PyErr_Restore` to ensure you
+   don't clobber a preexisting error indicator (the deallocation could
+   have occurred while processing a different error):
+
+   .. code-block:: c
+
+     static void foo_dealloc(foo_object *self) {
+         PyObject *et, *ev, *etb;
+         PyErr_Fetch(&et, &ev, &etb);
+         ...
+         PyErr_Restore(et, ev, etb);
+     }
+
    Finally, if the type is heap allocated (:const:`Py_TPFLAGS_HEAPTYPE`), the
    deallocator should decrement the reference count for its type object after
    calling the type deallocator. In order to avoid dangling pointers, the

[vstinner]

I'm not sure that it's a feature. Maybe we should modify to never call tp_dealloc with an exception set.

[vstinner]

I wrote #32357 to check in debug mode that tp_dealloc functions leave the current exception unchanged. You can use attached unicode_dealloc_bug.patch to inject a bug to test my PR. Example ("make" is enough to trigger the bug):
---
Fatal Python error: _Py_Dealloc: Deallocator of type 'str' cleared the current exception
Python runtime state: initialized

Current thread 0x00007ff28d45a740 (most recent call first):
File "/home/vstinner/python/main/Lib/sysconfig.py", line 349 in _parse_makefile
File "/home/vstinner/python/main/Lib/sysconfig.py", line 470 in _generate_posix_vars
File "/home/vstinner/python/main/Lib/sysconfig.py", line 845 in _main
File "/home/vstinner/python/main/Lib/sysconfig.py", line 857 in <module>
File "<frozen runpy>", line 88 in _run_code
File "<frozen runpy>", line 198 in _run_module_as_main
generate-posix-vars failed
---

[vstinner]

I merged #32357: commit 364ed94