_pickle should implement the module finalisation protocol

[ncoghlan]

BPO	11349
Nosy	@ncoghlan, @pitrou, @vstinner, @avassalotti, @benjaminp, @Trundle

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/avassalotti'
closed_at = <Date 2013-12-01.21:52:09.998>
created_at = <Date 2011-02-28.01:29:10.979>
labels = ['extension-modules', 'type-feature']
title = '_pickle should implement the module finalisation protocol'
updated_at = <Date 2013-12-01.21:52:09.996>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2013-12-01.21:52:09.996>
actor = 'alexandre.vassalotti'
assignee = 'alexandre.vassalotti'
closed = True
closed_date = <Date 2013-12-01.21:52:09.998>
closer = 'alexandre.vassalotti'
components = ['Extension Modules']
creation = <Date 2011-02-28.01:29:10.979>
creator = 'ncoghlan'
dependencies = []
files = []
hgrepos = []
issue_num = 11349
keywords = []
message_count = 8.0
messages = ['129677', '129679', '129728', '129730', '129748', '129749', '129774', '204971']
nosy_count = 6.0
nosy_names = ['ncoghlan', 'pitrou', 'vstinner', 'alexandre.vassalotti', 'benjamin.peterson', 'Trundle']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue11349'
versions = ['Python 3.3']

[ncoghlan]

As discussed in bpo-11321, _pickle allocates a number of module level objects, but doesn't use the appropriate PEP-3121 mechanisms to manage their lifecycle.

It should be updated to follow the relevant development guidelines in the documentation.

[benjaminp]

Unfortunately "proper" module finalization is an invitation to more segfaults because modules can be finalized before objects in them are, resulting in segfaults when its attempted to access modules.

[ncoghlan]

In that case, the request should probably read "_pickle should implement the module finalisation protocol, and module finalisation should be designed so that doing so isn't a recipe for segfaults".

It strikes me as being very similar to the existing problems we have with modules setting everything to None during shutdown, so my estimation of the difficulty of this request has risen accordingly :P

[benjaminp]

It's a chicken and egg problem because proper module finalization techniques like bpo-812369 are bogged down by static globals in extension modules, and proper state storage in modules is prevented by proper module finalization.

[ncoghlan]

We might be able to tapdance our way around the issue by falling back to the old mechanism when dealing with extension modules without traverse/clear/dealloc methods.

[pitrou]

Not sure why bpo-812369 is bogged down. The patch simply needs updating for 3.x.

[benjaminp]

http://mail.python.org/pipermail/python-dev/2009-October/093057.html

2011/3/1 Antoine Pitrou <report@bugs.python.org>:

Antoine Pitrou <pitrou@free.fr> added the comment:

Not sure why bpo-812369 is bogged down. The patch simply needs updating for 3.x.

----------
nosy: +pitrou

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue11349\>

---

[avassalotti]

I have implemented PEP-3121 module finalization for _pickle in 64c6d52793be.