gc fixes for module m_copy attribute

[nascheme]

BPO	10241
Nosy	@loewis, @nascheme, @pitrou, @benjaminp, @skrah
Files	module_m_copy.txt module_m_copy2.txt valgrind.out

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-09-29.17:24:18.473>
created_at = <Date 2010-10-30.03:48:49.342>
labels = ['interpreter-core', 'performance']
title = 'gc fixes for module m_copy attribute'
updated_at = <Date 2013-10-27.16:36:30.192>
user = 'https://github.com/nascheme'

bugs.python.org fields:

activity = <Date 2013-10-27.16:36:30.192>
actor = 'skrah'
assignee = 'none'
closed = True
closed_date = <Date 2013-09-29.17:24:18.473>
closer = 'pitrou'
components = ['Interpreter Core']
creation = <Date 2010-10-30.03:48:49.342>
creator = 'nascheme'
dependencies = []
files = ['19423', '19509', '32377']
hgrepos = []
issue_num = 10241
keywords = []
message_count = 22.0
messages = ['119957', '120063', '120553', '194108', '194109', '194110', '194120', '194127', '194131', '194194', '194195', '194844', '194845', '198609', '201390', '201391', '201452', '201464', '201468', '201469', '201470', '201476']
nosy_count = 7.0
nosy_names = ['loewis', 'nascheme', 'pitrou', 'benjamin.peterson', 'Arfrever', 'skrah', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue10241'
versions = ['Python 3.4']

[nascheme]

I'm trying implement some saner module shutdown procedure (similar to bpo-812369). One of the many problems I've run into is that the GC doesn't know about the m_copy attribute of modules.

I think the attached patch is correct. tp_tranverse exposes m_copy if it is non-NULL. tp_clear calls Py_CLEAR on it.

[nascheme]

Oops, my patch doesn't work since m_base can be shared by more than one module instance. I guess a different solution would be to cleanup the m_copy references on interpreter shutdown. Somehow they would have to be found though.

[nascheme]

The attached patch seems better. The copies of module dicts stored in the interpreter state are dereferenced when the interpreter shuts down.

[pitrou]

Ok, the patch doesn't apply cleanly but looks good regardless.
(although I wonder whether you really reclaim significant stuff here: most C extension dicts should be mostly references to C methods and types)

[pitrou]

I will still empty modules_by_index rather than Py_CLEAR it, though.

[python-dev]

New changeset 62658d9d8926 by Antoine Pitrou in branch 'default':
Issue bpo-10241: Clear extension module dict copies at interpreter shutdown.
http://hg.python.org/cpython/rev/62658d9d8926

[pitrou]

Reopening. This seems to produce segfaults on several of the buildbots, e.g.:
http://buildbot.python.org/all/builders/x86%20XP-4%203.x/builds/8947
http://buildbot.python.org/all/builders/x86%20Windows%20Server%202003%20%5BSB%5D%203.x/builds/1257
http://buildbot.python.org/all/builders/x86%20Gentoo%203.x/builds/4671