Chained __slots__ dealloc segfault

[jhogg]

BPO	574207
Nosy	@gvanrossum, @tim-one
Files	a.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/gvanrossum'
closed_at = <Date 2002-08-06.21:43:59.000>
created_at = <Date 2002-06-26.18:35:34.000>
labels = ['interpreter-core']
title = 'Chained __slots__ dealloc segfault'
updated_at = <Date 2002-08-06.21:43:59.000>
user = 'https://bugs.python.org/jhogg'

bugs.python.org fields:

activity = <Date 2002-08-06.21:43:59.000>
actor = 'gvanrossum'
assignee = 'gvanrossum'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2002-06-26.18:35:34.000>
creator = 'jhogg'
dependencies = []
files = ['532']
hgrepos = []
issue_num = 574207
keywords = []
message_count = 9.0
messages = ['11332', '11333', '11334', '11335', '11336', '11337', '11338', '11339', '11340']
nosy_count = 4.0
nosy_names = ['gvanrossum', 'tim.peters', 'nnorwitz', 'jhogg']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue574207'
versions = ['Python 2.2']

[jhogg]

With the following script ('foo.py'):

-----

class foo1(object):
    def __init__(self, n):
        self.n = n

class foo2(object):
    def __init__(self, n):
        self.n = n
    __slots__ = ['n']

o = None
for i in xrange(50000):
    o = foo1(o)
print 'alloc OK'
del o 
print 'dealloc OK'

o = None 
for i in xrange(50000):
    o = foo2(o)
print 'alloc OK'
del o 
print 'OK'

---

Running this results in a segmentation fault in the
second 'del':

-----
onegood1-2% python foo.py
alloc OK
dealloc OK
alloc OK
zsh: 18593 segmentation fault python foo.py
-----

A gdb stack trace shows:

-----
Program received signal EXC_BAD_ACCESS, Could not
access memory.
0x00037bbc in lookdict_string (mp=0xa7fb8, key=0x191d0,
hash=3882000) at Objects/dictobject.c:306
306 {
(gdb) where
#0 0x00037bbc in lookdict_string (mp=0xa7fb8,
key=0x191d0, hash=3882000) at Objects/dictobject.c:306
#1 0x00018994 in _PyType_Lookup (type=0x3d14f0,
name=0x3b3c10) at Objects/typeobject.c:1227
#2 0x00017348 in lookup_maybe (self=0xbb88f0,
attrstr=0x3d14f0 "", attrobj=0xa0228) at
Objects/typeobject.c:460
#3 0x00016f38 in call_finalizer (self=0xbb88f0) at
Objects/typeobject.c:280
#4 0x00017054 in subtype_dealloc (self=0xbb88f0) at
Objects/typeobject.c:328
#5 0x00017118 in subtype_dealloc (self=0xbb88d0) at
Objects/typeobject.c:353
#6 0x00017118 in subtype_dealloc (self=0xbb88b0) at
Objects/typeobject.c:353
[...]
bpo-5431 0x00017118 in subtype_dealloc (self=0xbe3410) at
Objects/typeobject.c:353
bpo-5432 0x00017118 in subtype_dealloc (self=0xbe33f0) at
Objects/typeobject.c:353
bpo-5433 0x00017118 in subtype_dealloc (self=0xbe33d0) at
Objects/typeobject.c:353
bpo-5434 0x000382f4 in PyDict_DelItem (op=0x3b3300,
key=0x405520) at Objects/dictobject.c:588
bpo-5435 0x0005b348 in eval_frame (f=0x0) at
Python/ceval.c:1550
bpo-5436 0x0005d6fc in PyEval_EvalCodeEx (co=0x3fd760,
globals=0x402d60, locals=0x3804c, args=0x0, argcount=0,
kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0)
at Python/ceval.c:2585
bpo-5437 0x000588e0 in PyEval_EvalCode (co=0x3d14f0,
globals=0x402d60, locals=0x46257b48) at Python/ceval.c:483
bpo-5438 0x00027850 in run_node (n=0x3bfba0,
filename=0x402d60 "", globals=0x3b3300,
locals=0x3b3300, flags=0x3d16e8) at Python/pythonrun.c:1079
bpo-5439 0x000277f4 in run_err_node (n=0x3804c,
filename=0x402d60 "", globals=0x46257b48, locals=0x0,
flags=0x3d16e8) at Python/pythonrun.c:1066
bpo-5440 0x000277c0 in PyRun_FileExFlags (fp=0x80008a98,
filename=0xbffffb46 "foo.py", start=1176861512,
globals=0x3b3300, locals=0x3b3300, closeit=1,
flags=0xbffff9c8) at Python/pythonrun.c:1057
bpo-5441 0x0002683c in PyRun_SimpleFileExFlags
(fp=0x80008a98, filename=0xbffffb46 "foo.py",
closeit=1, flags=0xbffff9c8) at Python/pythonrun.c:685
bpo-5442 0x00026254 in PyRun_AnyFileExFlags
(fp=0x80008a98, filename=0xbffffb46 "foo.py",
closeit=1, flags=0xbffff9c8) at Python/pythonrun.c:495
bpo-5443 0x00005e10 in Py_Main (argc=2, argv=0xbffffab0)
at Modules/main.c:364
bpo-5444 0x00001e2c in _start ()
bpo-5445 0x00001c5c in start ()
-----

The number of objects that can be deallocated depends
on the stacksize, so it appears to be a stack overrun
problem.

This is on a Mac OS X 10.1.5 system, but was originally
reported by Glyph Lefkowitz using Python 2.2.1 on PPC
Linux 2.4.18 (Debian I think) and has been verified
also by Aahz using Python 2.2.1 on Mandrake 8.1.

[nnorwitz]

Logged In: YES
user_id=33168

I do not get this problem with current CVS.
I do get this problem with 2.2.1+ (current).
Attaching the test file I used.

[tim-one]

Logged In: YES
user_id=31435

Neal, it's odd that you can't provoke a problem. Try
boosting MAX. The del here triggers a chain of
deallocations that requires call-stack space proportional to
the length of the object chain (MAX). This is the kind of
situation Python's infamous TRASHCAN macros aim at
worming around, but the slot cleanup code doesn't play
that game (yet).

Assigned to Guido just because he'll enjoy it <wink>.

[nnorwitz]

Logged In: YES
user_id=33168

I bumped MAX up to 500000 and 2.3 didn't crash.
But then I dropped my max stack size down from 8M to 1M
and that caused the crash. So the stack handling must be
better in 2.3, but as you point out, still not fixed.

[jhogg]

Logged In: YES
user_id=10036

I've uploaded a patch (against HEAD today) that fixes this
problem by putting the objects in the slots into a temporary
list and appending that to the PyTRASHCAN 'delete_later'
chain when the maximum deallocation recursion depth is reached.

[ 575073 ] PyTRASHCAN slots deallocation

[jhogg]

Logged In: YES
user_id=10036

I updated my original patch (575073) to match recent changes
to HEAD. I also added a new patch: 581742 "Alternative
PyTRASHCAN subtype_dealloc", which wraps 'subtype_dealloc'
in the normal (though slightly modified) PyTRASHCAN macros -
but it has to do ugly things to keep the original GC semantics.

[gvanrossum]

Logged In: YES
user_id=6380

Tim can review this better, since AFAIK he rewrote the
trashcan macros most recently.

[tim-one]

Logged In: YES
user_id=31435

Back to Guido, since he's looking at this now.

[gvanrossum]

Logged In: YES
user_id=6380

Fixed in CVS. Now, what to do about 2.2? The problem still
exists in that lineage; it's not easy to backport the patch
there because the trashcan macros work very differently...