copy raises SystemError when getstate raises exception

[gerrit]

BPO	749831
Files	brian.sav: result of succesful pickle

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2003-06-08.13:20:10.000>
created_at = <Date 2003-06-05.22:04:03.000>
labels = ['interpreter-core']
title = 'copy raises SystemError when getstate raises exception'
updated_at = <Date 2003-06-08.13:20:10.000>
user = 'https://bugs.python.org/gerrit'

bugs.python.org fields:

activity = <Date 2003-06-08.13:20:10.000>
actor = 'nnorwitz'
assignee = 'nnorwitz'
closed = True
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2003-06-05.22:04:03.000>
creator = 'gerrit'
dependencies = []
files = ['910']
hgrepos = []
issue_num = 749831
keywords = []
message_count = 6.0
messages = ['16262', '16263', '16264', '16265', '16266', '16267']
nosy_count = 2.0
nosy_names = ['nnorwitz', 'gerrit']
pr_nums = []
priority = 'high'
resolution = 'fixed'
stage = None
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue749831'
versions = ['Python 2.3']

[gerrit]

I'm sorry that I'm so far unable to track this down
better, but my app is quite complex I think. But if I'm
right cPickle should never raise a SystemError:

$ ./brian.py
mcop warning: user defined signal handler found for
SIG_PIPE, overriding
Traceback (most recent call last):
  File "./brian.py", line 265, in ?
    main()
  File "./brian.py", line 235, in main
    game.mainloop()
  File "./brian.py", line 100, in mainloop
    self.save(self.savefile)
  File "./brian.py", line 216, in save
    self.level.save(fp)
  File "level.py", line 113, in save
    cPickle.dump(self, f, -1)
SystemError: NULL object passed to Py_BuildValue
$ python -c 'import sys;print sys.version'
2.3b1+ (#2, Jun  4 2003, 17:16:59)
[GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)]

When I use (Py)pickle instead of cPickle, the traceback
becomes:

$ ./brian.py
mcop warning: user defined signal handler found for
SIG_PIPE, overriding
Traceback (most recent call last):
  File "./brian.py", line 266, in ?
    main()
  File "./brian.py", line 236, in main
    game.mainloop()
  File "./brian.py", line 100, in mainloop
    self.save(self.savefile)
  File "./brian.py", line 217, in save
    self.level.save(fp)
  File "/home/gerrit/cvs/brian/level.py", line 114, in save
    cPickle.dump(self, f, -1)
  File "/usr/local/lib/python2.3/pickle.py", line 1382,
in dump
    Pickler(file, protocol, bin).dump(obj)
  File "/usr/local/lib/python2.3/pickle.py", line 231,
in dump
    self.save(obj)
  File "/usr/local/lib/python2.3/pickle.py", line 338,
in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/local/lib/python2.3/pickle.py", line 433,
in save_reduce
    save(state)
  File "/usr/local/lib/python2.3/pickle.py", line 293,
in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/local/lib/python2.3/pickle.py", line 663,
in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/local/lib/python2.3/pickle.py", line 695,
in _batch_setitems
    save(v)
  File "/usr/local/lib/python2.3/pickle.py", line 313,
in save
    rv = reduce(self.proto)
SystemError: NULL object passed to Py_BuildValue

'self' is an object whose __dict__ has 4 items. The
'data' item is a set (Sets.set) with approx. 450 items.
Those are all instances of a class Sprite or one of its
subclasses. This is where it goes wrong.

The interesting part is that if a reference to
__class__ in __getstate__ of this Sprite is commented
out, (line 200 of
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/pybrian/brian/basesprites.py?annotate=1.18)
this bug no longer occurs. The file then pickled is
attached (it is gzipped). Unfortunately, the
__getstate__ relies on some images available.

The full CVS tree of the app concerned is available via
SF:
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/pybrian/brian/
When 'brian' is called without basesprites.py#200
commented out, this error occurs for Python 2.3.

I am going to try to track this down further tomorrow
and beyond, because I think that with the line in
getstate I am very far already.

Gerrit.

[gerrit]

Logged In: YES
user_id=13298

OK; one step further.
The __class__ does not exist in the dictionairy so it should
raise a KeyError. The above is true for any non-existing key
in this dictionairy.

[gerrit]

Logged In: YES
user_id=13298

Heh, sorry for replying to my own post so soon, but this
problem occurs for any exception: 1/0 in the code of
__getstate__ yields the same result.

[gerrit]

Logged In: YES
user_id=13298

OK, I tracked it even further:

Copying an old-style class behaves as expected:
 47 >>> class Foo:
 47 ...  def __getstate__(self): 1/0
 47 ...
 48 >>> copy.copy(Foo())
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.3/copy.py", line 75, in copy
    return copier(x)
  File "/usr/local/lib/python2.3/copy.py", line 148, in
_copy_inst
    state = x.__getstate__()
  File "<stdin>", line 2, in __getstate__
ZeroDivisionError: integer division or modulo by zero

However, copying a new-style class instance does not:

 49 >>> class Foo(object):
 49 ...  def __getstate__(self): 1/0
 49 ...
 50 >>> copy.copy(Foo())
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.3/copy.py", line 87, in copy
    rv = reductor(2)
SystemError: NULL object passed to Py_BuildValue

...or even easier...

 51 >>> class Foo(object):
 51 ...  def __getstate__(self): raise
 51 ...
 52 >>> copy.copy(Foo())
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/usr/local/lib/python2.3/copy.py", line 87, in copy
    rv = reductor(2)
SystemError: NULL object passed to Py_BuildValue

That's as close as I can get.

[gerrit]

Logged In: YES
user_id=13298

Python 2.2 does it right!

[nnorwitz]

Logged In: YES
user_id=33168

Checked in as:

• Objects/typeobject.c 2.235
• Lib/test/test_copy.py 1.9