New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cdef classes fails at pickling using `__setstate__` interface in Python 2. #2757

Open
rainwoodman opened this Issue Dec 12, 2018 · 1 comment

Comments

Projects
None yet
1 participant
@rainwoodman
Copy link

rainwoodman commented Dec 12, 2018

Here is an example (fails for Python 2, works for Python 3)

def _create_A_from_state(kls, state):
    obj = A.__new__(kls)
    obj.__setstate__(state)
    return obj

cdef class A:
    def __cinit__(self): pass
    def __init__(self): pass
    def __reduce__(self): return (_create_A_from_state, (type(self), self.__getstate__()))
    def __getstate__(self): return ()
    def __setstate__(self, state): return None

cdef class B:
    def __cinit__(self): pass
    def __init__(self): pass
    def __getstate__(self): return ()

    def __setstate__(self, state): return None

class AA(A): pass
class BB(B): pass

def main():
    import pickle
    obj = AA()
    s = pickle.dumps(obj)
    obj2 = pickle.loads(s)
    assert type(obj2) == AA

    obj = BB()
    s = pickle.dumps(obj)
    obj2 = pickle.loads(s)
    assert type(obj2) == BB

If one provides __reduce__, pickling works (AA, the first assertion).
If one omits __reduce__, the second case (BB, before the second assertion) fails at line 31:

$ python -c 'import pyximport; pyximport.install(); import failcy; failcy.main()'

...

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "failcy.pyx", line 31, in failcy.main
    s = pickle.dumps(obj)
  File "/home/yfeng1/anaconda3/install/envs/2.7/lib/python2.7/pickle.py", line 1380, in dumps
    Pickler(file, protocol).dump(obj)
  File "/home/yfeng1/anaconda3/install/envs/2.7/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/home/yfeng1/anaconda3/install/envs/2.7/lib/python2.7/pickle.py", line 306, in save
    rv = reduce(self.proto)
  File "/home/yfeng1/anaconda3/install/envs/2.7/lib/python2.7/copy_reg.py", line 71, in _reduce_ex
    state = base(self)
TypeError: __init__() takes exactly 0 positional arguments (1 given)

@rainwoodman

This comment has been minimized.

Copy link

rainwoodman commented Dec 12, 2018

The documentation was not clear that the BB case shall fail, and a google search on 'cython pickle' lands to an example that uses the BB case:

https://snorfalorpagus.net/blog/2016/04/16/pickling-cython-classes/

Since Python is not going through the __new__ function, I do not think the BB case shall work at all. But it could be just a bug in Cython that reported a wrong signature for init to Python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment