Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support pickling slots in subclasses of common classes #70766

Closed
serhiy-storchaka opened this issue Mar 17, 2016 · 5 comments
Closed

Support pickling slots in subclasses of common classes #70766

serhiy-storchaka opened this issue Mar 17, 2016 · 5 comments
Labels
3.11 extension-modules C modules in the Modules dir stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@serhiy-storchaka
Copy link
Member

serhiy-storchaka commented Mar 17, 2016

BPO 26579
Nosy @rhettinger, @pitrou, @vstinner, @avassalotti, @bitdancer, @berkerpeksag, @serhiy-storchaka
PRs
  • bpo-26579: Add object.__getstate__(). #2821
  • Files
  • copyreg_getstate.patch
  • copyreg_getstate2.patch
  • object_getstate.patch: Add object.getstate
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2022-04-06.17:02:02.310>
    created_at = <Date 2016-03-17.08:43:06.421>
    labels = ['extension-modules', 'type-feature', 'library', '3.11']
    title = 'Support pickling slots in subclasses of common classes'
    updated_at = <Date 2022-04-07.14:57:28.953>
    user = 'https://github.com/serhiy-storchaka'

    bugs.python.org fields:

    activity = <Date 2022-04-07.14:57:28.953>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2022-04-06.17:02:02.310>
    closer = 'serhiy.storchaka'
    components = ['Extension Modules', 'Library (Lib)']
    creation = <Date 2016-03-17.08:43:06.421>
    creator = 'serhiy.storchaka'
    dependencies = []
    files = ['42183', '43462', '43716']
    hgrepos = []
    issue_num = 26579
    keywords = ['patch']
    message_count = 5.0
    messages = ['261903', '268830', '270403', '416888', '416931']
    nosy_count = 7.0
    nosy_names = ['rhettinger', 'pitrou', 'vstinner', 'alexandre.vassalotti', 'r.david.murray', 'berker.peksag', 'serhiy.storchaka']
    pr_nums = ['2821']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'enhancement'
    url = 'https://bugs.python.org/issue26579'
    versions = ['Python 3.11']

    @serhiy-storchaka
    Copy link
    Member Author

    serhiy-storchaka commented Mar 17, 2016

    Pickling and copying instances of subclasses of some basic classes pickles and copies instance attributes. Example:

    >>> class BA(bytearray):
    ...     pass
    ... 
    >>> b = BA(b'abc')
    >>> b.x = 10
    >>> c = copy.copy(b)
    >>> c.x
    10
    >>> c = pickle.loads(pickle.dumps(b))
    >>> c.x
    10

    But this doesn't work if attributes are saved not in instance dictionary, but in slots.

    >>> class BA(bytearray):
    ...     __slots__ = ('x',)
    ... 
    >>> b = BA(b'abc')
    >>> b.x = 10
    >>> c = copy.copy(b)
    >>> c.x
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: x
    >>> c = pickle.loads(pickle.dumps(b))
    >>> c.x
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: x

    Since using __slots__ is implementation detail, this failure can be considered as a bug.

    Proposed patch adds support of pickling and copying slots in subclasses of all classes that already support pickling and copying non-slot attributes. It is backward compatible, classes with slots can be unpickled on older Python versions without slots. Affected classes: bytearray, set, frozenset, weakref.WeakSet, collections.OrderedDict, collections.deque, datetime.tzinfo.

    The patch adds the copyreg._getstate() function for Python classes and exposes the _PyObject_GetState() function for extension classes. An alternative (and simpler for end user) solution would be to add default implementation as object.__getstate__(). But this is not easy to reject non-pickleable classes (bpo-22995) in this case, since __getstate__ is looked up as instance attribute, not as other special methods.

    @serhiy-storchaka serhiy-storchaka added extension-modules C modules in the Modules dir stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Mar 17, 2016
    @serhiy-storchaka
    Copy link
    Member Author

    serhiy-storchaka commented Jun 18, 2016

    Synchronized with current sources.

    @serhiy-storchaka
    Copy link
    Member Author

    serhiy-storchaka commented Jul 14, 2016

    An alternative way is to expose a default state as object.__getstate__(). It is more efficient since it is implemented in C. Following patch implements this approach.

    @serhiy-storchaka
    Copy link
    Member Author

    serhiy-storchaka commented Apr 6, 2022

    New changeset 884eba3 by Serhiy Storchaka in branch 'main':
    bpo-26579: Add object.__getstate__(). (GH-2821)
    884eba3

    @vstinner
    Copy link
    Member

    vstinner commented Apr 7, 2022

    bpo-26579: Add object.__getstate__(). (GH-2821)
    884eba3

    This change introduced reference leaks: see bpo-47250.

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    oscarbenjamin added a commit to oscarbenjamin/sympy that referenced this issue May 16, 2022
    This is needed after the changes in
    
      python/cpython#70766
    
    Since object now has a __getstate__ method we need to make sure not to
    call it with any arguments. Probably the __getstate__ methods can be
    simplified (or removed?) in light of the cpython changes but for now
    this is a quick fix to restore previous behaviour.
    oscarbenjamin added a commit to oscarbenjamin/sympy that referenced this issue May 16, 2022
    This is needed after the changes in
    
      python/cpython#70766
    
    Since object now has a __getstate__ method we need to make sure not to
    call it with any arguments. Probably the __getstate__ methods can be
    simplified (or removed?) in light of the cpython changes but for now
    this is a quick fix to restore previous behaviour.
    oscarbenjamin added a commit to oscarbenjamin/sympy that referenced this issue May 16, 2022
    This is needed after the changes in
    
      python/cpython#70766
    
    Since object now has a __getstate__ method we need to make sure not to
    call it with any arguments. Probably the __getstate__ methods can be
    simplified (or removed?) in light of the cpython changes but for now
    this is a quick fix to restore previous behaviour.
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.11 extension-modules C modules in the Modules dir stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants