pickling instance which inherited from Exception with keyword only parameter

[liugang93]

BPO	37489
Nosy	@pitrou, @avassalotti, @iritkatriel
Superseder	bpo-27015: pickling and repr of exceptions with kwargs

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2022-01-06.13:22:02.926>
created_at = <Date 2019-07-03.03:12:31.576>
labels = ['3.7', 'type-bug', 'library']
title = 'pickling instance which inherited from Exception with keyword only parameter'
updated_at = <Date 2022-01-06.13:22:02.925>
user = 'https://bugs.python.org/liugang93'

bugs.python.org fields:

activity = <Date 2022-01-06.13:22:02.925>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2022-01-06.13:22:02.926>
closer = 'iritkatriel'
components = ['Library (Lib)']
creation = <Date 2019-07-03.03:12:31.576>
creator = 'liugang93'
dependencies = []
files = []
hgrepos = []
issue_num = 37489
keywords = []
message_count = 4.0
messages = ['347178', '347519', '409840', '409844']
nosy_count = 5.0
nosy_names = ['pitrou', 'alexandre.vassalotti', 'liyang1025@gmail.com', 'liugang93', 'iritkatriel']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '27015'
type = 'behavior'
url = 'https://bugs.python.org/issue37489'
versions = ['Python 3.7']

[liugang93]

-- code 1
import pickle

class MyException():
    def __init__(self, desc, *, item):
        super().__init__()
        self.desc = desc
        self.item = item

    def __getnewargs_ex__(self):
        print('called in {}.__getnewargs_ex__'.format(self.__class__.__name__))
        return (self.desc,), self.__dict__

e = MyException('testing', item='cpu')
s = pickle.dumps(e, protocol=-1)

x = pickle.loads(s)

-- code 2
import pickle

class MyException(Exception):
    def __init__(self, desc, *, item):
        super().__init__()
        self.desc = desc
        self.item = item

    def __getnewargs_ex__(self):
        print('called in {}.__getnewargs_ex__'.format(self.__class__.__name__))
        return (self.desc,), self.__dict__

e = MyException('testing', item='cpu')
s = pickle.dumps(e, protocol=-1)

x = pickle.loads(s)

in code 1, the class is inherted from object, __getnewargs_ex__ is called and the returned args, kwargs are passed to __new__/init to construct object when pickling.

in code 2, the class is inherted from Exception, __getnewargs_ex__ is not called, and rasie an exception of "TypeError: __init__() missing 1 required positional argument: 'desc'"

I think this is not python issue, it should be the right behavior of Exception. I want to known why it is, and how to implement the logic in code 2 (passing keyword only parameter to __new__/init when pickling for class inherted from Exception)

[liyang1025gmailcom]

yesterday i have test two case, first i find the code 1 not execute the __init__ when execute loads. second the code 1 and code 2 dumps result s is not equal,because the dumps is the buildin function ,so i can't find more message.but if the code2 only have keyword parameter the code2 is not raise exception. i want to debug the interpreter that can get the reason.

[iritkatriel]

I want to known why it is

It's because Exception implements __reduce__ and that takes precedence over __getnewargs_ex__. You can see that with this example:

""""
import pickle

class MyException():
    def __init__(self, desc, item):
        super().__init__()
        self.desc = desc
        self.item = item

    def __getnewargs_ex__(self):
        print('called in {}.__getnewargs_ex__'.format(self.__class__.__name__))
        return (self.desc,), self.__dict__

    def __reduce__(self):
        print('called in {}.__reduce__'.format(self.__class__.__name__))
        return MyException, (self.desc, self.item),

e = MyException('testing', item='cpu')
s = pickle.dumps(e, protocol=-1)

x = pickle.loads(s)
""""

Output: called in MyException.__reduce__

[iritkatriel]

I want to known [...] how to implement the logic in code 2 (passing keyword only parameter to __new__/init when pickling for class inherted from Exception)

This is currently a problem, see bpo-27015.