[Python] support serialize local function

[chaokunyang]

Feature Request

Pickle doesn't support serialize local class object, for example:

def f():
    from dataclasses import dataclass

    @dataclass
    class LocalClass:
        a: int
        b: str

    local_class_instance = LocalClass(a=1, b="2")
    return local_class_instance


import pickle

pickle.dumps(f())

will get error:

AttributeError                            Traceback (most recent call last)
Cell In[6], line 15
     10     return local_class_instance
     13 import pickle
---> 15 pickle.dumps(f())

AttributeError: Can't pickle local object 'f.<locals>.LocalClass'

Fory can serialize such object:

In [23]: fory.register_type(type(o))

In [24]: o
Out[24]: f.<locals>.LocalClass(a=1, b='2')

In [25]: fory.register_type(type(o))

In [26]: fory.serialize(o)
Out[26]: b'\xd4b\x06\x02\xff\x8f\x8a\x01\x97@\x00\x00\xff\x02\xff\x042'

In [27]: fory.deserialize(o)

But when it comes to function or lambda, fory doesn't support:

In [35]: lam = lambda x: x*2

In [36]: fory.register_type(type(lam))
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[36], line 1
----> 1 fory.register_type(type(lam))

File python/pyfory/_serialization.pyx:648, in pyfory._serialization.Fory.register_type()

File python/pyfory/_serialization.pyx:467, in pyfory._serialization.TypeResolver.register_type()

File ~/Desktop/chaokun/fury_open_source/python/pyfory/_registry.py:284, in TypeResolver.register_type(self, cls, type_id, namespace, typename, serializer)
    275 def register_type(
    276     self,
    277     cls: Union[type, TypeVar],
   (...)
    282     serializer=None,
    283 ):
--> 284     return self._register_type(
    285         cls,
    286         type_id=type_id,
    287         namespace=namespace,
    288         typename=typename,
    289         serializer=serializer,
    290     )

File ~/Desktop/chaokun/fury_open_source/python/pyfory/_registry.py:324, in TypeResolver._register_type(self, cls, type_id, namespace, typename, serializer, internal)
    322     raise TypeError(f"{cls} registered already")
    323 register_type = self._register_xtype if self.fory.language == Language.XLANG else self._register_pytype
--> 324 return register_type(
    325     cls,
    326     type_id=type_id,
    327     namespace=namespace,
    328     typename=typename,
    329     serializer=serializer,
    330     internal=internal,
    331 )

File ~/Desktop/chaokun/fury_open_source/python/pyfory/_registry.py:348, in TypeResolver._register_xtype(self, cls, type_id, namespace, typename, serializer, internal)
    346         type_id = TypeId.NAMED_ENUM if type_id is None else ((type_id << 8) + TypeId.ENUM)
    347     else:
--> 348         serializer = DataClassSerializer(self.fory, cls, xlang=True)
    349         type_id = TypeId.NAMED_STRUCT if type_id is None else ((type_id << 8) + TypeId.STRUCT)
    350 elif not internal:

File ~/Desktop/chaokun/fury_open_source/python/pyfory/serializer.py:300, in DataClassSerializer.__init__(self, fory, clz, xlang)
    298 self._xlang = xlang
    299 # This will get superclass type hints too.
--> 300 self._type_hints = typing.get_type_hints(clz)
    301 self._field_names = sorted(self._type_hints.keys())
    302 self._has_slots = hasattr(clz, "__slots__")

File /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/typing.py:1227, in get_type_hints(obj, globalns, localns)
   1225     base_globals = globalns
   1226 ann = base.__dict__.get('__annotations__', {})
-> 1227 for name, value in ann.items():
   1228     if value is None:
   1229         value = type(None)

AttributeError: 'getset_descriptor' object has no attribute 'items'

pickle can't serialize such objects:

In [37]: pickle.dumps(lam)
---------------------------------------------------------------------------
PicklingError                             Traceback (most recent call last)
Cell In[37], line 1
----> 1 pickle.dumps(lam)

PicklingError: Can't pickle <function <lambda> at 0x7f970be17280>: attribute lookup <lambda> on __main__ failed

cloudpickle can serialize such objects:

In [39]: import cloudpickle

In [40]: cloudpickle.dumps(lam)
Out[40]: b'\x80\x05\x95\xb6\x01\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x0e_make_function\x94\x93\x94(h\x00\x8c\r_builtin_type\x94\x93\x94\x8c\x08CodeType\x94\x85\x94R\x94(K\x01K\x00K\x00K\x01K\x02KCC\x08|\x00d\x01\x14\x00S\x00\x94NK\x02\x86\x94)\x8c\x01x\x94\x85\x94\x8c\x1f<ipython-input-35-9493230fda08>\x94\x8c\x08<lambda>\x94K\x01C\x00\x94))t\x94R\x94}\x94(\x8c\x0b__package__\x94N\x8c\x08__name__\x94\x8c\x08__main__\x94uNNNt\x94R\x94h\x00\x8c\x12_function_setstate\x94\x93\x94h\x16}\x94}\x94(h\x13\x8c\x08<lambda>\x94\x8c\x0c__qualname__\x94\x8c\x08<lambda>\x94\x8c\x0f__annotations__\x94}\x94\x8c\x0e__kwdefaults__\x94N\x8c\x0c__defaults__\x94N\x8c\n__module__\x94h\x14\x8c\x07__doc__\x94N\x8c\x0b__closure__\x94N\x8c\x17_cloudpickle_submodules\x94]\x94\x8c\x0b__globals__\x94}\x94u\x86\x94\x86R0.'

Is your feature request related to a problem? Please describe

No response

Describe the solution you'd like

No response

Describe alternatives you've considered

No response

Additional context

No response

[chaokunyang]

@esafak

[esafak]

Done, I think