Unpacking dictionary with attrs class as attribute

[jhossbach]

Hey there,
I am not sure if this has been answered before, but I noticed something unintuitive when I tried to unstructure a class with another attrs class. This works fine:

@attrs.define
class A:
    a: int
    list_: list[int]
    
dict_ = {"a": 3, "list_": [2,3,4]}
A(**dict_)

# >>> A(a=3, list_=[2, 3, 4])

This on the other hand does not:

import attrs

@attrs.define
class A:
    a: int

@attrs.define
class B:
    a: A
    a_s: list[A]
    
a1, a2 = A(2), A(3)
b = B(a1, [a1, a2])
dict_ = attrs.asdict(b)
 # >>> {'a': {'a': 2}, 'a_s': [{'a': 2}, {'a': 3}]}
B(**dict_)
# >>> B(a={'a': 2}, a_s=[{'a': 2}, {'a': 3}])

Notice how neither a nor a_s is converted back to the proper type. Now I know that cattrs provides a solution to this, namely that I can structure back using cattrs.structure(dict_, B), but I was curious if this is in fact intended behavior or if this is unwanted.
I'd love to hear an explanation why this behavior is desired (if it is)!

[hynek]

Yes this is "on purpose" insofar that __init__ and addict only do exactly what's on the tin and what you'd expect from a vanilla class. :) attrs tries to keep its classes a "plain" as possible while empowering extensions with hooks and metadata.

[jhossbach]

Thanks for the answer, I have an implementation involving the register_structure_hook_func from cattrs like this:

conv.register_structure_hook_func(
    lambda cls: getattr(cls, "_serializable", False),
    lambda data, t: t._deserialize(data),
)

conv.register_unstructure_hook_func(
    lambda cls: getattr(cls, "_serializable", False),
    lambda t: t._serialize(),
)

where I write _serialize and _deserialize methods for my custom classes. In the _deserialize I essentially need something like ret = cls(**data), but then I need to "manually" structure the separate attributes that are not structured as shown above, so somethint like

@classmethod
def _deserialize(cls, data):
    ret = cls(**data)
    ret.as_ = conv.structure(data["as_"], list[A])
    ret.a = conv.structure(data["a"], A)
    return ret

I am not sure if this is the ideal way to do this, but any feedback would be very welcome :)

[Tinche]

Hi, I'm the author of cattrs and can help you. But can you explain what exactly are you trying to do? Your A and B classes will work with cattrs out of the box without any customization.

[jhossbach]

Hey @Tinche, thanks a lot for the offer!
I realized the problem is a bit more complicated than originally proposed, but I think my problem essentially solves itself if I can tell cattrs to ignore attributes, e.g. if they have a leading underscore:

import attrs
import h5py
from tempfile import TemporaryFile

@attrs.define
class Experiment:
    name:  str
    _file: h5py.File = attrs.field(init=False, repr=False)
    
    def __attrs_post_init__(self):
        self._file = h5py.File(TemporaryFile(), "w")

ex1 = Experiment(name="ex1")

conv.unstructure(ds)
# Returns {'name': 'ex1', '_file': <HDF5 file "<_io.BufferedRandom name=60>" (mode r+)>}

I can avoided this problem through the _serialize and _deserialize methods, but while this works I would be happy with an alternative that doesn`t require me to add these functions every time I have non-serializable attributes.

[Tinche]

So if I understand correctly, you'd like to ignore certain attributes for de/serialization?

Here's a simple way to do it, albeit a bit manual:

from tempfile import TemporaryFile

import attrs
import h5py

from cattrs import Converter
from cattrs.gen import make_dict_unstructure_fn, override


@attrs.define
class Experiment:
    name: str
    _file: h5py.File = attrs.field(
        repr=False, factory=lambda: h5py.File(TemporaryFile(), "w")
    )


ex1 = Experiment(name="ex1")

conv = Converter()
conv.register_unstructure_hook(
    Experiment, make_dict_unstructure_fn(Experiment, conv, _file=override(omit=True))
)

print(conv.unstructure(ex1))  # {'name': 'ex1'}

(See here for more details.)

There are other ways of doing this that are a little more complex and a little more generic. If this isn't good enough, we can get into those.

[jhossbach]

I had also considered this, but I like the idea of each class having its own deserialize method if necessary. That being said, I was able to solve my problem by changing the classes with problems with attrs classes (I didn't realize that they were not dataclasses).
My other two cents: when doing SQLAlchemy queries the result will be of type InstrumentedList which messes around when structuring/unstructuring with cattrs (one of the problems for me).
Thanks alot for the helpful answers!