Deserialization doesn't work with dynamically created dataclasses

[omaxx]

This code works:

@dataclass
class User(DataClassDictMixin):
    name: str
    pw: str

@dataclass
class Device(DataClassDictMixin):
    name: str
    user: User

device = Device.from_dict({"name": "device", "user": {"name": "jon", "pw": "snow"}})

and this code doesn't:

@dataclass
class Device(DataClassDictMixin):
    name: str
    user: make_dataclass('User', [
        ("name", str),
        ("pw", str),
    ], bases=(DataClassDictMixin,))

device = Device.from_dict({"name": "device", "user": {"name": "jon", "pw": "snow"}})

Error:

In [3]: device = Device.from_dict({"name": "device", "user": {"name": "jon", "pw": "snow"}})
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File <string>:19, in from_dict(cls, d)

AttributeError: module 'types' has no attribute 'User'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
Cell In[3], line 1
----> 1 device = Device.from_dict({"name": "device", "user": {"name": "jon", "pw": "snow"}})

File <string>:21, in from_dict(cls, d)

AttributeError: module 'types' has no attribute 'User'

[Fatal1ty]

Hi @omaxx

There are two problems here, but they are solvable. Fortunately for us.

Firstly, a class created using make_dataclass doesn't contain correct information about the module in which it was created until you specify it in the namespace parameter. This is what you see in the error message:

AttributeError: module 'types' has no attribute 'User'

To fix this you should pass the correct module name which is the module where your Device class exists:

make_dataclass(..., namespace={"__module__": __name__, ...})

Secondly, to deserialize an object of the User class, we need to have something like a reference to this User class, since we need to call its from_dict method. The qualified name is exactly this reference. But by default make_dataclass creates an attribute __qualname__ equal to the class name, well, "User". The namespace of the module doesn't have this name, so you can't call something like your.module.User.from_dict. To fix this you should specify the correct qualified name in the namespace parameter. This part is tricky. In your example the User class isn't located in the namespace of the Device class, because it's created in the value of the "user" field. If it's really what you want, qualified name of such a class can be obtained with the following code:

f'{__qualname__}.__annotations__["user"]'

To summarize, your example needs to be modified as follows to make it work:

@dataclass
class Device(DataClassDictMixin):
    name: str
    user: make_dataclass(
        'User',
        [
            ("name", str),
            ("pw", str),
        ],
        namespace={
            "__module__": __name__,
            "__qualname__": f'{__qualname__}.__annotations__["user"]',
        },
        bases=(DataClassDictMixin,)
    )

device = Device.from_dict({"name": "device", "user": {"name": "jon", "pw": "snow"}})
print(device)
# Device(name='device', user=Device.__annotations__["user"](name='jon', pw='snow'))
print(device.to_dict())
# {'name': 'device', 'user': {'name': 'jon', 'pw': 'snow'}}

If it's not essential for you to create the User class directly in the field value, then you can move this operation to the module level. In this case, the code will be simplified:

User = make_dataclass(
        'User',
        [
            ("name", str),
            ("pw", str),
        ],
        namespace={
            "__module__": __name__,
        },
        bases=(DataClassDictMixin,)
    )
@dataclass
class Device(DataClassDictMixin):
    name: str
    user: User

device = Device.from_dict({"name": "device", "user": {"name": "jon", "pw": "snow"}})
print(device)
# Device(name='device', user=User(name='jon', pw='snow'))
print(device.to_dict())
# {'name': 'device', 'user': {'name': 'jon', 'pw': 'snow'}}

Well, after all this investigation, I would like to finally know what are you doing if you need to create dataclasses in such a non-standard way? :)

[omaxx]

Hi @Fatal1ty

I only try to figure out how is it possible to define dataclass with hierarchical structure in the simplest way.

[Fatal1ty]

Hi @Fatal1ty

I only try to figure out how is it possible to define dataclass with hierarchical structure in the simplest way.

What do you mean by hierarchical structure? Using make_dataclass can hardly be called the simplest solution. I can help you find the better way if you elaborate more on what you are trying to achieve.

[omaxx]

Under "hierarchical structure" I mean case when dataclass has fields with type of another dataclass, and that dataclass also could have dataclass fields. Defining field dataclass type inside fields for me looks more readable.

@dataclass
class Device(DataClassDictMixin):
    name: str
    user: make_dataclass('User', [
        ("name", str),
        ("pw", str),
    ], bases=(DataClassDictMixin,))

vs

@dataclass
class User(DataClassDictMixin):
    name: str
    pw: str

@dataclass
class Device(DataClassDictMixin):
    name: str
    user: User

[Fatal1ty]

Ah, I see. What about this variant?

@dataclass
class Device(DataClassDictMixin):
    name: str
    user: "User"

    @dataclass
    class User(DataClassDictMixin):
        name: str
        credentials: "Credentials"

        @dataclass
        class Credentials(DataClassDictMixin):
            login: str
            pw: str

data = {
    "name": "device",
    "user": {
        "name": "jon",
        "credentials": {"login": "john", "pw": "snow"}
    }
}
device = Device.from_dict(data)
# Device(
#     name='device',
#     user=User(
#         name='john',
#         credentials=Device.User.Credentials(
#             login='john',
#             pw='snow'
#         )
#     )
# )
assert device.to_dict() == data

[Fatal1ty]

Firstly, a class created using make_dataclass doesn't contain correct information about the module in which it was created until you specify it in the namespace parameter. This is what you see in the error message:

AttributeError: module 'types' has no attribute 'User'

To fix this you should pass the correct module name which is the module where your Device class exists:

make_dataclass(..., namespace={"__module__": __name__, ...})

Meanwhile, incorrect __module__ attribute is apparently going to be fixed in python/cpython#102103

[omaxx]

Ah, I see. What about this variant?

@dataclass
class Device(DataClassDictMixin):
    name: str
    user: "User"

    @dataclass
    class User(DataClassDictMixin):
        name: str
        credentials: "Credentials"

        @dataclass
        class Credentials(DataClassDictMixin):
            login: str
            pw: str

Thank you! This variant is better!

[Fatal1ty]

Ok, good to hear that. If there is another problem, open an issue. I’m closing this one.