Adding keys to unstructure dict

[isohedronpipeline]

• cattrs version: 24.2.0
• Python version: 3.10
• Operating System: Windows

Description

I have an attrs class with a bunch of subclasses. These attrs dataclasses point to other attrs dataclasses.
I would like to take control of the way it unstructures myself, such that the resulting dictionary has an extra key.

I am specifically looking to add a fully dotted path to the class so that I can use pydoc.locate to find the class instead of relying on the cattrs subclass scheme, which should allow me to avoid importing everything first and avoid a problem with subclasses that happen to be named the same.

If the data type was a "leaf" then I could simply call

result = attrs.asdict(obj)
result["_type_path"] = get_fully_dotted_path_to_class(obj.__class__)

But since the class has a container of other types that need unstructure hooks, I can't use attrs.asdict(). I need to use the unstructuriing dispatcher.

What I Did

I tried to do this:

def unstructure_job(obj: Any) -> Any:
    result = converter._unstructure_func.dispatch(obj.__class__)(obj)
    result['_type_path'] = get_dotted_path_to_class(obj.__class__)
    return result

But that ends up with an infinite recursion.

How can I hop in the middle to add a new key?

Thanks!

[Tinche]

Hello,

you can use a hook factory that delegates to cattrs.gen.make_dict_unstructure_fn and wraps the resulting hook. Like this:

from typing import Any, Callable

from attrs import define

from cattrs import Converter
from cattrs.gen import make_dict_unstructure_fn

converter = Converter()


def get_dotted_path_to_class(cls):
    return "test"


@define
class RootClass:
    pass


@define
class Subclass(RootClass):
    a: int


@converter.register_unstructure_hook_factory(lambda t: issubclass(t, RootClass))
def unstructure_factory(cls: type[RootClass], converter: Converter) -> Callable:
    base_hook = make_dict_unstructure_fn(cls, converter)

    def unstructure_job(obj: Any) -> dict:
        result = base_hook(obj)
        result["_type_path"] = get_dotted_path_to_class(obj.__class__)
        return result

    return unstructure_job


print(converter.unstructure(Subclass(1)))

This pattern can be used to apply and pre- or post-processing to hooks you need! Let me know if it works for you.

[isohedronpipeline]

Amazing! Thanks Tin.
Maybe I'm doing something wrong, but I'm getting this error: TypeError: BaseConverter.register_unstructure_hook_factory() missing 1 required positional argument: 'factory' when using register_unstructure_hook_factory as a decorator.

I'm using cattrs 23.1.2, due to a requirement for python 3.7 unfortunately, so maybe the interface changed a little for that.

This worked well though:

def _unstructure_job_factory(cls: type[Job], _converter: cattrs.Converter) -> Callable:
    base_hook = make_dict_unstructure_fn(cls, _converter)

    def _unstructure_job(obj: Any) -> dict:
        result = base_hook(obj)
        result["_type_path"] = get_dotted_path_to_class(obj.__class__)
        return result

    return _unstructure_job


converter.register_unstructure_hook_factory(
    predicate=lambda t: issubclass(t, Job), 
    factory=lambda cl: _unstructure_job_factory(cl, converter),
)

[isohedronpipeline]

Sorry, I may have spoken too soon.
The unstructuring works, but I may need an example for how to structure it again.
Looking at make_dict_structure_fn, I don't see how to inspect the serialized keys to decide which class to restructure.

e.g. if i have:

{
    "name": "[EchoJob] 2648505075072",
    "id": "77db1a3c688d4248b984d81fd247e908",
    "_type_path": "path.to.job_types.EchoJob"
}

How can I change the restructuring class based on the value of _type_path?

[isohedronpipeline]

Nevermind!
Sorted it out.

I'm not sure what the second argument to the callable returned by make_dict_structure_fn is supposed to be, but passing in None seemed to still work.

def _unstructure_job_factory(cls: type[Job], _converter: cattrs.Converter) -> Callable:
    def _unstructure(obj: Any) -> dict[str, Any]:
        _cls = obj.__class__
        base_hook = cattrs.gen.make_dict_unstructure_fn(_cls, _converter)
        result = base_hook(obj)
        result["_type_path"] = fully_qualified_class_path(_cls)
        return result
    return _unstructure

def _structure_job_factory(cls: type[Job], _converter: cattrs.Converter) -> Callable:
    def _structure(d: dict[str, Any], _: type) -> Any:
        _cls_path = d.pop("_type_path")
        _cls = pydoc.locate(_cls_path)
        base_hook = cattrs.gen.make_dict_structure_fn(_cls, _converter)
        result = base_hook(d, None)
        return result
    return _structure

converter.register_unstructure_hook_factory(
    predicate=lambda t: issubclass(t, Job),
    factory=lambda _cls: _unstructure_job_factory(_cls, converter),
)        
converter.register_structure_hook_factory(
    predicate=lambda t: issubclass(t, Job),
    factory=lambda _cls: _structure_job_factory(_cls, converter),
)            

Thanks again, Tin!

[Tinche]

Great!

Maybe I'm doing something wrong, but I'm getting this error: TypeError: BaseConverter.register_unstructure_hook_factory() missing 1 required positional argument: 'factory' when using register_unstructure_hook_factory as a decorator.

Oh yeah, these are new in 24.1.

I'm not sure what the second argument to the callable returned by make_dict_structure_fn is supposed to be

It's supposed to be the type that's being structured (as for all structure hooks), but many hooks produced by factories ignore this parameter since they're not using it. Don't worry about it here.

Glad you solved it, closing this now to trim the issues page.