Using several factories simultaneously

[AdrianSosic]

Hi @Tinche, my previous issue was basically pointless but I'm now getting closer to the heart of my problem. While I think I understand how regular hooks can be composed, it's not quite clear to me how somethign similar can be achieved using predicates/factories.

In essence, it would help me to better understand how hooks/predicates/factories interplay and how it is possible to "activate" certain customizations at once. The problem that I get with factories is that only ever one of them is active. Here a little example what I mean:

import cattrs
from attrs import define

converter = cattrs.Converter()


@define
class Base:
    x: int = 0


@define
class Sub(Base):
    y: int = 1


@define
class SubSub(Sub):
    z: int = 1


@define
class Container:
    contains: Base


@converter.register_unstructure_hook
def some_special_configuration(obj: SubSub) -> dict:
    fn = cattrs.gen.make_dict_unstructure_fn(
        SubSub, converter, x=cattrs.override(rename="renamed")
    )
    return fn(obj)


@converter.register_unstructure_hook_factory(lambda c: c is Base)
def add_type(_):
    def hook(obj):
        hook = cattrs.gen.make_dict_unstructure_fn(type(obj), converter)
        return {"type": obj.__class__.__name__, **hook(obj)}

    return hook


@converter.register_unstructure_hook_factory(lambda c: issubclass(c, Sub))
def indicate_if_sub(_):
    def hook(obj):
        hook = cattrs.gen.make_dict_unstructure_fn(type(obj), converter)
        return {"is_sub": obj.__class__.__name__, **hook(obj)}

    return hook


contains_base = Container(Base())
contains_sub = Container(Sub())
contains_subsub = Container(SubSub())

print(converter.unstructure(contains_base))
print(converter.unstructure(contains_sub))
print(converter.unstructure(contains_subsub))

Gives:

{'contains': {'type': 'Base', 'x': 0}}
{'contains': {'type': 'Sub', 'x': 0, 'y': 1}}
{'contains': {'type': 'SubSub', 'x': 0, 'y': 1, 'z': 1}}

How can I change the logic such that all rules are "active" at the same time, that is:

• Because everything is unstructured as Base, the type field should be always present.
• Both sub and subsub are of type Sub, meaning I want to get their additional is_sub field.
• Finally, subsub has a special renaming that should be considered.

So what I want is rather:

{'contains': {'type': 'Base', 'x': 0}}
{'contains': {'type': 'Sub', 'is_sub': 'Sub', 'x': 0, 'y': 1}}
{'contains': {'type': 'SubSub', 'is_sub': 'SubSub', 'renamed': 0, 'y': 1, 'z': 1}}

I'd really appreciate if you could point me to the right track 🙃

[AdrianSosic]

Maybe to add: What I would ideally like to do, but which leads me into endless recursion I don't know how to avoid, is to replace the make_dict_unstructure_fn with get_unstructure_hook like:

@converter.register_unstructure_hook_factory(lambda c: c is Base)
def add_type(_):
    def hook(obj):
        fn = converter.get_unstructure_hook(type(obj))
        return {"type": obj.__class__.__name__, **fn(obj)}

    return hook


@converter.register_unstructure_hook_factory(lambda c: issubclass(c, Sub))
def indicate_if_sub(_):
    def hook(obj):
        fn = converter.get_unstructure_hook(type(obj))
        return {"is_sub": obj.__class__.__name__, **fn(obj)}

    return hook

[Tinche]

In your example, cattrs will look for the hook for the Base type, since that's the annotation used in Container. Then the problem becomes combining several hooks into that one hook, which is something cattrs can't do currently, so you have to do it yourself.

Your approach in #659 (comment) won't work, since indicate_if_sub will never be used with your class hierarchy.

The easiest would be to stick all the logic into the hook factory for Base:

@converter.register_unstructure_hook_factory(lambda c: issubclass(c, Base))
def handle_base(type):
    fn = make_dict_unstructure_fn(type, converter)

    def hook(obj: Base) -> dict:
        res = {"type": obj.__class__.__name__, **fn(obj)}

        if isinstance(obj, Sub):
            res["is_sub"] = obj.__class__.__name__

        return res

    return hook

If you want them separate, you will probably have to build your own dispatch mechanism and use it in handle_base. Something like this:

def add_sub(object: Sub, payload: dict):
    payload["is_sub"] = object.__class__.__name__


hooks = [(lambda t: issubclass(t, Sub), add_sub)]


@converter.register_unstructure_hook_factory(lambda c: issubclass(c, Base))
def handle_base(type):
    fn = make_dict_unstructure_fn(type, converter)

    def hook(obj: Base) -> dict:
        res = {"type": obj.__class__.__name__, **fn(obj)}

        for pred, handler in hooks:
            if pred(obj.__class__):
                handler(obj, res)

        return res

    return hook

Cattrs is built to avoid this kind of logic at unstructure time, doing most of the work at hook creation time. This is a little at odds with how you're trying to use it here, so you have to do the work yourself.

There is another potential approach with making Container generic, and then explicitly doing converter.unstructure(contains_sub, unstructure_as=Container[Sub]).

[AdrianSosic]

Hi @Tinche, as always, thank you very much for taking the time to provide such a detailed answer. I've already expected a situation like this, and the manual dispatching approach you describe seems like a viable workaround, indeed 👏🏼

The other two suggestions won't work in my case, though:

• Assembling everything in the base hook directly is suboptimal, as you probably expected, because I wouldn't want to couple the logic of that one very special subclass into the main logic of the base class but rather keep it isolated in the right submodule.
• Also, the approach via generics is not a viable option in my case since the subclass type is only known at runtime.

So while we can consider this issue closed, one related follow-up question. Here, you mentioned the idea of injecting PreviousHook as additional optional argument into the factories. Of course, this mechanism does not yet exist, but wouldn't this offer a "proper" solution to my problem?

[Tinche]

I might try implementing a PrevHook thingy in cattrs next, to see how easy it would be.