Regression of unstructuring typing.Any based on runtime types

[xaviergmail]

• cattrs version: 22.2.0
• Python version: 3.9
• Operating System: macOS / Linux

Description

There is a regression in unstructuring of statically typed Any field in versions published after 1.1.2. It returns the value as-is, regardless of whether the runtime type is an attrs class.

What I Did

Using the following code:

try:
    from attrs import define
    try:
        from cattrs.converters import BaseConverter, GenConverter
    except ImportError:
        from cattrs.converters import Converter as BaseConverter
        from cattrs.converters import GenConverter
except ImportError:
    from attr import define
    from cattr.converters import Converter as BaseConverter, GenConverter


bc = BaseConverter()
gc = GenConverter()


@define
class Container:
    many: ty.List[ty.Any]
    one: ty.Any


@define
class Thing:
    field: int


c = Container(many=[Thing(field=1)], one=Thing(field=2))

print("BaseConverter", bc.unstructure(c))
print("GenConverter", gc.unstructure(c))

I ran the following:

cattrs 1.1.2

List[Any] and Any get unstructured fine.

❯ pip install cattrs==1.1.2 --force-reinstall &> /dev/null && pip show cattrs | grep Version && python thing.py
Version: 1.1.2
BaseConverter {'many': [{'field': 1}], 'one': {'field': 2}}
GenConverter {'many': [{'field': 1}], 'one': {'field': 2}}

cattrs 1.2.0

List[Any] gets unstructured but not Any

❯ pip install cattrs==1.2.0 --force-reinstall &> /dev/null && pip show cattrs | grep Version && python thing.py
Version: 1.2.0
BaseConverter {'many': [{'field': 1}], 'one': Thing(field=2)}
GenConverter {'many': [{'field': 1}], 'one': Thing(field=2)}

cattrs 1.4.0 onwards

GenConverter no longer unstructures Any fields
BaseConverter retained the inconsistent 1.2.0 behavior

❯ pip install cattrs==1.4.0 --force-reinstall &> /dev/null && pip show cattrs | grep Version && python thing.py
Version: 1.4.0
BaseConverter {'many': [{'field': 1}], 'one': Thing(field=2)}
GenConverter {'many': [Thing(field=1)], 'one': Thing(field=2)}

The behavior then stays consistent from 1.4.0 onward. We are still slowly updating our application code to use our internal libraries which are now using cattrs 22.2.0, which is how I came across this. We have a very specific use case where using generics would be impractical; we're just now moving from 3.7 to 3.9 and therefore can't use 3.11's variadic generics and defining a handful of TypeVars would be unsightly.

[Tinche]

Hi,

short answer: you can override the hook for Any like this to get the behavior you want:

import typing as ty

from attrs import define

from cattrs import GenConverter

gc = GenConverter()


@define
class Container:
    many: ty.List[ty.Any]
    one: ty.Any


@define
class Thing:
    field: int


c = Container(many=[Thing(field=1)], one=Thing(field=2))


def unstructure(val):
    return gc.unstructure(val, unstructure_as=val.__class__)


gc.register_unstructure_hook_func(lambda t: t is ty.Any, unstructure)


print("GenConverter", gc.unstructure(c))

Since Any is basically a passthrough when structuring, I think it makes sense for it to also be just a passthrough when unstructuring.

[jwsloan]

Any as passthrough when structuring is really the only option. It would be very surprising to do anything else.

That is not the case when unstructuring, though. At that point, you have the runtime type available. You could choose to unstructure that object, as your example shows. It is surprising behavior to not unstructure when asked to, when that is a straightforward possibility.