Field hooks are too clunky with Python 3.10 / string annotations

[hynek]

Currently I had to exclude test_hooks.py on Python 3.10, because it would be too painful to make them work with string annotations.

The problem is that there is no good helper to transform "str" into str. We're supposed to be able to eval() the string.

I've talked to @ambv and he suggested the following:

If faced with a string, first do eval(str, globalns, localns). Then on any type you get, do typing._eval_type(type, globalns, localns) which will deal with recursed ForwardRefs.

e.g.

>>> class G(Generic[T]):
...   def method(self) -> 'List["T"]':
...     return 1
...
>>> G.method.__annotations__
{'return': 'List["T"]'}
>>> eval(G.method.__annotations__['return'], globals(), locals())
typing.List[ForwardRef('T')]
>>> return_type=_
>>> typing._eval_type(return_type, globals(), locals())
typing.List[~T]

typing._eval_type has been part of the stdlib since 3.5 and I don't expect it to go anywhere.

---

I'm not sure how to call that function. attr.resolve_type() sounds good but that's a bit too close to attr.resolve_types() so I'm open to better suggestions.

---

I'm not sure we can get it done by 21.1.0 but it should be on our short term agenda.

cc @sscherfke @euresti

[euresti]

Another option is to do

            import typing
            hints = typing.get_type_hints(cls)

The hints will have the evalled rhs of the colon

example: {'x': <class 'int'>, 'y': <class 'str'>}

You can't call attr.resolve_types because cls is not an attrs class yet.

[hynek]

What I mean is this:

>>> import typing
>>> typing.get_type_hints("str")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/hynek/.asdf/installs/python/3.7.9/Python.framework/Versions/3.7/lib/python3.7/typing.py", line 1005, in get_type_hints
    'or function.'.format(obj))
TypeError: 'str' is not a module, class, method, or function.

There is just no good way for this.

[euresti]

I meant do something like this:

         def hook(cls, attribs):
            import typing
            hints = typing.get_type_hints(cls)
            results[:] = [(a.name, hints.get(a.name, a.type)) for a in attribs]
            return attribs

Sadly we can't just call typing.get_type_hints whenever because of forward references. But this would fix the tests.

[hynek]

Ah right, you're correct as usual.

I tend to think it would be nice to have helpers for ergonomics, but less APIs is better for now of course so I guess we can fix it and add docs for people who run into the same problem.

[hynek]

Hm so fun fact, it has to be even more complicated due to the various ways of defining types:

        def hook(cls, attribs):
            hints = get_type_hints(cls)
            results[:] = [
                (a.name, hints.get(a.name) or a.type) for a in attribs
            ]
            return attribs

[euresti]

I'm not sure about the or. What is returned if the annotation is x : None? I guess it doesn't matter much.

If you really wanted an API the only one I came up with was:

def resolve_attribs(cls, attribs):
   hints = get_type_hints(cls)
   for a in attribs:
        a.type = hints.get(a.name) or a.type

[hynek]

attribs are on cls, so shouldn't the following work:

def resolve_fields(cls):
    fields = attr.fields(cls)
    hints = get_type_hints(cls)

    for f in attr.fields(cls):
        t = hints.get(f.name)
        if t is not None:
            f.type = t

    return fields

? 🤔

(I'm trying to use "fields" in new APIs in anticipation of import attrs)

[euresti]

At the moment the hook is run cls is not an attrs class yet so attr.fields fails with NotAnAttrsClassError

[hynek]

hmm, it seems to make sense to allow for both cases, would you agree?

i.e. resolve_fields(cls, fields=None) and if it stays None, we look them up ourselves?

[euresti]

Hmm. We can probably just build it into resolve_types.

def resolve_types(cls, fields_=None, globalns=None, localns=None):
    try:
        # Since calling get_type_hints is expensive we cache whether we've
        # done it already.
        cls.__attrs_types_resolved__
    except AttributeError:
        import typing

        hints = typing.get_type_hints(cls, globalns=globalns, localns=localns)
        for field in fields(cls) if fields_ is None else fields_:
            if field.name in hints:
                # Since fields have been frozen we must work around it. 
                _obj_setattr(field, "type", hints[field.name])
        cls.__attrs_types_resolved__ = True.  # <---- Don't know if we should do this yet.

    # Return the class so you can use it as a decorator too.
    return cls

Or we could add a new method.

[hynek]

I would love to have it part of the old API, but how big is the risk of this breaking something?

[hynek]

fixed by #774 🎉