[Feature Request] Postponed evaluation of non-self-referential type hints

[euanong]

Thanks for developing a really cool library :)

Just one issue I found when using forward references with beartype: the following code snippets (which statically typecheck with MyPy) seem to throw BearType exceptions:

from dataclasses import dataclass
from typing import List, TypeVar
from beartype import beartype

@beartype
def my_fn(xs: "List[MyCls]"):
    return xs[0]

@dataclass
class MyCls:
    value: int

throws

BeartypeDecorHintForwardRefException: Forward reference "__main__.List[MyCls]" syntactically invalid as module attribute name.

and

from dataclasses import dataclass
from typing import List, TypeVar, Generic
from beartype import beartype

T = TypeVar("T")

@beartype
def my_fn(x: "MyCls[int]"):
    return x.value

@dataclass
class MyCls(Generic[T]):
    value: T

throws

BeartypeDecorHintForwardRefException: Forward reference "__main__.MyCls[int]" syntactically invalid as module attribute name.

(P.S. do you have an ETA on this feature #43 or is there a hacky workaround I could use for the time being?)

[leycec]

Thanks so much for both your generous interest and this perfect issue report, @eohomegrownapps. Smiley cat is most pleased. 😸

Does @beartype Suck As Much As I Think?

Sadly, you are indeed correct; forward references are our Achilles heel.

It's trivial for a static type-checker like mypy to resolve forward references, because static type-checkers don't actually run anything; they just parse. It's comparably harder for a runtime type-checker like @beartype to resolve non-trivial forward references like this, because @beartype needs to dynamically generate type-checking Python code at runtime for attributes that have yet to be defined. In this case, the stringified type hint "List[MyCls]" references:

• An attribute List that already has been defined. No problem, @beartype!
• An attribute MyCls that has yet to be defined. This is problem, @beartype.

How Can I Just Make This Work Now?

Thankfully, @beartype already basically supports exactly what you want. Just replace the string "List[MyCls]" in the above example with List["MyCls"]. That is to say, stringify only the exact class that has yet to be defined: e.g.,

from beartype import beartype
from beartype.typing import List
from beartype.vale import Is
from dataclasses import dataclass

@beartype
def my_fn(xs: List['MyCls']):
    return xs[0]

@dataclass
class MyCls:
    value: int

babes_everywhere = MyCls(value=0xCAFEBABE)
assert my_fn([babes_everywhere]) is babes_everywhere

Mypy and @beartype alike should now both be happy. It's a party and you're invited.

But My Second Example Is Still Busted, Right?

...yeah. Sadly, the same approach fails to extend to your second example.

Happily, that example can be rewritten to leverage the standard typing.TYPE_CHECKING type hint. typing.TYPE_CHECKING is the Ultimate Power™. When you want to do both static and runtime type-checking, typing.TYPE_CHECKING lets you conditionally create different type hints at static and runtime: e.g.,

from dataclasses import dataclass
from beartype import beartype
from beartype.typing import TYPE_CHECKING, Annotated, Generic, TypeVar

T = TypeVar("T")

# If a static type-checker like mypy is currently running, prefer a
# standard forward reference.
if TYPE_CHECKING:
    MyClsInt = 'MyCls[int]'
# Else, a runtime type-checker like @beartype is currently running.
# In this case, fallback to a simple reference to "MyCls". Since
# user-defined subscripted generics (e.g., the "[int]" in "MyCls[int]")
# convey no meaningful semantics at runtime, @beartype would just
# silently ignore them anyway. Help a @beartype out, bro!
else:
    MyClsInt = 'MyCls'

@beartype
def my_fn(x: MyClsInt):
    return x.value

@dataclass
class MyCls(Generic[T]):
    value: T

assert my_fn(MyCls(value=0xCAFEBABE)) == 0xCAFEBABE

In this case, we leverage typing.TYPE_CHECKING to:

• Preserve your desired MyCls[int] for analysis by mypy.
• Reduce that to simply MyCls for analysis by @beartype. Nothing is lost, since @beartype already internally ignores child type hints like [int] on user-defined subscripted generics. They convey no meaningful semantics at runtime. Right? Right!

How Do We Help @beartype Suck Less?

What follows is mostly notes to myself. I will forget this. Now, I won't. Checkmate, ageing brain. 🧠

To resolve this, @beartype will need to (in order):

1. Internally implement a full-blown forward reference parser – probably using precompiled (but still slow) regular expressions to parse stringified type hints like "List[MyCls]" into their constituent Python identifiers (e.g., "List", "MyCls").
2. For each such identifier:
   1. Decide whether an attribute with that name has already been defined.
   2. If already defined like "List", no problem!
   3. If undefined like "MyCls", problem! Fix this by globally replacing in that type hint all instances of that name with __beartypistry[{name}] (e.g., __beartypistry['MyCls'] in this case). As that code snippet suggests, __beartypistry is a @beartype-specific caching dictionary that drives our existing forward reference implementation.

I am shuddering already.

When Will You Do This For Everybody?

Why, I'm so glad you asked. We'll probably nail this to floor during our upcoming beartype 0.14 release cycle – to be released sometime later this Summer, surely. Would @leycec lie? 😓

[TeamSpen210]

Instead of writing the parser manually, there's a simpler approach, somewhat described here for PEP 649 (Deferred Evaluation Of Annotations Using Descriptors). Instead of executing the annotation normally, instead a custom dict subclass could be passed as the locals dictionary. That allows intercepting all namespace lookups. So we could do a normal lookup of globals/locals, but if a lookup fails, create and inject a proxy object which has all the annotation-relevant special methods defined. Or even unconditionally return a proxy for every name, perhaps to construct a tree of door objects instead.

[euanong]

@leycec ... wow. that probably wins the award for most characterful (and fastest) github issue response I've ever encountered 😸

Unfortunately, I'm not sure how scalable this temporary fix will be, considering the codebase I'm developing is a type-based abomination (I miss typeclasses ;-;) with very extensive use of (nested) forward references... but great to know this will be patched eventually :)

Nothing is lost, since https://github.com/beartype already internally ignores child type hints like [int] on user-defined subscripted generics. They convey no meaningful semantics at runtime. Right? Right!

This is a slightly off-topic thing, but it would be really awesome to have beartype inspect user-defined subscripted generics to determine (in most cases, with high probability) whether they typecheck... I imagine, though, this would require a little too much python hackery / performance cost of tracking typevars to be palatable (?)

(P.S. do you have an ETA on this feature #43 or is there a hacky workaround I could use for the time being?)

(Also, this was referring to issue #43, not this feature :) )

[leycec]

Unfortunately, I'm not sure how scalable this temporary fix will be...

Yup. You're too kind, @eohomegrownapps. Let's admit that everything I wrote is totally anti-scalable, because it puts the burden of type-checking forward references almost entirely on you: the hard-working developer. That's awful! I acknowledge this is badness incarnate.

@beartype should address this as soon as humanly feasible. I promise the Summer will bring blue skies, empty beaches, and a @beartype that actually works. 🌞

it would be really awesome to have @beartype inspect user-defined subscripted generics to determine (in most cases, with high probability) whether they typecheck...

Yes! I'd love to have that, too. Sadly...

I imagine, though, this would require a little too much python hackery / performance cost of tracking typevars to be palatable (?)

...heh. That's the concern, really. To do this sanely, @beartype will need to begin injecting @beartype-specific instance variables into each instance of a @beartype-decorated generic. Exactly as you suggest, these variables would then track all prior TypeVar associations across the lifetime of that instance. For instance, ^{hyuk, hyuk} @beartype might inject:

• A new __beartype_typevar_to_type dictionary into each object dictionary (i.e., obj.__dict__). __beartype_typevar_to_type would map:
  • From each TypeVar object subscripting that generic (e.g., the T = TypeVar('T') in class MyCls(T))...
  • To either:
    • If a corresponding type has already been associated with that TypeVar of that generic (e.g., the int in my_instance = MyCls[int]), that type.
    • Else, None.

@beartype will then need to dynamically generic type-checking that looks up types in the __beartype_typevar_to_type dictionary.

Clearly, that's tractable. Well, probably. Given sufficient volunteer man-hours, @beartype can definitely do that – probably. Yeah! Clearly, however, that's also non-trivial. This will definitely happen – eventually. But "eventually" might be a very long wait indeed. Runtime type-checking of generic type variables isn't even something that anyone thought was possible before. PEP 484 literally instructs runtime type-checkers to just ignore type variables.

Scaredy emoji cat is distraught. 🙀

[leycec]

@TeamSpen210: OMG. You're a friggin' genius, Spence! Not only that, but... you're friggin' alive. Miraculously, you made it through another gruelling winter. I thought you were a goner for sure. It's been a literal lifetime since we've heard from you here. Spring, sunshine, and good fortune are surely smiling upon @beartype today.

Okay. Back to brass tacks. This...

Instead of writing the parser manually, there's a simpler approach, somewhat described here for PEP 649 (Deferred Evaluation Of Annotations Using Descriptors). Instead of executing the annotation normally, instead a custom dict subclass could be passed as the locals dictionary. That allows intercepting all namespace lookups. So we could do a normal lookup of globals/locals, but if a lookup fails, create and inject a proxy object which has all the annotation-relevant special methods defined. Or even unconditionally return a proxy for every name, perhaps to construct a tree of door objects instead.

...is all stupendous. You're absolutely right about everything, like usual. That's the aikido throw I've been searching for. Writing some ad-hoc hand-rolled slow-as-molasses fragile-as-bone-china parser would be terrible. You have saved me from a terrifying fate of darkness.

As additional notes to myself:

• Let us define these proxy objects as instances of a new BeartypeForwardRef class in a new beartype._decor._cache.cacheref submodule.
• The definition of BeartypeForwardRef should resemble:

from beartype._decor._cache.cachetype import bear_typistry

class _BeartypeForwardRefMeta(type):
    def __instancecheck__(cls: BeartypeForwardRef, obj: object) -> bool:
        '''
        *BOOM.*
        '''

        cls_resolved: type = cls.resolve()
        return isinstance(obj, cls_resolved)


class BeartypeForwardRef(object, metaclass=_BeartypeForwardRefMeta):
    def __init__(self, forwardref: str) -> None:
        #FIXME: Do some preliminary validation of this string for sanity.
        #Notably, validate that this string is a valid Python identifier.
        self.forwardref = forwardref


    def resolve(self) -> type:
        '''
        Resolve this forward reference to the type it refers to.
        '''

        # Good enough for now. Our existing "bear_typistry"
        # dictionary already handles lookup-based resolution and
        # caching of forward references at runtime; so, just defer
        # to that for now as the trivial solution.
        #
        # Next!
        return bear_typistry[self.forwardref]


    def __class_getitem__(cls, *args: object): -> None:
        '''
        Record the passed child type hints subscripting this
        forward reference to a user-defined generic (e.g.,
        the :class:`int` in ``MuhGeneric[int]``).
        '''

        return _BeartypeForwardRefGenericSubscripted(
            forwardref=forwardref, args=args)


class _BeartypeForwardRefGenericSubscripted(BeartypeForwardRef):
    '''
    Forward reference to a user-defined generic subscripted
    by one or more child type hints (e.g., ``MuhGeneric[int]``).
    '''

    def __init__(self, forwardref: str, args: tuple) -> None:
        super().__init__(forwardref)

        #FIXME: Actually do something with subscripted child type hints.
        #For now, simply ignoring them outright is the sane thing to do.
        self.args = args

This is actually incredible. Thanks to metaclass __instancecheck__() shenanigans, now @beartype can just treat BeartypeForwardRef instances like standard types. @beartype no longer needs to treat forward references and standard types as fundamentally different things. If fortune favours the brave, then simple constraints like the following should now hold:

>>> isinstance(MyCls(), BeartypeForwardRef('MyCls'))
True
>>> isinstance(MyCls[int](), BeartypeForwardRef('MyCls')[int])
True

This should substantially streamline @beartype's code generation. Minds are being blown. 💥 🤯

[st--]

I'm encountering a similar issue, but in a different setup, where the exception is
beartype.roar.BeartypeDecorHintForwardRefException: Forward reference "gpjax.kernels.approximations.rff.Optional[List[int]]" syntactically invalid as module attribute name.
and the issue is a base class in one file

@dataclass
class AbstractKernel(Module):
    ...
    active_dims: Optional[List[int]] = static_field(None)

and a subclass in another file

@dataclass
class RFF(AbstractKernel):
    ...

which doesn't redefine active_dims. (If I copy&paste the definition from the base class around, this error is fixed, but of course that does away with all the conveniences of inheriting fields...)

Would it be helpful for you to have a MRE of this, too? (Asking first because I'm using beartype with jaxtyping and jaxtyping.install_import_hook, and Module and static_field are custom as well so it'll be a bit more work to disentangle it into a standalone example.)

[leycec]

Gah! Tremendous apologies for the woes that @beartype has brought, @st--. And might I add... what a username that is. I didn't even know you could dangerously suffix usernames by arbitrarily many hyphens. Your valiant attempts to break GitHub impress me.

I wonder why @beartype is failing you in this seemingly reasonable @dataclass use case, though. From the little that I can tell, there are no stringified type hints and thus no forward references in your example; there are only standard type hints (e.g., Optional[List[int]]). In fact, this painfully unhelpful MLE behaves as expected:

from beartype import beartype
from beartype.typing import Optional, List
from dataclasses import dataclass

@beartype
@dataclass
class AbstractKernel(object):
    active_dims: Optional[List[int]] = None

@beartype
@dataclass
class RFF(AbstractKernel):
    pass

I beg of you, @st--. Please shower us with a brilliant MLE that will stun me even further! Actually, would you mind just submitting a new issue? I suspect this is probably a unique issue specific to some combination of JAX, jaxtyping, jax-dataclasses, and/or PEP 563 (i.e., from __future__ import annotations).

There's a lot of third-party intensity going on here. I didn't even know that jaxtyping had its own import hook, for example. That's... possibly not good. I mean, that's good for jaxtyping. Clearly. But import hooks have a tendency to play poorly with other import hooks.

I'm currently banging on a native import hook for @beartype. It's intended to be the Ultimate-Super-Saiyan import hook, because it augments @beartype into the first hybrid static-runtime type-checker for any dynamic language. Think mypy, except it runs at runtime and lies a bit less.

Please find enclosed an accurate visual depiction of @beartype's upcoming import hook.

^{kinda afraid to ask what the end user codebase is in this suspicious metaphor}

[st--]

It's the from __future__ import annotations. MRE:

File mre/__init__.py:

from jaxtyping import install_import_hook

with install_import_hook("mre", "beartype.beartype"):
    from .base import *
    from .rff import *

File mre/base.py:

from __future__ import annotations

from beartype.typing import List, Optional
from dataclasses import dataclass

@dataclass
class AbstractKernel:
    """Base kernel class."""

    active_dims: Optional[List[int]] = None

File mre/rff.py:

#from __future__ import annotations
from dataclasses import dataclass

from .base import AbstractKernel

@dataclass
class RFF(AbstractKernel):
    pass

Then import mre gives the following error:

BeartypeDecorHintForwardRefException: Forward reference "mre2.rff.Optional[List[int]]" syntactically invalid as module attribute name.

If you comment out the from __future__ import annotations in the base.py, the error disappears.

I previously got this error:

BeartypePep563Exception: RFF.__init__() parent lexical scope RFF() not found on call stack.

when both modules had the from __future__ import annotations, but I stripped away too much and now that one I'm not sure what it was caused by...

(st-- is a legacy user name. You can't have trailing dashes anymore, and unfortunately one implication is that I can't have an st--.github.io GitHub pages. It also means I can't contribute to Google open-source packages such as TensorFlow (or JAX) because their CLA validator refuses to accept my username... How do you make the font size smaller?)

[leycec]

Ah-ha! So. The horrifying truth reveals itself at last. from __future__ import annotations, we meet again. The years pass. The seasons turn. The seas swell. Yet PEP 563 still sucks.

Thanks so much for that awesome MRE, @st--. Since I kinda need to push out a new beartype 0.13.2 release tonight, I lack volunteer time to run your MRE through the gauntlet like I should. But all's not for naught! I suspect I know how to make everything work again:

# This is painfully dumb, but sometimes miracles happen.
from __future__ import annotations  # <-- *THIS. TRY THIS.*

from jaxtyping import install_import_hook

with install_import_hook("mre", "beartype.beartype"):
    from .base import *
    from .rff import *

That's it. Just prepend mre/__init__.py by from __future__ import annotations. If a miracle erupts, I can explain why. Else, I hang my bald head and weep salty tears at midnight.

It saddens me that both GitHub and Google are united in their shared hatred for your username. It's fascinating that Google would rather reject volunteer contributions¹ than fix their clearly busted CLA validator. Like, how hard is it to just accept legacy usernames? It's not. We know. There's a poorly documented and thoroughly untested regex somewhere deep within the bowels of GitHub and Google. It alone is to blame for all the bad things in this world.

^{1. Volunteer contributions are the highest value contributions that you can get. They're even more valuable than paid employee contributions – because you had to pay employees to make them, which means they didn't really care in the first place. Code is where the heart is.}

I salute your busted username, @st--. Even if nobody else cares, @beartype cares. 🐻

[st--]

I've tried it out but I'm afraid adding the future annotations import to __init__.py doesn't fix the issue. Only adding it to all the files resolves that [if I can reproduce the error I got then, I'll share that], or removing it everywhere..

[leycec]

@st--: I summon thee to issue #233, where we will avoid bludgeoning these other hapless victims with unrelated pabulum that can only enrage and disappoint them.

We now return to our regularly scheduled issue.

[leycec]

Relatedly, @Asana typing giant @skeggse submitted this excellent minimal-reproducible example (MRE) exhibiting the same issue. Clearly, this is more mission-critical than I wanted to believe:

import abc
from typing import Type, TypeVar

from beartype import beartype

A = TypeVar("A", bound="Mixin")


class Mixin(abc.ABC):
    @classmethod
    def schema(cls: Type[A]) -> "SchemaType[A]":
        pass


def mix(cls):
    cls.schema = classmethod(Mixin.schema.__func__)
    Mixin.register(cls)
    return cls


@beartype
@mix
class Custom:
    pass

...which raises the expected bad exception:

beartype.roar.BeartypeDecorHintForwardRefException: Forward reference
"__main__.SchemaType[A]" syntactically invalid as module attribute name.

I've been delaying addressing this due to #226, which will either redefine the face of Python as we know it or catastrophically blow up in my face. Both of those things might even happen.

[leycec]

[leycec]

omg! OMG! OMGGGGG!!! @TeamSpen210: Your brilliant five-sentence abstract on the nature of Python vis-a-vis forward references actually worked! It actually worked, bro! Punch the air like you just don't care, 'cause that's what we're doin'.

Naturally, your "simpler approach" was horrifying. My once-beautific bald face is crinkled with wrinkles, my twitchy eye is even more twitchy, and our cat won't stop pacing the room. I spent the entirety of my vacation doing this – and I don't even need this in production code. It just became a titanic struggle of @leycec versus @beartype versus Python: a three-way untelevised cage match that could only end with one man hurtling fifty feet onto another man lying prostrate on a table perched on top of another man curled in the fetal position while bleeding out live on GitHub.

^{...dat the best u can do, python?}

^{wat is even happening here}

^{When your masterplan only ends in tragedy for everybody.}

Seriously. I've got five hundred line comments devsplaining single one-liners. I've got 100KB of incomprehensible pure-Python isolated to four new submodules sprawled across a new subpackage devoted exclusively to inchoate madness. I've got abstract base classes, abstract metaclasses, dynamic type factories, dunder methods on sunder methods on abstract cached properties. I've got friggin' metaclass __getattr__() methods performing deferred module attribute importation via dynamic class attribute lookups of attributes that don't actually physically exist anywhere. I've got two whole dict.__missing__() implementations that don't even really have anything to do with one another. This is inchoate madness.

I don't even know how any of this works anymore. I've already jettisoned all knowledge pertaining to this topic so I can just sleep tonight without fitfully crying out in the night for my childhood stuffed octopus. I wish to awaken from the real-world nightmare I have unwittingly exposed myself to. It turns out Bloodborne is more a state of mind rather than a mere video game.

^{Python. It do be like that.}

Forward references: do not go there, lest this animated reenactment of @leycec's life corrupt your once-pure essence.

I'm pretty sure (but not certain) that I may have inadvertently summoned the Eldritch She-God Shub-Niggurath, The Black Goat of the Woods with a Thousand Young, during the resolution of this feature request. I apologize, humanity. I'm so sorry! I didn't mean to open that portal. It just opened of its own accord while the guard cat was indolently passed out last night.