[Feature Request] Pseudo-callable (i.e., non-function object implementing the __call__() protocol) support

[patrick-kidger]

With beartype version 0.12.0:

from beartype import beartype
from functools import update_wrapper

def f(x: int):
    pass

class A:
    def __call__(self, a):
        pass

a = A()
a = update_wrapper(a, f)
beartype(a)
# beartype.roar.BeartypeDecorWrappeeException: <__main__.A object at 0x7d03b3c19d90> not pure-Python callable backed by code object (i.e., either C-based callable or pure-Python callable backed by __call__() dunder method).

:(

[leycec]

ohnoes. A bug, everybody! An actual bug! It's been months since I had one of these. I almost don't know what to do, anymore. Paaaaani–

Wait. Panic never solved anything except causing me to bald yet further. It's time to roll up the unwashed sleeves and get greasy with the @beartype codebase again. 🐒

[leycec]

...heh. So, it comes to this. @beartype doesn't appear to explicitly support pseudo-callables (i.e., objects masquerading as functions by defining the __call__() dunder method) yet. That is a fairly serious omission, which is bad.

Let's rectify that, stat! And... there goes the beartype 0.12.1 patch release I was gonna push out tonight. This is why @leycec exhales with exhaustion. See also this surprisingly accurate emoji: 😮‍💨

[leycec]

Resolved by cab50aa. Bwa-ha! Against all odds, we did something. Interestingly, resolving this pushes @beartype considerably closer to a full-blown runtime type-checking monkey-patcher. After all, @beartype is now only one small step for mankind away from being able to:

• (A) Accept any arbitrary Python object.
• (B) Monkey-patch (i.e., replace) all methods of that object with comparable methods augmented by @beartype-based runtime type-checking.

Oh – and I think your example's a bit busted, maybe? The low-level functools.update_wrapper() function doesn't work quite the way everyone wants it to. If your code can be refactored to leverage the higher-level @functools.wraps() decorator instead, that would be just noice: e.g.,

from beartype import beartype
from functools import wraps

def f(self, x: int):
    return x

class A:
    @wraps(f)
    def __call__(self, x):
        return f(self, x)

a = A()
beartype(a)

# This prints:
#     0xFEEDFACE
print(a(0xFEEDFACE))

# This raises the expected type-checking exception:
#     beartype.roar.BeartypeCallHintParamViolation: @beartyped __main__.f()
#     parameter x='What horror hath @beartype wraught this time!?' violates
#     type hint <class 'int'>, as str 'What horror hath @beartype wraught
#     this time!?' not instance of int.
print(a('What horror hath @beartype wraught this time!?'))

Thanks so much for flying Air Beartype. We hope you enjoy your flight. To your left, you can see a giant balloon shaped like a bear currently being filled with Sidewinder missiles by an F-35A Lightning II. Meals consisting exclusively of honey and maple syrup will begin shortly. 🎈

[patrick-kidger]

It's worth bearing (ahem) in mind that your example does something different to my original -- I was wrapping the class, not the __call__ method.

What do you mean? wraps is just a functools.partial + update_wrapper.

[leycec]

Ahh... I'm afraid your example doesn't work, though. It is sad. I tried it, too; functools.update_wrapper fails to transfer the type hint annotating the f() function onto the A.__call__() method, which then causes beartype(a) to just silently reduce to a noop, because A.__call__() remains unannotated.

You can readily verify this yourself at a REPL: e.g.,

$ ipython3.11
>>> from beartype import beartype
>>> from functools import update_wrapper
>>> def f(x: int): pass
>>> class A:
...     def __call__(self, a): pass
>>> a = A()
>>> a = update_wrapper(a, f)
>>> a.__call__.__annotations__
{}

A.__call__() remains unannotated. So, update_wrapper() did nothing. Or mostly nothing, anyway. Or... am I missing something here? I'm probably missing something. That often happens when you in a live in a cabin down by the river. </sigh>

[patrick-kidger]

It ends up being placed on the class instance as a.__annotations__ instead.

Perhaps beartype could check for __annotations__ dictionaries on both the class instance and the __call__ method?

Alternatively I could probably also just call both update_wrapper(a, f) and update_wrapper(a.__call__, f). Actually, that might take some more thought, as a.__call__ is an ephemeral bound method. And adding annotations to A.__call__ would be wrong as different instances of A may wrap different functions. Probably this could be fixed by wrapping __call__ in a custom descriptor? Just update_wrapper(a, f) would still be the usual approach here though.

[leycec]

It ends up being placed on the class instance as a.__annotations__ instead.

nods furiously

Perhaps beartype could check for __annotations__ dictionaries on both the class instance and the __call__ method?

That... sounds pretty non-standard, magical, and implicit. I mean, sure. Technically, @beartype can do anything. Nobody in CPython's typing community acknowledges the existence of runtime type-checking. They've left everything up to our perverse discretion. Always a mistake, Guido! Now, we're making up the rules as we go along.

But you should probably just propagate type hints onto the __annotations__ dictionary of the __call__() dunder method like everybody else. You can do that. So, you should just do that. That's the standard, right? Just pass A.__call__ rather than a to the update_wrapper() function. I can confirm that behaves as expected: e.g.,

from beartype import beartype
from functools import update_wrapper

def f(self, x: int):
    return x

class A:
    def __call__(self, x):
        return f(self, x)

a = A()
update_wrapper(A.__call__, f)
beartype(a)

# Works! Totally works! I am shocked face as well.
print(a(0xFEEDFACE))
print(a('What horror hath @beartype wraught this time!?'))

Do not ask what @beartype can do for Google X. Ask what Google X can do for @beartype. 😄

Kidding! Just kidding! Still, the sentiment stands. Typing is an anarchic battle royale and you're personally invited. Or... does even the above not work for your specific use case? Does Google really need @beartype to start making up its own normative pseudo-standards? We could do that, if Google is asking. What Google wants, Google gets. Even cray-cray stuff is on the table. </sigh>

[patrick-kidger]

Being placed on A.__call__ would be wrong. That would result in all instances of A sharing the same annotations.

Intended behaviour here is that each instance of A wrap a different function.

I think doing so is canonical / not sketchy!

(Sadly, I'm only using beartype in hobby projects right now. FWIW the complexity of beartype's codebase probably makes it a tough security review before it could be used anywhere internally.)

[leycec]

Being placed on A.__call__ would be wrong. That would result in all instances of A sharing the same annotations.

Exactly. That's exactly it. Oh... I see. @patrick-kidger doesn't know yet. Gods. Should I be the one to break it to him? I hate being that guy. 😶‍🌫️

Sadly, you've just hit a deep Python constraint without knowing it. Unlike almost everything else, __call__() only exists on classes. Technically, you can try to forcefully set a __call__ attribute on the dictionary of a non-class object. Pragmatically, nobody does that, because Python silently ignores any attempts to do so.

Thus, A.__call__. a.__call__ basically doesn't exist; it's just a thin bound method descriptor around A.__call__. In particular, a.__call__ has no unique annotations of its own, because:

>>> assert a.__call__.__func__ is A.__call__
True

When you call beartype(a) from your code, you might think you're just monkey-patching the object a without touching its class A. That's what it looks, right? In actuality, it's the opposite; beartype(a) monkey-patches the class A without touching the object a. Why? Because @beartype has no choice.

Again, this is a low-level Python constraint. Although most methods don't work like that, __call__() does. Python probably does this for space and/or time efficiency reasons. I can only shrug politely and refer you to official documentation on the subject.

FWIW the complexity of beartype's codebase probably makes it a tough security review before it could be used anywhere internally.

Hah! The jokes on you, Google. Against all odds, @beartype has already insinuated itself into over 100 upstream open-source packages – some of which I guarantee Google is already using, because they are literally Google projects. You and I are equally shocked.

AudioLM-Pytorch is my personal favourite. It's Google. It's @beartype. As an obsessive-compulsive RyMer (i.e., RateYourMusic user), I proudly look forward to the day when Google AI will synthesize a realtime feed of @beartype-checked afro-samba música popular brasileira synthwave that I can jack directly into my neocortex. 🎧 🎶

[patrick-kidger]

Sadly, you've just hit a deep Python constraint without knowing it. Unlike almost everything else, call() only exists on classes. Technically, you can try to forcefully set a call attribute on the dictionary of a non-class object. Pragmatically, nobody does that, because Python silently ignores any attempts to do so.

Thus, A.call. a.call basically doesn't exist; it's just a thin bound method descriptor around A.call. In particular, a.call has no unique annotations of its own, because

This isn't quite right. You are correct that a.__call__ is just an ephemeral descriptor. But that's not the reason for your statement "Python silently ignores any attempts to do so".

The reason is that built-in magic methods are looked up on the type: foo() resolves to type(foo).__call__(foo). This was chosen as a performance optimisation within Python, to avoid looking through potentially large instance dictionaries.

Incidentally, this has an exception. If the class is a types.ModuleType, then __getattr__ and __dir__ are looked up on the instance rather than the class. This is to allow writing def __getattr__(...): ... as top-level functions in Python files (=instances of ModuleType), as for example you do in beartype/__init__.py.

Ephemerality of bound methods is unrelated.

---

This shouldn't affect beartype's ability to check class instances as callables. For example, here's a wrapper that fixes beartype's behaviour here (just by wrapping the instance into a function, which has the desired behaviour):

import beartype
import functools as ft

# Library code

def fixed_beartype(fn):
  @ft.wraps(fn)
  def wrapped(*args, **kwargs):
    return fn(*args, **kwargs)
  return beartype.beartype(wrapped)

# User code

class Kls:
  def __call__(self, x):
    return x * 2

a = Kls()
b = Kls()

def f(x: int):
  ...

def g(x: str):
  ...

ft.update_wrapper(a, f)
ft.update_wrapper(b, g)
assert a.__annotations__ == {"x": int}
assert b.__annotations__ == {"x": str}
a = fixed_beartype(a)
b = fixed_beartype(b)

a(1)  # okay
b("hi")  # okay

try:
  a("hi")
except beartype.roar.BeartypeCallHintParamViolation:
  pass  # okay
else:
  raise Exception

try:
  b(1)
except beartype.roar.BeartypeCallHintParamViolation:
  pass  # okay
else:
  raise Exception

This works because f.__annotations__ gets transferred on to a.__annotations__ which gets transferred on to fixed_beartype.<locals>.wrapped.__annotations__, and beartype correctly looks up annotations on functions.

My expectation is that beartype(foo)(*args) will check *args against foo.__annotations__, regardless of whether foo is an instance or a function.

AudioLM-Pytorch is my personal favourite. It's Google. It's https://github.com/beartype. As an obsessive-compulsive RyMer (i.e., RateYourMusic user), I proudly look forward to the day when Google AI will synthesize a realtime feed of @beartype-checked afro-samba música popular brasileira synthwave that I can jack directly into my neocortex. 🎧 🎶

I don't think that's a Google project.

Lucidrains does reimplementations of the work of various research groups; Google amongst them.

Beartype isn't currently used internally.