[BUG]: SupportsInt and SupportsFloat changes cause type checking failures on python classes that override these functions

[daltairwalter]

Required prerequisites

• Make sure you've read the documentation. Your issue may be addressed there.
• Search the issue tracker and Discussions to verify that this hasn't already been reported. +1 or comment there if it has.
• Consider asking first in the Gitter chat room or in a Discussion.

What version (or hash if on master) of pybind11 are you using?

3.0.0

Problem description

#5540 changed type hints from int and float to SupportsInt and SupportsFloat. While these may be nice for use cases where a C++ module is exposed to python, this is causing a lot of type problems for cases where a C++ function is trampolined and overridden in python. The base class is now using SupportsInt and SupportsFloat and this is more generic than the python derived classes that uses int, float so mypy issues errors like:

Argument 1 of "pdf" is incompatible with supertype "Distribution"; supertype defines the argument
type as "SupportsFloat" [override]
def pdf(self, x: float) -> float:

Am I missing something here, or is there a way to turn this feature off?

Reproducible example code

Is this a regression? Put the last known working version here if it is.

smartholder branch on 2024-08-09

[timohl]

Is changing the type hint for the derived class an option in order to match the new type hints?
I suppose you would have to add a x = float(x) as first line if the code does require actual floats.

Otherwise, I think there is currently no way for turning this off.

For fixing your issue I could imagine two options:

1. Adding a global macro to set the type hints for int and float.
   The following code already changes type hints depending on the python version, so something similar could be used to globally change the type hint for int/float:
   

   pybind11/include/pybind11/detail/common.h

   Lines 271 to 299 in 7f5eea4

   	// 3.13 Compatibility
   	#if 0x030D0000 <= PY_VERSION_HEX
   	# define PYBIND11_TYPE_IS_TYPE_HINT "typing.TypeIs"
   	# define PYBIND11_CAPSULE_TYPE_TYPE_HINT "types.CapsuleType"
   	#else
   	# define PYBIND11_TYPE_IS_TYPE_HINT "typing_extensions.TypeIs"
   	# define PYBIND11_CAPSULE_TYPE_TYPE_HINT "typing_extensions.CapsuleType"
   	#endif
   	// 3.12 Compatibility
   	#if 0x030C0000 <= PY_VERSION_HEX
   	# define PYBIND11_BUFFER_TYPE_HINT "collections.abc.Buffer"
   	#else
   	# define PYBIND11_BUFFER_TYPE_HINT "typing_extensions.Buffer"
   	#endif
   	// 3.11 Compatibility
   	#if 0x030B0000 <= PY_VERSION_HEX
   	# define PYBIND11_NEVER_TYPE_HINT "typing.Never"
   	#else
   	# define PYBIND11_NEVER_TYPE_HINT "typing_extensions.Never"
   	#endif
   	// 3.10 Compatibility
   	#if 0x030A0000 <= PY_VERSION_HEX
   	# define PYBIND11_TYPE_GUARD_TYPE_HINT "typing.TypeGuard"
   	#else
   	# define PYBIND11_TYPE_GUARD_TYPE_HINT "typing_extensions.TypeGuard"
   	#endif

   
   While this fixes this issue, I do not like it, since it also turns it off at other more useful places.

2. Adding py::typing::int and py::typing::float to pybind11/typing.h that forces using int/float.
   Using these in the C++ trampline class could maybe fix this, if I understand your usecase correctly.
   I would prefer this since it is more localized.

Both options are just thoughts on how a fix for your usecase could be implemented, so I am happy to hear other thoughts.

[InvincibleRMC]

You can also do the following.

    m.def("float_passthrough_noconvert", [](float arg) { return arg; }, py::arg{}.noconvert());

Generates

def float_passthrough_noconvert(arg0: float) -> float:

[timohl]

You can also do the following.

m.def("float_passthrough_noconvert", [](float arg) { return arg; }, py::arg{}.noconvert());

Generates

def float_passthrough_noconvert(arg0: float) -> float:

Oh, how could I forget, probably too late in the evening.

But yes, that is the way to go! 👍

[daltairwalter]

I will look into the noconvert option. Is there any way to do this at the module level rather than at the function level?

[daltairwalter]

noconvert() does produce the type hints I want, but the documentation also indicates that this changes the functionality too. According to the noconvert documentation it will prevent an int from being used as a float. If I understand all this correctly, it seems like this implementation of noconvert directly conflicts with PEP 484 which states that when an argument is annotated as having type float, an argument of type int is acceptable.

[daltairwalter]

Looking back at @timohl's options for fixing this, it seems like the global macro would definitely work. If I understand correctly, this would effectively just replace typing.SupportsInt and typing.SupportsFloat with int and float everywhere.

I am not sure I understand the option for using py::typing::int and py::typing::float in the trampolines. My understanding is that the type hints are coming from the module definition and not the trampoline so it seems like this would also require changes in the module definition level and this would snowball into a huge amount of changes and boiler plate code.

I don't know how this would be done, but as a user I would most prefer to have a module level option for this where I see the option multiple_interpreters::per_interpreter_gil something like type_hinting::use_int_and_float

[gentlegiantJGC]

2. Adding py::typing::int and py::typing::float to pybind11/typing.h that forces using int/float.

I think this is redundant.
The current casting behaviour of py::int_ only accepts python ints and py::float_ only accepts python floats.
I believe type type hints for py::int_ should be int and py::float_ should be float. See #5839
I believe py::float_ should also accept python ints per PEP 484. See #5840 See #5839

This would allow the SupportsInt/Float behaviour through the native int and float and python-like behaviour through py::int_ and py::float_

[daltairwalter]

This would allow the SupportsInt/Float behaviour through the native int and float and python-like behaviour through py::int_ and py::float_

@gentlegiantJGC this behavior does not fix the problem. It would require wrapping everything that should have normal int/float type hints in py::int_ and py::float_ and this egregious. Some more general way of suppressing the SupportsInt and SupportsFloat is needed.

[gentlegiantJGC]

Can I suggest creating a pull request.
As timohl said here the simple solution is to add a configuration macro and a preprocessor directive to switch the behaviour.

Something like this

// common.h
#ifndef PYBIND11_ENABLE_NUMERIC_SUPPORTS_HINT
#define PYBIND11_ENABLE_NUMERIC_SUPPORTS_HINT 1
#endif

And this
https://github.com/pybind/pybind11/blob/326b10637a9fe2d1c9f290b6e052097ffc2413c3/include/pybind11/cast.h#L348C1-L350C93
Would be changed to something like this

    PYBIND11_TYPE_CASTER(T,
                         io_name<std::is_integral<T>::value>(
#if PYBIND11_ENABLE_NUMERIC_SUPPORTS_HINT
                             "typing.SupportsInt", "int", "typing.SupportsFloat", "float"));
#else 
                             "int", "int", "float", "float"));
#endif

Maybe switching the whole section might be eaiser to read. I will leave the implementation to you.