feat(types): add TypeVars / method generics typing

[InvincibleRMC]

Description

Add supports for Typevars.
Requires __cpp_nontype_template_parameter_class feature.

To create the new a new typing::TypeVar use the following syntax.

typedef py::typing::TypeVar<"T"> TypeVarT;

This allows use to create type annotations like the following.

def foo(arg: list[T]) -> T: ...

Suggested changelog entry:

Adds support for python typing ``TypeVar`` to annotations.

[InvincibleRMC]

TypeVars seem to be working for all versions of compilers with C++20 that support string literals in templates. Now I ask should the feature be dropped to maintain capability with older C++ standards. Or is there precedent for C++ version guarding features?

[rwgk]

I converted this to Draft since this looks like you're still working on it.

Is the idea to add ifdef(C++20)?

[InvincibleRMC]

Yeah. I wasn't sure if that was fine since some projects want equal support for different C++ standards.

[henryiii]

There are C++14 (py::overload_cast) and C++17 (std::variant and such) only features. Not sure if we have any C++20 features yet.

[rwgk]

@henryiii wrote:

Not sure if we have any C++20 features yet.

Me too; I don't think so. I'm only aware of accommodations for C++20 (module_ and PYBIND11_CPP20).

But I cannot think of reasons against adding conditional support for C++20 features. Are there concerns I'm missing?

[rwgk]

Looks good to me.

Could you please make a pass through the PR description (quick and simple, I think): Is "Prototyping" still what you have in mind? And we need a Suggested changlog entry.

[InvincibleRMC]

A potential problem I discovered in testing is stub generators like mypy's stubgen or pybind11-stubgen will need to recognize TypeVars from the compiled annotations to create the TypeVar objects themselves to then be used. I'm not sure of the best way of notifying the stub generators to create TypeVar instantiations. Another option would be to modify the order resolution of of the annotation generation to use the Python3.12 inline TypeVar declaration.

Python < 3.12

from typing import TypeVar
T = TypeVar('T')

def foo(arg0: T) -> T: ...

Python >= 3.12

def foo[T](arg0: T) -> T: ...

[rwgk]

A potential problem I discovered in testing is stub generators like mypy's stubgen or pybind11-stubgen will need to recognize TypeVars from the compiled annotations to create the TypeVar objects themselves to then be used.

I only have a very sketchy idea of what mypy's stubgen is doing. But while working on #4888 I learned that stubgen has fallback mechanisms if it cannot process some annotations.

Do you believe this PR will make things worse?

[InvincibleRMC]

It is not ideal, but does not make anything worse. If that is acceptable I'm fine merging it in.

[rwgk]

It is not ideal, but does not make anything worse. If that is acceptable I'm fine merging it in.

I'm for merging this: it's clearly making pybind11 more complete/correct, with very little extra code. — The very minimal two-way collaboration between pybind11 and mypy stubgen is a bigger problem that hopefully will get addressed down the road (something like #4888).

@henryiii What's your opinion?