Support type stub generation for `staticmethod` #14934

WeilerMarcel · 2023-03-21T18:36:01Z

Fixes #13574

This PR fixes the generation of type hints for static methods of pybind11 classes. The code changes are based on the suggestions in #13574.

The fix introduces an additional check if the property under inspection is of type staticmethod. If it is, the type information is read from the staticmethod's __func__ attribute, instead of the staticmethod instance itself.

I added a test for C++ classes with static methods bound using pybind11. Both, an overloaded and a non-overloaded static method are tested.

mypy/stubgenc.py

WeilerMarcel · 2023-05-30T05:52:51Z

Hi @sobolevn and @AlexWaygood, I updated this PR to work with the latest changes on master. Could you please review the current changes and merge the PR if everything is fine?

WeilerMarcel · 2023-06-28T08:47:48Z

Hey @sobolevn and @AlexWaygood, what is the status of this pull request? I would really like to see it merged into MyPy. Do you need anything from my side to proceed?

d4n1elchen · 2023-07-12T23:20:13Z

Looking forward to this getting merged as well +1

WeilerMarcel · 2023-10-13T09:07:54Z

Hi @sobolevn and @AlexWaygood, please give me an update on this PR. Is there a reason why it has not been merged? As far as I can see, I can not merge it myself since I don't have the necessary permissions.

sobolevn · 2023-10-13T09:20:11Z

I can't merge this PR, because I don't fully understand the C++ code, but I think that @JukkaL or @msullivan can help us :)

AlexWaygood · 2023-10-13T09:21:48Z

I can't merge this PR because I'm not a maintainer

WeilerMarcel · 2023-10-16T07:32:09Z

Thanks for the info :) I was under the impression that you could merge, since you were reviewing my changes.
I will wait for a reply from @JukkaL and @msullivan.

sobolevn · 2023-10-16T07:33:37Z

In the meantime you can resolve conflicts :)

for more information, see https://pre-commit.ci

…l/mypy into support_staticmethod

WeilerMarcel · 2023-11-08T10:51:53Z

Hi everyone, I integrated my changes into the new master branch and from my perspective it is ready for merging.

I noticed, though, that the mypyc runtime tests with py39-macos seem to be unstable. Since I did not find a way to rerun tests for a commit, I added two empty dummy commits to get the pipeline to succeed. See #16420.

mypy/stubgenc.py

JukkaL · 2023-11-08T13:55:44Z

You can ignore the mypyc runtime test failures, since it's a known issue happening on master.

AlexWaygood · 2023-11-09T11:43:07Z

mypy/stubgenc.py

            return False
+        elif self.is_c_module:
+            raw_lookup = getattr(class_info.cls, "__dict__")  # noqa: B009


Why has this been # noqa'd? Wouldn't it be cleaner to do this, as the linter suggests?

Suggested change

raw_lookup = getattr(class_info.cls, "__dict__") # noqa: B009

raw_lookup = class_info.cls.__dict__

To be honest, I wrote it that way because there are two other places in stubgenc.py where this pattern was used:

mypy/mypy/stubgenc.py

Lines 453 to 455 in 8121e3c

def get_members(self, obj: object) -> list[tuple[str, Any]]:

obj_dict: Mapping[str, Any] = getattr(obj, "__dict__") # noqa: B009

results = []

mypy/mypy/stubgenc.py

Lines 718 to 725 in 8121e3c

def generate_class_stub(self, class_name: str, cls: type, output: list[str]) -> None:

"""Generate stub for a single class using runtime introspection.

The result lines will be appended to 'output'. If necessary, any

required names will be added to 'imports'.

"""

raw_lookup = getattr(cls, "__dict__") # noqa: B009

items = self.get_members(cls)

I will check the file history to see if I can find a specific reason why getattr is used.

The getattr call originally comes from commit 6bcaf40 and was added to make typeshed happy:

mypy/mypy/stubgenc.py

Lines 140 to 142 in 6bcaf40

# typeshed gives obj.__dict__ the not quite correct type Dict[str, Any]

# (it could be a mappingproxy!), which makes mypyc mad, so obfuscate it.

obj_dict = getattr(obj, '__dict__') # type: Mapping[str, Any]

The # noqa was added in 4287af4:

mypy/mypy/stubgenc.py

Line 246 in 4287af4

obj_dict = getattr(obj, '__dict__') # type: Mapping[str, Any] # noqa

WeilerMarcel · 2023-12-05T13:19:36Z

Hi @JukkaL and @msullivan, I resolved conflicts with the current master branch. Can this PR be merged?

WeilerMarcel · 2024-01-03T10:32:05Z

Happy new year everyone! 🥳
Are there plans to merge this PR yet? I would really like to get this off my mind.

JukkaL · 2024-01-03T11:29:28Z

I can merge this PR if somebody who has more context on the relevant code than I have gives a quick thumbs up.

JukkaL · 2024-01-03T11:30:27Z

more context on the relevant code than I have

Or somebody who has any familiarity with the code :-)

hauntsaninja

Not that familiar with stubgen, but this mostly looks good to me. I had one question

mypy/stubgenc.py

hauntsaninja

Thank you! :-)

sobolevn reviewed Mar 26, 2023

View reviewed changes

mypy/stubgenc.py Outdated Show resolved Hide resolved

AlexWaygood added the topic-stubgen label Mar 26, 2023

WeilerMarcel requested review from sobolevn and AlexWaygood May 31, 2023 20:32

WeilerMarcel mentioned this pull request Jul 10, 2023

stubgenc.py does not generate correct lines for staticmethods #13574

Closed

Marcel Weiler added 3 commits November 7, 2023 09:35

Add test for static methods of pybind11 class

bb15d87

Fix is_staticmethod for c modules

feebc70

Updated stub files for stubgen test

a634dee

WeilerMarcel force-pushed the support_staticmethod branch from 19f3832 to a634dee Compare November 7, 2023 14:52

pre-commit-ci bot and others added 5 commits November 7, 2023 14:53

[pre-commit.ci] auto fixes from pre-commit.com hooks

cae94e6

for more information, see https://pre-commit.ci

Fix failing type check

1aacfcc

Merge branch 'support_staticmethod' of https://github.com/WeilerMarce…

32389c1

…l/mypy into support_staticmethod

dummy

8142772

dummy to rerun unstable macos tests

bc42c57

WeilerMarcel mentioned this pull request Nov 8, 2023

mypyc runtime tests with py39-macos is unstable #16420

Closed

JukkaL reviewed Nov 8, 2023

View reviewed changes

mypy/stubgenc.py Outdated Show resolved Hide resolved

Simplify is_staticmethod check

8121e3c

AlexWaygood reviewed Nov 9, 2023

View reviewed changes

Marcel Weiler added 2 commits November 15, 2023 13:56

Add type hint for mapping proxies

2504078

Merge remote-tracking branch 'origin/master' into support_staticmethod

e33685c

Fix stubgenc docstring test

e6f61ec

hauntsaninja reviewed Jan 4, 2024

View reviewed changes

mypy/stubgenc.py Outdated Show resolved Hide resolved

Replace string comparison by isinstance

14fd95c

hauntsaninja approved these changes Jan 8, 2024

View reviewed changes

hauntsaninja merged commit 35f402c into python:master Jan 8, 2024
14 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support type stub generation for `staticmethod` #14934

Support type stub generation for `staticmethod` #14934

WeilerMarcel commented Mar 21, 2023

WeilerMarcel commented May 30, 2023

WeilerMarcel commented Jun 28, 2023

d4n1elchen commented Jul 12, 2023

WeilerMarcel commented Oct 13, 2023

sobolevn commented Oct 13, 2023

AlexWaygood commented Oct 13, 2023

WeilerMarcel commented Oct 16, 2023

sobolevn commented Oct 16, 2023

WeilerMarcel commented Nov 8, 2023

JukkaL commented Nov 8, 2023

AlexWaygood Nov 9, 2023

WeilerMarcel Nov 9, 2023 •

edited

WeilerMarcel Nov 10, 2023 •

edited

WeilerMarcel commented Dec 5, 2023

WeilerMarcel commented Jan 3, 2024

JukkaL commented Jan 3, 2024

JukkaL commented Jan 3, 2024

hauntsaninja left a comment

hauntsaninja left a comment

	raw_lookup = getattr(class_info.cls, "__dict__") # noqa: B009
	raw_lookup = class_info.cls.__dict__

	def get_members(self, obj: object) -> list[tuple[str, Any]]:
	obj_dict: Mapping[str, Any] = getattr(obj, "__dict__") # noqa: B009
	results = []

	def generate_class_stub(self, class_name: str, cls: type, output: list[str]) -> None:
	"""Generate stub for a single class using runtime introspection.

	The result lines will be appended to 'output'. If necessary, any
	required names will be added to 'imports'.
	"""
	raw_lookup = getattr(cls, "__dict__") # noqa: B009
	items = self.get_members(cls)

	# typeshed gives obj.__dict__ the not quite correct type Dict[str, Any]
	# (it could be a mappingproxy!), which makes mypyc mad, so obfuscate it.
	obj_dict = getattr(obj, '__dict__') # type: Mapping[str, Any]

Support type stub generation for staticmethod #14934

Support type stub generation for staticmethod #14934

Conversation

WeilerMarcel commented Mar 21, 2023

Fixes #13574

WeilerMarcel commented May 30, 2023

WeilerMarcel commented Jun 28, 2023

d4n1elchen commented Jul 12, 2023

WeilerMarcel commented Oct 13, 2023

sobolevn commented Oct 13, 2023

AlexWaygood commented Oct 13, 2023

WeilerMarcel commented Oct 16, 2023

sobolevn commented Oct 16, 2023

WeilerMarcel commented Nov 8, 2023

JukkaL commented Nov 8, 2023

AlexWaygood Nov 9, 2023

Choose a reason for hiding this comment

WeilerMarcel Nov 9, 2023 • edited

Choose a reason for hiding this comment

WeilerMarcel Nov 10, 2023 • edited

Choose a reason for hiding this comment

WeilerMarcel commented Dec 5, 2023

WeilerMarcel commented Jan 3, 2024

JukkaL commented Jan 3, 2024

JukkaL commented Jan 3, 2024

hauntsaninja left a comment

Choose a reason for hiding this comment

hauntsaninja left a comment

Choose a reason for hiding this comment

Support type stub generation for `staticmethod` #14934

Support type stub generation for `staticmethod` #14934

WeilerMarcel Nov 9, 2023 •

edited

WeilerMarcel Nov 10, 2023 •

edited