Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[typing] Add typing to the class checkers #7190

Merged

Conversation

Pierre-Sassoulas
Copy link
Member

@Pierre-Sassoulas Pierre-Sassoulas commented Jul 14, 2022

Type of Changes

Type
✨ New feature
📜 Docs

Description

Refs #2079

@Pierre-Sassoulas Pierre-Sassoulas added Maintenance Discussion or action around maintaining pylint or the dev workflow Skip news 🔇 This change does not require a changelog entry labels Jul 14, 2022
@coveralls
Copy link

coveralls commented Jul 14, 2022

Pull Request Test Coverage Report for Build 2833516279

  • 57 of 57 (100.0%) changed or added relevant lines in 3 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.002%) to 95.259%

Totals Coverage Status
Change from base Build 2828349969: 0.002%
Covered Lines: 16839
Relevant Lines: 17677

💛 - Coveralls

@github-actions

This comment has been minimized.

Copy link
Collaborator

@DanielNoord DanielNoord left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was quite a lot to review. I have made an initial pass and think I got all but I'll probably need to do another pass next time.

Let's merge this one before doing the other typing one.

pylint/checkers/classes/class_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/class_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/class_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/class_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/class_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/class_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/special_methods_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/special_methods_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/special_methods_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/special_methods_checker.py Outdated Show resolved Hide resolved
@jacobtylerwalls jacobtylerwalls changed the title [typing] Add typing to the classe checkers [typing] Add typing to the class checkers Jul 14, 2022
Pierre-Sassoulas added a commit to Pierre-Sassoulas/pylint that referenced this pull request Jul 15, 2022
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

pylint/checkers/classes/class_checker.py Outdated Show resolved Hide resolved
@@ -1443,7 +1458,9 @@ def _check_redefined_slots(
node=slots_node,
)

def _check_slots_elt(self, elt, node):
def _check_slots_elt(
self, elt: nodes.Attribute | nodes.Const, node: nodes.ClassDef
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is the type of elt determined? In the calling function it seems to be SuccessfulInferenceResult?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the values are based on what were the actual type of the node when launching the tests. There's often a lof of condition applied on things after the inference is done so it narrows the type of sub-functions quite a lot sometime.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but just based on the current typing and code this typing is incorrect. We should mostly rely on the code rather than on tests as the tests will never be cover every edge case.

pylint/checkers/classes/class_checker.py Show resolved Hide resolved
return isinstance(func, nodes.FunctionDef) and (
func.type == "classmethod" or func.name == "__class_getitem__"
)

@staticmethod
def _is_inferred_instance(expr, klass):
def _is_inferred_instance(expr: nodes.NodeNG | None, klass: nodes.ClassDef) -> bool:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The typing and the docstring in astroid don't correspond with each other here. Shall we just remove the typing and fix that first?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would work but maybe we could add best effort values in pylint and see where mypy is unhappy once astroid is actually typed ? (The overload for infer depending on what node we're inferring and based on condition on the node can become quite complicated, and won't be done in a month).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that we won't know which signatures to revisit when we finally do update astroid. For any typing for which we are not sure that astroid is correct I think it is often better to leave them be and try and fix astroid first.

pylint/checkers/classes/special_methods_checker.py Outdated Show resolved Hide resolved
pylint/checkers/classes/special_methods_checker.py Outdated Show resolved Hide resolved
if not self._is_iterator(inferred):
self.add_message("non-iterator-returned", node=node)

def _check_len(self, node, inferred):
def _check_len(self, node: nodes.FunctionDef, inferred: nodes.Const) -> None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just picking this one as it is the easiest to illustrate, I don't really understand where these typings are coming from? Don't they all come from _protocol_map and called with Callable[[nodes.FunctionDef, Any], None]?

The Any being the result of infer_call_result? Why can this only be Const?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For:

    def __len__(self):  # [invalid-length-returned]
        return lambda: 3

It's not a const but a ``<class 'astroid.nodes.scoped_nodes.scoped_nodes.Lambda>'. pyannotate must have a bug.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is not necessarily a bug with pyannotate but just shows that our test case doesn't cover all edge cases. Like I said, perhaps revert the changes here and keep this PR focused on those that are definetly correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

our test case doesn't cover all edge cases.

Although true in the general case, I added an assert in def _check_len, the example come from invalid_length_returned.py that is supposed to be launched during functional tests so either pyannotate has a bug or the way we launch tests is not understood.

Co-authored-by: Daniël van Noord <13665637+DanielNoord@users.noreply.github.com>
@DanielNoord
Copy link
Collaborator

@Pierre-Sassoulas Please don't rebase or force push these larger PRs, pretty difficult to keep track of the changes I already reviewed that way 😅

node: nodes.FunctionDef,
caller: nodes.FunctionDef,
context: InferenceContext | None = None,
) -> Iterator[nodes.NodeNG | Uninferable | None] | None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous comments, why isn't this InferenceResult?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -1653,7 +1652,7 @@ def get_subscript_const_value(node: nodes.Subscript) -> nodes.Const:
return inferred


def get_import_name(importnode: ImportNode, modname: str | None) -> str | None:
def get_import_name(importnode: nodes.ImportNode, modname: str | None) -> str | None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't work. ImportNode is not part of the nodes module.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm; it's true I don't see it in astroid.nodes's API. But the tests are green.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is because mypy doesn't actually check astroid since it has no py.typed file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha, I thought the import would have failed but it does not because it's using a namespace and not a direct import (nodes.ImportNode, with from astroid import nodes not from astroid.nodes import ImportNode.

@github-actions

This comment has been minimized.

@DanielNoord
Copy link
Collaborator

@Pierre-Sassoulas Would you be okay with me finishing this PR with some final changes?

@Pierre-Sassoulas
Copy link
Member Author

Sure, I would appreciate that !

@DanielNoord
Copy link
Collaborator

@Pierre-Sassoulas PR is now ready for review!

@github-actions

This comment has been minimized.

Copy link
Member Author

@Pierre-Sassoulas Pierre-Sassoulas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm, thank you @DanielNoord !

@DanielNoord DanielNoord enabled auto-merge (squash) August 10, 2022 14:44
@DanielNoord DanielNoord merged commit 8372180 into pylint-dev:main Aug 10, 2022
@github-actions
Copy link
Contributor

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit b1d0403

@Pierre-Sassoulas Pierre-Sassoulas deleted the add-typing-to-checker-classes branch August 19, 2022 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Maintenance Discussion or action around maintaining pylint or the dev workflow Skip news 🔇 This change does not require a changelog entry
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants