Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug pylint 4206 #921

Merged
merged 61 commits into from
Apr 6, 2021
Merged
Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
1beb629
Takes into account the fact that inferring subscript when the node is…
hippo91 Mar 13, 2021
4c85bd0
OrderedDict in the collections module inherit from dict which is C co…
hippo91 Mar 13, 2021
99a078b
check_metaclass becomes a static class method because we need it in t…
hippo91 Mar 13, 2021
32d89e2
The brain_typing module does not add anymore _typing suffixed classes…
hippo91 Mar 13, 2021
65a8f88
The OrderedDict class inherits from C coded dict class and thus doesn…
hippo91 Mar 13, 2021
0dee7f8
When trying to inherit from typing.Pattern the REPL says : TypeError…
hippo91 Mar 13, 2021
450fae1
The REPL says that Derived as ABCMeta for metaclass and the mro is De…
hippo91 Mar 13, 2021
eac3ad1
Adds comments
hippo91 Mar 14, 2021
2c711e7
Starting with Python39 some collections of the collections.abc module…
hippo91 Mar 14, 2021
542eb9c
Thanks to __class_getitem__ method there is no need to hack the metac…
hippo91 Mar 14, 2021
f5aef79
SImplifies the inference system for typing objects before python3.9. …
hippo91 Mar 14, 2021
84c576c
check_metaclass_is_abc become global to be shared among different cla…
hippo91 Mar 14, 2021
20ca3a1
Create a test class dedicated to the Collections brain
hippo91 Mar 14, 2021
66c4c7c
Rewrites and adds test
hippo91 Mar 14, 2021
8c7d344
Corrects syntax error
hippo91 Mar 14, 2021
c392180
Deque, defaultdict and OrderedDict are part of the _collections modul…
hippo91 Mar 14, 2021
b0d13d1
Formatting according to black
hippo91 Mar 14, 2021
ab26675
Adds two entries
hippo91 Mar 14, 2021
230fee4
Extends the filter to determine what is subscriptable to include Orde…
hippo91 Mar 14, 2021
79b502c
Formatting according to black
hippo91 Mar 14, 2021
91ce864
Takes into account the fact that inferring subscript when the node is…
hippo91 Mar 13, 2021
136169b
OrderedDict in the collections module inherit from dict which is C co…
hippo91 Mar 13, 2021
4e4e27f
check_metaclass becomes a static class method because we need it in t…
hippo91 Mar 13, 2021
324f02c
The brain_typing module does not add anymore _typing suffixed classes…
hippo91 Mar 13, 2021
3babaf4
The OrderedDict class inherits from C coded dict class and thus doesn…
hippo91 Mar 13, 2021
a1f1dd6
When trying to inherit from typing.Pattern the REPL says : TypeError…
hippo91 Mar 13, 2021
92a9422
The REPL says that Derived as ABCMeta for metaclass and the mro is De…
hippo91 Mar 13, 2021
7afcd62
Adds comments
hippo91 Mar 14, 2021
7b16e00
Starting with Python39 some collections of the collections.abc module…
hippo91 Mar 14, 2021
5b56836
Thanks to __class_getitem__ method there is no need to hack the metac…
hippo91 Mar 14, 2021
6f4328a
SImplifies the inference system for typing objects before python3.9. …
hippo91 Mar 14, 2021
a38e82d
check_metaclass_is_abc become global to be shared among different cla…
hippo91 Mar 14, 2021
121a3db
Create a test class dedicated to the Collections brain
hippo91 Mar 14, 2021
0654a69
Rewrites and adds test
hippo91 Mar 14, 2021
2e382de
Corrects syntax error
hippo91 Mar 14, 2021
86bd75a
Deque, defaultdict and OrderedDict are part of the _collections modul…
hippo91 Mar 14, 2021
eb87d79
Formatting according to black
hippo91 Mar 14, 2021
5d58132
Adds two entries
hippo91 Mar 14, 2021
fa7b6e7
Extends the filter to determine what is subscriptable to include Orde…
hippo91 Mar 14, 2021
d8759f3
Formatting according to black
hippo91 Mar 14, 2021
d8ee527
Takes into account @AWhetter remarks
hippo91 Mar 21, 2021
8ff2893
Deactivates access to __class_getitem__ method
hippo91 Mar 21, 2021
01c7d3c
Merge branch 'bug_pylint_4206' of https://github.com/hippo91/astroid …
hippo91 Mar 27, 2021
0f875c8
OrderedDict appears in typing module with python3.7.2
hippo91 Mar 27, 2021
edf1503
_alias function in the typing module appears with python3.7
hippo91 Mar 27, 2021
3357204
Formatting according to black
hippo91 Mar 27, 2021
38cf33c
_alias function is used also for builtins type and not only for colle…
hippo91 Mar 27, 2021
15bdc78
Adds tests for both builtins type that are subscriptable and typing b…
hippo91 Mar 29, 2021
c5d1c40
No need to handle builtin types in this brain. It is better suited in…
hippo91 Mar 29, 2021
ede0e00
Adds brain to handle builtin types that are subscriptable starting wi…
hippo91 Mar 29, 2021
a09f838
Formatting according to black
hippo91 Mar 29, 2021
d71a7aa
Uses partial function instead of closure in order pylint acceptance t…
hippo91 Mar 31, 2021
1be6d3f
Handling the __class_getitem__ method associated to EmptyNode for bui…
hippo91 Mar 31, 2021
aaa9543
infer_typing_alias has to be an inference_tip to avoid interferences …
hippo91 Mar 31, 2021
56ab112
Formatting
hippo91 Mar 31, 2021
9c42d28
Merge branch 'master' into bug_pylint_4206
hippo91 Mar 31, 2021
80b8ca9
Removes useless code
hippo91 Mar 31, 2021
30b1e79
Adds comment
hippo91 Mar 31, 2021
0bc7d6b
Takes into account @cdce8p remarks
hippo91 Apr 2, 2021
5cd0fcb
Formatting
hippo91 Apr 4, 2021
2d56141
Style changes
cdce8p Apr 6, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 44 additions & 1 deletion astroid/brain/brain_collections.py
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ def __rmul__(self, other): pass"""
if PY39:
base_deque_class += """
@classmethod
def __class_getitem__(self, item): pass"""
def __class_getitem__(self, item): return cls"""
return base_deque_class


Expand All @@ -77,7 +77,50 @@ def _ordered_dict_mock():
class OrderedDict(dict):
def __reversed__(self): return self[::-1]
def move_to_end(self, key, last=False): pass"""
if PY39:
base_ordered_dict_class += """
@classmethod
def __class_getitem__(cls, item): return cls"""
return base_ordered_dict_class


astroid.register_module_extender(astroid.MANAGER, "collections", _collections_transform)


PY39 = sys.version_info >= (3, 9)
hippo91 marked this conversation as resolved.
Show resolved Hide resolved

def _looks_like_subscriptable(node: astroid.nodes.ClassDef) -> bool:
"""
Returns True if the node corresponds to a ClassDef of the Collections.abc module that
supports subscripting

:param node: ClassDef node
"""
if node.qname().startswith('_collections_abc'):
try:
node.getattr('__class_getitem__')
return True
except:
pass
return False

CLASS_GET_ITEM_TEMPLATE = """
@classmethod
def __class_getitem__(cls, item):
return cls
"""

def easy_class_getitem_inference(node, context=None):
# Here __class_getitem__ exists but is quite a mess to infer thus
# put instead an easy inference tip
func_to_add = astroid.extract_node(CLASS_GET_ITEM_TEMPLATE)
node.locals["__class_getitem__"] = [func_to_add]

if PY39:
# Starting with Python39 some objects of the collection module are subscriptable
# thanks to the __class_getitem__ method but the way it is implemented in
# _collection_abc makes it difficult to infer. (We would have to handle AssignName inference in the
# getitem method of the ClassDef class) Instead we put here a mock of the __class_getitem__ method
astroid.MANAGER.register_transform(
astroid.nodes.ClassDef, easy_class_getitem_inference, _looks_like_subscriptable
)
hippo91 marked this conversation as resolved.
Show resolved Hide resolved
15 changes: 14 additions & 1 deletion astroid/scoped_nodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -2617,7 +2617,20 @@ def getitem(self, index, context=None):
try:
methods = dunder_lookup.lookup(self, "__getitem__")
except exceptions.AttributeInferenceError as exc:
raise exceptions.AstroidTypeError(node=self, context=context) from exc
if isinstance(self, ClassDef):
# subscripting a class definition may be
# achieved thanks to __class_getitem__ method
# which is a classmethod defined in the class
# that supports subscript and not in the metaclass
try:
methods = self.getattr("__class_getitem__")
# Here it is assumed that the __class_getitem__ node is
# a FunctionDef. One possible improvment would be to deal
# with more generic inference.
except exceptions.AttributeInferenceError:
raise exceptions.AstroidTypeError(node=self, context=context) from exc
else:
raise exceptions.AstroidTypeError(node=self, context=context) from exc

method = methods[0]

Expand Down
52 changes: 12 additions & 40 deletions tests/unittest_brain.py
Original file line number Diff line number Diff line change
Expand Up @@ -1211,25 +1211,18 @@ class CustomTD(TypedDict):
assert len(typing_module.locals["TypedDict"]) == 1
assert inferred_base == typing_module.locals["TypedDict"][0]

@staticmethod
def check_metaclass_is_abc(node: nodes.ClassDef):
meta = node.metaclass()
assert isinstance(meta, nodes.ClassDef)
assert meta.name == "ABCMeta"

@test_utils.require_version("3.8")
def test_typing_alias_type(self):
"""
Test that the type aliased thanks to typing._alias function are
correctly inferred.
"""

def check_metaclass(node: nodes.ClassDef):
meta = node.metaclass()
assert isinstance(meta, nodes.ClassDef)
assert meta.name == "ABCMeta_typing"
assert "ABCMeta" == meta.basenames[0]
assert meta.locals.get("__getitem__") is not None

abc_meta = next(meta.bases[0].infer())
assert isinstance(abc_meta, nodes.ClassDef)
assert abc_meta.name == "ABCMeta"
assert abc_meta.locals.get("__getitem__") is None

node = builder.extract_node(
"""
from typing import TypeVar, MutableSet
Expand All @@ -1242,12 +1235,11 @@ class Derived1(MutableSet[T]):
"""
)
inferred = next(node.infer())
check_metaclass(inferred)
self.check_metaclass_is_abc(inferred)
assertEqualMro(
inferred,
[
"Derived1",
"MutableSet_typing",
"MutableSet",
"Set",
"Collection",
Expand All @@ -1266,36 +1258,17 @@ class Derived2(typing.OrderedDict[int, str]):
"""
)
inferred = next(node.infer())
check_metaclass(inferred)
self.assertIsNone(inferred.metaclass())
assertEqualMro(
inferred,
[
"Derived2",
"OrderedDict_typing",
"OrderedDict",
"dict",
"object",
],
)

node = builder.extract_node(
"""
import typing
class Derived3(typing.Pattern[str]):
pass
"""
)
inferred = next(node.infer())
check_metaclass(inferred)
assertEqualMro(
inferred,
[
"Derived3",
"Pattern",
"object",
],
)

@test_utils.require_version("3.8")
def test_typing_alias_side_effects(self):
"""Test that typing._alias changes doesn't have unwanted consequences."""
Expand All @@ -1309,15 +1282,14 @@ class Derived(collections.abc.Iterator[int]):
"""
)
inferred = next(node.infer())
assert inferred.metaclass() is None # Should this be ABCMeta?
self.check_metaclass_is_abc(inferred)
assertEqualMro(
inferred,
[
"Derived",
# Should this be more?
# "Iterator_typing"?
# "Iterator",
# "object",
"Iterator",
"Iterable",
"object",
],
)

Expand Down