New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support PEP 572 #6899
Merged
Merged
Support PEP 572 #6899
Changes from 7 commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
264069e
make basic things work
JelleZijlstra e765e32
more tests
JelleZijlstra 4a46238
add tests back (file got renamed)
JelleZijlstra 81b2d4a
support hoisting for comprehensions (old semanal)
JelleZijlstra 8a08ed2
fix under the new semanal
JelleZijlstra e3b3ecd
a few more tests
JelleZijlstra 6eff366
fix lint and self-check
JelleZijlstra 0f382bb
address Jukka's comments
JelleZijlstra 7c184d2
support Final, check the binder
JelleZijlstra 7e2eaf1
partial type test
JelleZijlstra 91a87e4
fix mypyc crash
JelleZijlstra ee0bc14
Merge remote-tracking branch 'upstream/master' into walrus
JelleZijlstra 90d2c76
fix merge conflict for real
JelleZijlstra c3c44a1
more tests
JelleZijlstra 8f050fe
more tests
JelleZijlstra 40d0c4a
comment on why it's broken
JelleZijlstra 050f1ff
Merge branch 'master' into walrus
JelleZijlstra 0056e1f
Merge branch 'walrus' of github.com:JelleZijlstra/mypy into walrus
JelleZijlstra 44bb3ec
fix some reveal_type Notes
JelleZijlstra d10efb5
fix test failure
JelleZijlstra ca36e90
Revert "fix test failure"
JelleZijlstra ecb66d3
Merge remote-tracking branch 'upstream/master' into walrus
JelleZijlstra 7ae2f35
fix test failure
JelleZijlstra adb6b8c
Merge branch 'master' into walrus
JelleZijlstra fdbd668
Merge remote-tracking branch 'upstream/master' into walrus
JelleZijlstra 08c926b
fix bad merge
JelleZijlstra File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -73,7 +73,7 @@ | |
PlaceholderNode, COVARIANT, CONTRAVARIANT, INVARIANT, | ||
nongen_builtins, get_member_expr_fullname, REVEAL_TYPE, | ||
REVEAL_LOCALS, is_final_node, TypedDictExpr, type_aliases_target_versions, | ||
EnumCallExpr | ||
EnumCallExpr, AssignmentExpr, | ||
) | ||
from mypy.tvar_scope import TypeVarScope | ||
from mypy.typevars import fill_typevars | ||
|
@@ -186,6 +186,8 @@ class NewSemanticAnalyzer(NodeVisitor[None], | |
nonlocal_decls = None # type: List[Set[str]] | ||
# Local names of function scopes; None for non-function scopes. | ||
locals = None # type: List[Optional[SymbolTable]] | ||
# Whether each scope is a comprehension scope. | ||
is_comprehension_stack = None # type: List[bool] | ||
# Nested block depths of scopes | ||
block_depth = None # type: List[int] | ||
# TypeInfo of directly enclosing class (or None) | ||
|
@@ -240,6 +242,7 @@ def __init__(self, | |
errors: Report analysis errors using this instance | ||
""" | ||
self.locals = [None] | ||
self.is_comprehension_stack = [False] | ||
# Saved namespaces from previous iteration. Every top-level function/method body is | ||
# analyzed in several iterations until all names are resolved. We need to save | ||
# the local namespaces for the top level function and all nested functions between | ||
|
@@ -510,9 +513,14 @@ def file_context(self, | |
# | ||
|
||
def visit_func_def(self, defn: FuncDef) -> None: | ||
# Visit default values because they may contain assignment expressions. | ||
for arg in defn.arguments: | ||
if arg.initializer: | ||
arg.initializer.accept(self) | ||
|
||
defn.is_conditional = self.block_depth[-1] > 0 | ||
|
||
# Set full names even for those definitionss that aren't added | ||
# Set full names even for those definitions that aren't added | ||
# to a symbol table. For example, for overload items. | ||
defn._fullname = self.qualified_name(defn.name()) | ||
|
||
|
@@ -1120,13 +1128,15 @@ def enter_class(self, info: TypeInfo) -> None: | |
# Remember previous active class | ||
self.type_stack.append(self.type) | ||
self.locals.append(None) # Add class scope | ||
self.is_comprehension_stack.append(False) | ||
self.block_depth.append(-1) # The class body increments this to 0 | ||
self.type = info | ||
|
||
def leave_class(self) -> None: | ||
""" Restore analyzer state. """ | ||
self.block_depth.pop() | ||
self.locals.pop() | ||
self.is_comprehension_stack.pop() | ||
self.type = self.type_stack.pop() | ||
|
||
def analyze_class_decorator(self, defn: ClassDef, decorator: Expression) -> None: | ||
|
@@ -1831,6 +1841,10 @@ def visit_import_all(self, i: ImportAll) -> None: | |
# Assignment | ||
# | ||
|
||
def visit_assignment_expr(self, s: AssignmentExpr) -> None: | ||
s.value.accept(self) | ||
self.analyze_lvalue(s.target, escape_comprehensions=True) | ||
|
||
def visit_assignment_stmt(self, s: AssignmentStmt) -> None: | ||
tag = self.track_incomplete_refs() | ||
s.rvalue.accept(self) | ||
|
@@ -2396,16 +2410,22 @@ def check_and_set_up_type_alias(self, s: AssignmentStmt) -> bool: | |
|
||
def analyze_lvalue(self, lval: Lvalue, nested: bool = False, | ||
explicit_type: bool = False, | ||
is_final: bool = False) -> None: | ||
is_final: bool = False, | ||
escape_comprehensions: bool = False) -> None: | ||
"""Analyze an lvalue or assignment target. | ||
|
||
Args: | ||
lval: The target lvalue | ||
nested: If true, the lvalue is within a tuple or list lvalue expression | ||
explicit_type: Assignment has type annotation | ||
escape_comprehensions: If we are inside a comprehension, set the variable | ||
in the enclosing scope instead. This implements | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Style nit: indent the second and following lines in an argument specification like this:
|
||
https://www.python.org/dev/peps/pep-0572/#scope-of-the-target | ||
""" | ||
if escape_comprehensions: | ||
assert isinstance(lval, NameExpr), "assignment expression target must be NameExpr" | ||
if isinstance(lval, NameExpr): | ||
self.analyze_name_lvalue(lval, explicit_type, is_final) | ||
self.analyze_name_lvalue(lval, explicit_type, is_final, escape_comprehensions) | ||
elif isinstance(lval, MemberExpr): | ||
self.analyze_member_lvalue(lval, explicit_type, is_final) | ||
if explicit_type and not self.is_self_member_ref(lval): | ||
|
@@ -2431,7 +2451,8 @@ def analyze_lvalue(self, lval: Lvalue, nested: bool = False, | |
def analyze_name_lvalue(self, | ||
lvalue: NameExpr, | ||
explicit_type: bool, | ||
is_final: bool) -> None: | ||
is_final: bool, | ||
escape_comprehensions: bool) -> None: | ||
"""Analyze an lvalue that targets a name expression. | ||
|
||
Arguments are similar to "analyze_lvalue". | ||
|
@@ -2455,7 +2476,7 @@ def analyze_name_lvalue(self, | |
if (not existing or isinstance(existing.node, PlaceholderNode)) and not outer: | ||
# Define new variable. | ||
var = self.make_name_lvalue_var(lvalue, kind, not explicit_type) | ||
added = self.add_symbol(name, var, lvalue) | ||
added = self.add_symbol(name, var, lvalue, escape_comprehensions=escape_comprehensions) | ||
# Only bind expression if we successfully added name to symbol table. | ||
if added: | ||
lvalue.is_new_def = True | ||
|
@@ -3948,7 +3969,8 @@ def add_symbol(self, | |
context: Context, | ||
module_public: bool = True, | ||
module_hidden: bool = False, | ||
can_defer: bool = True) -> bool: | ||
can_defer: bool = True, | ||
escape_comprehensions: bool = False) -> bool: | ||
"""Add symbol to the currently active symbol table. | ||
|
||
Generally additions to symbol table should go through this method or | ||
|
@@ -3970,7 +3992,7 @@ def add_symbol(self, | |
node, | ||
module_public=module_public, | ||
module_hidden=module_hidden) | ||
return self.add_symbol_table_node(name, symbol, context, can_defer) | ||
return self.add_symbol_table_node(name, symbol, context, can_defer, escape_comprehensions) | ||
|
||
def add_symbol_skip_local(self, name: str, node: SymbolNode) -> None: | ||
"""Same as above, but skipping the local namespace. | ||
|
@@ -3998,7 +4020,8 @@ def add_symbol_table_node(self, | |
name: str, | ||
symbol: SymbolTableNode, | ||
context: Optional[Context] = None, | ||
can_defer: bool = True) -> bool: | ||
can_defer: bool = True, | ||
escape_comprehensions: bool = False) -> bool: | ||
"""Add symbol table node to the currently active symbol table. | ||
|
||
Return True if we actually added the symbol, or False if we refused | ||
|
@@ -4017,7 +4040,7 @@ def add_symbol_table_node(self, | |
can_defer: if True, defer current target if adding a placeholder | ||
context: error context (see above about None value) | ||
""" | ||
names = self.current_symbol_table() | ||
names = self.current_symbol_table(escape_comprehensions=escape_comprehensions) | ||
existing = names.get(name) | ||
if isinstance(symbol.node, PlaceholderNode) and can_defer: | ||
self.defer() | ||
|
@@ -4250,13 +4273,16 @@ def enter(self, function: Union[FuncItem, GeneratorExpr, DictionaryComprehension | |
"""Enter a function, generator or comprehension scope.""" | ||
names = self.saved_locals.setdefault(function, SymbolTable()) | ||
self.locals.append(names) | ||
is_comprehension = isinstance(function, (GeneratorExpr, DictionaryComprehension)) | ||
self.is_comprehension_stack.append(is_comprehension) | ||
self.global_decls.append(set()) | ||
self.nonlocal_decls.append(set()) | ||
# -1 since entering block will increment this to 0. | ||
self.block_depth.append(-1) | ||
|
||
def leave(self) -> None: | ||
self.locals.pop() | ||
self.is_comprehension_stack.pop() | ||
self.global_decls.pop() | ||
self.nonlocal_decls.pop() | ||
self.block_depth.pop() | ||
|
@@ -4283,10 +4309,19 @@ def current_symbol_kind(self) -> int: | |
kind = GDEF | ||
return kind | ||
|
||
def current_symbol_table(self) -> SymbolTable: | ||
def current_symbol_table(self, escape_comprehensions: bool = False) -> SymbolTable: | ||
if self.is_func_scope(): | ||
assert self.locals[-1] is not None | ||
names = self.locals[-1] | ||
if escape_comprehensions: | ||
for i, is_comprehension in enumerate(reversed(self.is_comprehension_stack)): | ||
if not is_comprehension: | ||
names = self.locals[-1 - i] | ||
break | ||
else: | ||
assert False, "Should have at least one non-comprehension scope" | ||
else: | ||
names = self.locals[-1] | ||
assert names is not None | ||
elif self.type is not None: | ||
names = self.type.names | ||
else: | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to me this direct call will result in allowing assignments to final variables.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, I'll add
self.chk.check_final
.