Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support exported names starting with an underscore in __all__ #3746

Merged
merged 3 commits into from
Jul 21, 2017
Merged

Support exported names starting with an underscore in __all__ #3746

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5aca8d8
Support exported names starting with an underscore in __all__
daboross Jul 20, 2017
86a085a
Undo whitespace change.
daboross Jul 20, 2017
74e42ff
Simplify logic to only modify a single section of the code.
daboross Jul 20, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion mypy/nodes.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2286,6 +2286,10 @@ class SymbolTableNode:
# If False, this name won't be imported via 'from <module> import *'.
# This has no effect on names within classes.
module_public = True
# If True, this name will be imported via 'from <module> import *'
# even if it starts with an underscore. This has no effect on names
# within classes nor names which do not start with an underscore.
module_public_even_with_underscore = False
# For deserialized MODULE_REF nodes, the referenced module name;
# for other nodes, optionally the name of the referenced object.
cross_ref = None # type: Optional[str]
Expand All @@ -2302,12 +2306,14 @@ def __init__(self,
module_public: bool = True,
normalized: bool = False,
alias_tvars: Optional[List[str]] = None,
implicit: bool = False) -> None:
implicit: bool = False,
module_public_even_with_underscore: bool = False) -> None:
self.kind = kind
self.node = node
self.type_override = typ
self.mod_id = mod_id
self.module_public = module_public
self.module_public_even_with_underscore = module_public_even_with_underscore
self.normalized = normalized
self.alias_tvars = alias_tvars
self.implicit = implicit
Expand Down Expand Up @@ -2354,6 +2360,8 @@ def serialize(self, prefix: str, name: str) -> JsonDict:
} # type: JsonDict
if not self.module_public:
data['module_public'] = False
if self.module_public_even_with_underscore:
data['module_public_even_with_underscore'] = True
if self.normalized:
data['normalized'] = True
if self.implicit:
Expand Down Expand Up @@ -2395,6 +2403,8 @@ def deserialize(cls, data: JsonDict) -> 'SymbolTableNode':
stnode.alias_tvars = data['alias_tvars']
if 'module_public' in data:
stnode.module_public = data['module_public']
if 'module_public_even_with_underscore' in data:
stnode.module_public_even_with_underscore = data['module_public_even_with_underscore']
if 'normalized' in data:
stnode.normalized = data['normalized']
if 'implicit' in data:
Expand Down
8 changes: 6 additions & 2 deletions mypy/semanal.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -301,7 +301,10 @@ def visit_file(self, file_node: MypyFile, fnam: str, options: Options,

if '__all__' in self.globals:
for name, g in self.globals.items():
if name not in self.all_exports:
if name in self.all_exports:
if name.startswith('_'):
g.module_public_even_with_underscore = True
else:
g.module_public = False

del self.options
Expand Down Expand Up @@ -1483,7 +1486,8 @@ def visit_import_all(self, i: ImportAll) -> None:
self.add_submodules_to_parent_modules(i_id, True)
for name, node in m.names.items():
node = self.normalize_type_alias(node, i)
if not name.startswith('_') and node.module_public:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why isn't it possible to fix this by only changing the logic here? For example add a nested if that ignores the check for underscore here if m.names contains '__all__' TBH, I don't really like inclusion of a new flag.

Copy link
Contributor Author

@daboross daboross Jul 20, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have access to __all__ here, not introducing a new flag does sound better! I wasn't sure of where we had access to what, so I was somewhat following how #1640 implemented __all__ originally.

Thank you for mentioning this, I'll change this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we don't actually need to access its content, we just need to know that it is there, since in this case all names not in __all__ will have module_public = False.

Copy link
Contributor Author

@daboross daboross Jul 20, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right! Awesome, that's even better than what I misread you as saying - thanks!

if ((node.module_public_even_with_underscore or not name.startswith('_'))
and node.module_public):
existing_symbol = self.globals.get(name)
if existing_symbol:
# Import can redefine a variable. They get special treatment.
Expand Down
3 changes: 2 additions & 1 deletion mypy/server/astdiff.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ def is_similar_node_shallow(n: SymbolTableNode, m: SymbolTableNode) -> bool:
# type_override
if (n.kind != m.kind
or n.mod_id != m.mod_id
or n.module_public != m.module_public):
or n.module_public != m.module_public
or n.module_public_even_with_underscore != m.module_public_even_with_underscore):
Copy link
Contributor Author

@daboross daboross Jul 20, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure of whether this should be included here or not, since it's affected by __all__ only. I would greatly appreciate if someone who has a better idea of the use of this function could comment on this.

return False
if type(n.node) != type(m.node): # noqa
return False
Expand Down
19 changes: 19 additions & 0 deletions test-data/unit/check-modules.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -427,6 +427,25 @@ __all__ = [u'a', u'b', u'c']

[out]

[case testUnderscoreExportedValuesInImportAll]
import typing
from m import *
_ = a
_ = _b
_ = __c__
_ = ___d
_ = e
_ = f # E: Name 'f' is not defined
_ = _g # E: Name '_g' is not defined
[file m.py]
__all__ = ['a']
__all__ += ('_b',)
__all__.append('__c__')
__all__.extend(('___d', 'e'))

a = _b = __c__ = ___d = e = f = _g = 1
[builtins fixtures/module_all.pyi]

[case testEllipsisInitializerInStubFileWithType]
import m
m.x = '' # E: Incompatible types in assignment (expression has type "str", variable has type "int")
Expand Down