Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support new ast features and node types introduced in Python 3.8 #2859

Merged
merged 2 commits into from Mar 12, 2019

Conversation

Projects
None yet
2 participants
@gvanrossum
Copy link
Member

commented Mar 11, 2019

Specifically:

  • New classes TypeIgnore, FunctionType, Constant and NamedExpr
  • New AST attributes end_lineno, end_col_offset, and type_comment
  • New attribute Module.type_ignores
  • New attribute Str.kind (that's new in 3.7 actually)
  • New keyword args for ast.parse(): type_comments=<bool> and feature_version=<int>

I had to adjust the return type of ast.parse() from Module to AST, which is more truthful anyways.

Support new ast features and node types introduced in Python 3.8
Had to adjust the return type of ast.parse() from Module to AST, which
is more truthful anyways.
@srittau
Copy link
Collaborator

left a comment

Thank you for the patch, but especially for incorporating these changes in the ast module, this will be very useful.

I'm not very versed in interpreting the ADSL or converting it to code, so please take my review with a big grain of salt.

col_offset: int # TODO: Not all nodes have this
if sys.version_info >= (3, 8):
end_lineno: int
end_col_offset: int # TODO: Not all nodes have this

This comment has been minimized.

Copy link
@srittau

srittau Mar 11, 2019

Collaborator

Shouldn't these two attributes be Optional?


class FunctionType(mod):
argtypes = ... # type: typing.List[expr]
returns = ... # type: expr

This comment has been minimized.

Copy link
@srittau

srittau Mar 11, 2019

Collaborator

I'd prefer variable annotation syntax for new code (even if existing code in the file still uses comment syntax). Applies to classes below as well.

Show resolved Hide resolved stdlib/3/_ast.pyi
@@ -253,6 +270,14 @@ class Bytes(expr):
class NameConstant(expr):
value = ... # type: Any

if sys.version_info >= (3, 8):
class Constant(expr):
value = ... # type: expr

This comment has been minimized.

Copy link
@srittau

srittau Mar 11, 2019

Collaborator

I guess this a an expr, because constants can be assigned an arbitrary expression? I mainly ask, because the old constant classes all had more concrete attribute types like 'str', 'float' (or Any in the case of NameConstant).

@gvanrossum

This comment has been minimized.

Copy link
Member Author

commented Mar 12, 2019

Thanks for the very thorough code review.

Note that I added Constant.kind, which isn't quite ready yet, but I am working on a PR.

@srittau srittau merged commit 6b6d8c8 into master Mar 12, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@gvanrossum gvanrossum deleted the ast-py38 branch Mar 12, 2019

gvanrossum added a commit to python/mypy that referenced this pull request Mar 15, 2019

In Python 3.8, use the stdlib ast instead of typed_ast (#6539)
To test with Python 3.8, this requires python/cpython#12295. The typeshed changes were in python/typeshed#2859 (and synced in #6540).

rchen152 added a commit to google/pytype that referenced this pull request Mar 21, 2019

Widen the return type of ast.parse in a Python 3 test.
python/typeshed#2859 changed the return type.
Module is a subclass of AST, so it's still correct, just less precise.
Based on the PR description, this change was intentional.

PiperOrigin-RevId: 239423111
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.