Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-35224: PEP 572 Implementation #10497

Merged
merged 67 commits into from Jan 24, 2019
Merged
Changes from all commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
11e9898
Add tokenization of :=
emilyemorehouse Jul 21, 2018
9e9156c
Add initial usage of := in grammar.
emilyemorehouse Jul 22, 2018
6eb766d
Update Python.asdl to match the grammar updates. Regenerated Include/…
emilyemorehouse Jul 22, 2018
4f42586
Update AST and compiler files in Python/ast.c and Python/compile.c. B…
emilyemorehouse Jul 22, 2018
e049ffa
Regenerate Lib/symbol.py using `./python Lib/symbol.py`
emilyemorehouse Jul 22, 2018
0b6bd95
Tests - Fix failing tests in test_parser.py due to changes in token n…
emilyemorehouse Jul 22, 2018
24e95fd
Tests - Add simple test for := token
emilyemorehouse Jul 23, 2018
d31135f
Tests - Add simple tests for named expressions using expr and suite
emilyemorehouse Jul 23, 2018
e19a900
Tests - Update number of levels for nested expressions to prevent sta…
emilyemorehouse Jul 23, 2018
aca4858
Update symbol table to handle NamedExpr
emilyemorehouse Aug 10, 2018
2c7b01e
Update Grammar to allow assignment expressions in if statements.
emilyemorehouse Aug 10, 2018
f0dad89
Tests - Add additional tests for named expressions in RoundtripLegalS…
emilyemorehouse Aug 31, 2018
fbea15f
Tests - Add temporary syntax test failure tests in test_parser.py
emilyemorehouse Sep 10, 2018
3cd271b
Add support for allowing assignment expressions as function argument …
emilyemorehouse Sep 10, 2018
7b34033
Tests - Move existing syntax tests out of test_parser.py and into tes…
emilyemorehouse Sep 10, 2018
13671f7
Add TargetScopeError exception to extend SyntaxError
emilyemorehouse Sep 11, 2018
5e4ba89
Tests - Update tests per PEP 572
emilyemorehouse Sep 11, 2018
c1c76ea
Documentation - Small updates to XXX/todo comments
emilyemorehouse Sep 11, 2018
0ebccb4
Fix assert in seq_for_testlist()
emilyemorehouse Sep 11, 2018
5500849
Cleanup - Denote "Not implemented -- No keyword args" on failing test…
emilyemorehouse Sep 11, 2018
38dfaa2
Tests - Wrap all file opens in `with...as` to ensure files are closed
emilyemorehouse Sep 11, 2018
3aaddc4
WIP: handle f(a := 1)
emilyemorehouse Sep 11, 2018
5d525a9
Tests and Cleanup - No longer skips keyword arg test. Keyword arg tes…
emilyemorehouse Sep 11, 2018
32dc443
Tests - Refactor last remaining test case that relied on on external …
emilyemorehouse Sep 11, 2018
d806088
Tests - Add better description of remaning skipped tests. Add test ch…
emilyemorehouse Sep 11, 2018
c1469bf
Tests - Add test for nested comprehension, testing value and scope. F…
emilyemorehouse Sep 11, 2018
e4e63ed
Handle restriction of LHS for named expressions - can only assign to …
emilyemorehouse Sep 12, 2018
db936c7
Tests - Update negative test case for assigning to lambda to match ne…
emilyemorehouse Sep 12, 2018
47e7367
Tests - Reorder test cases to group invalid syntax cases and named as…
emilyemorehouse Sep 12, 2018
787068e
Tests - Update test case for named expression in function argument - …
emilyemorehouse Sep 12, 2018
e21f5c8
Todo - Add todo for TargetScopeError based on Guido's comment (https:…
emilyemorehouse Sep 12, 2018
acd0f74
Tests - Add named expression tests for assignment operator in functio…
emilyemorehouse Sep 12, 2018
d0dd983
Add NamedStore to expr_context. Regenerate related code with `make re…
emilyemorehouse Sep 13, 2018
9b136d9
Add usage of NamedStore to ast_for_named_expr in ast.c. Update occura…
emilyemorehouse Sep 13, 2018
8fc5c99
Add ste_comprehension to _symtable_entry to track if the namespace is…
emilyemorehouse Sep 13, 2018
538dea4
s/symtable_add_def/symtable_add_def_helper. Add symtable_add_def to h…
emilyemorehouse Sep 13, 2018
ab34b0b
Refactor symtable_record_directive to take lineno and col_offset as a…
emilyemorehouse Sep 13, 2018
c04a08d
Handle elevating scope for named expressions in comprehensions.
emilyemorehouse Sep 14, 2018
f0297f8
Handle error for usage of named expression inside a class block
emilyemorehouse Sep 14, 2018
ee6d789
Tests - No longer skip scope tests. Add additional scope tests
emilyemorehouse Sep 14, 2018
c37e3c1
Cleanup - Update error message for named expression within a comprehe…
emilyemorehouse Sep 14, 2018
1db3fa9
Cleanup - Add missing case for NamedStore in expr_context_name. Remov…
emilyemorehouse Sep 14, 2018
0e29038
Refactor - Consolidate set_context and set_namedexpr_context to reduc…
emilyemorehouse Sep 14, 2018
2ff5744
Cleanup - Add additional use cases for ast_for_namedexpr in usage com…
emilyemorehouse Sep 14, 2018
2494008
Tests - Remove unnecessary test case. Renumber test case function names
emilyemorehouse Sep 14, 2018
794a1d1
Remove TargetScopeError for now. Will add back if needed
emilyemorehouse Sep 14, 2018
415d469
Cleanup - Small comment nit for consistency
emilyemorehouse Sep 14, 2018
162c8c4
Handle positional argument check with named expression
emilyemorehouse Sep 14, 2018
68e9e98
Add TargetScopeError exception definition. Add documentation for Targ…
emilyemorehouse Sep 14, 2018
1ad4b30
Increase stack size for parser by 200. This is a minimal change (appr…
emilyemorehouse Sep 14, 2018
1b1d84d
Add TargetScopeError to exception_hierarchy.txt for test_baseexceptio…
emilyemorehouse Sep 14, 2018
b80108b
Tests - Major update for named expression tests, both in test_named_e…
emilyemorehouse Sep 14, 2018
6e15f5f
Cleanup - Remove unnecessary comment
emilyemorehouse Sep 14, 2018
da11136
Cleanup - Comment nitpicks
emilyemorehouse Sep 14, 2018
051ad8c
Explicitly disallow assignment expressions to a name inside parenthes…
emilyemorehouse Sep 15, 2018
f893035
Cleanup - Wrap lines more strictly in test file
emilyemorehouse Sep 25, 2018
16d7137
Revert "Explicitly disallow assignment expressions to a name inside p…
emilyemorehouse Nov 13, 2018
070983b
Add NEWS.d entry
emilyemorehouse Nov 13, 2018
5133800
Tests - Fix error in test_pickle.test_exceptions by adding TargetScop…
emilyemorehouse Nov 13, 2018
bc2fed3
Tests - Update error message tests to reflect improved messaging conv…
emilyemorehouse Nov 20, 2018
f1cabf8
Remove cases that cannot be reached in compile.c. Small linting update.
emilyemorehouse Nov 27, 2018
0e0ca07
Update Grammar/Tokens to add COLONEQUAL. Regenerate all files
emilyemorehouse Jan 22, 2019
1153aae
Update TargetScopeError PRE_INIT and POST_INIT, as this was purposefu…
emilyemorehouse Jan 24, 2019
7775b0d
Add NamedStore back and regenerate files
emilyemorehouse Jan 24, 2019
10a71f8
Pass along line number and end col info for named expression
emilyemorehouse Jan 24, 2019
2b9a7bc
Simplify News entry
emilyemorehouse Jan 24, 2019
a63bf06
Fix compiler warning and explicity mark fallthrough
emilyemorehouse Jan 24, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.

Always

Just for now

@@ -800,6 +800,7 @@ the variables:
single: PyExc_SystemError
single: PyExc_SystemExit
single: PyExc_TabError
single: PyExc_TargetScopeError
single: PyExc_TimeoutError
single: PyExc_TypeError
single: PyExc_UnboundLocalError
@@ -901,6 +902,8 @@ the variables:
+-----------------------------------------+---------------------------------+----------+
| :c:data:`PyExc_TabError` | :exc:`TabError` | |
+-----------------------------------------+---------------------------------+----------+
| :c:data:`PyExc_TargetScopeError` | :exc:`TargetScopeError` | |
+-----------------------------------------+---------------------------------+----------+
| :c:data:`PyExc_TimeoutError` | :exc:`TimeoutError` | |
+-----------------------------------------+---------------------------------+----------+
| :c:data:`PyExc_TypeError` | :exc:`TypeError` | |

Some generated files are not rendered by default. Learn more.

@@ -69,7 +69,7 @@ assert_stmt: 'assert' test [',' test]

compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | with_stmt | funcdef | classdef | decorated | async_stmt
async_stmt: 'async' (funcdef | with_stmt | for_stmt)
if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite]
if_stmt: 'if' namedexpr_test ':' suite ('elif' namedexpr_test ':' suite)* ['else' ':' suite]
while_stmt: 'while' test ':' suite ['else' ':' suite]
for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite]
try_stmt: ('try' ':' suite
@@ -83,6 +83,7 @@ with_item: test ['as' expr]
except_clause: 'except' [test ['as' NAME]]
suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT

namedexpr_test: test [':=' test]
test: or_test ['if' or_test 'else' test] | lambdef
test_nocond: or_test | lambdef_nocond
lambdef: 'lambda' [varargslist] ':' test
@@ -108,7 +109,7 @@ atom: ('(' [yield_expr|testlist_comp] ')' |
'[' [testlist_comp] ']' |
'{' [dictorsetmaker] '}' |
NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False')
testlist_comp: (test|star_expr) ( comp_for | (',' (test|star_expr))* [','] )
testlist_comp: (namedexpr_test|star_expr) ( comp_for | (',' (namedexpr_test|star_expr))* [','] )

This comment has been minimized.

Copy link
@serhiy-storchaka

serhiy-storchaka Nov 20, 2018

Member

Does not [x := 1, 2] look too ambiguous? Why this syntax is allowed? {x := 1, 2}, {x := 1 : 2} and {1 : x := 2} are not allowed.

trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME
subscriptlist: subscript (',' subscript)* [',']
subscript: test | [test] ':' [test] [sliceop]
@@ -134,6 +135,7 @@ arglist: argument (',' argument)* [',']
# multiple (test comp_for) arguments are blocked; keyword unpackings
# that precede iterable unpackings are blocked; etc.
argument: ( test [comp_for] |
test ':=' test |

This comment has been minimized.

Copy link
@serhiy-storchaka

serhiy-storchaka Nov 20, 2018

Member

This looks like a bug magnet: f(x=1) vs f(x:=1). Is this desired?

test '=' test |
'**' test |
'*' test )
@@ -52,6 +52,7 @@ AT '@'
ATEQUAL '@='
RARROW '->'
ELLIPSIS '...'
COLONEQUAL ':='

OP
ERRORTOKEN

Some generated files are not rendered by default. Learn more.

Some generated files are not rendered by default. Learn more.

@@ -108,6 +108,7 @@ PyAPI_DATA(PyObject *) PyExc_NotImplementedError;
PyAPI_DATA(PyObject *) PyExc_SyntaxError;
PyAPI_DATA(PyObject *) PyExc_IndentationError;
PyAPI_DATA(PyObject *) PyExc_TabError;
PyAPI_DATA(PyObject *) PyExc_TargetScopeError;
PyAPI_DATA(PyObject *) PyExc_ReferenceError;
PyAPI_DATA(PyObject *) PyExc_SystemError;
PyAPI_DATA(PyObject *) PyExc_SystemExit;
@@ -50,6 +50,7 @@ typedef struct _symtable_entry {
including free refs to globals */
unsigned ste_generator : 1; /* true if namespace is a generator */
unsigned ste_coroutine : 1; /* true if namespace is a coroutine */
unsigned ste_comprehension : 1; /* true if namespace is a list comprehension */
unsigned ste_varargs : 1; /* true if block has varargs */
unsigned ste_varkeywords : 1; /* true if block has varkeywords */
unsigned ste_returns_value : 1; /* true if namespace uses return with

Some generated files are not rendered by default. Learn more.

@@ -128,6 +128,7 @@
"SystemError",
"SystemExit",
"TabError",
"TargetScopeError",
"TypeError",
"UnboundLocalError",
"UnicodeDecodeError",
@@ -61,44 +61,45 @@
with_item = 302
except_clause = 303
suite = 304
test = 305
test_nocond = 306
lambdef = 307
lambdef_nocond = 308
or_test = 309
and_test = 310
not_test = 311
comparison = 312
comp_op = 313
star_expr = 314
expr = 315
xor_expr = 316
and_expr = 317
shift_expr = 318
arith_expr = 319
term = 320
factor = 321
power = 322
atom_expr = 323
atom = 324
testlist_comp = 325
trailer = 326
subscriptlist = 327
subscript = 328
sliceop = 329
exprlist = 330
testlist = 331
dictorsetmaker = 332
classdef = 333
arglist = 334
argument = 335
comp_iter = 336
sync_comp_for = 337
comp_for = 338
comp_if = 339
encoding_decl = 340
yield_expr = 341
yield_arg = 342
namedexpr_test = 305
test = 306
test_nocond = 307
lambdef = 308
lambdef_nocond = 309
or_test = 310
and_test = 311
not_test = 312
comparison = 313
comp_op = 314
star_expr = 315
expr = 316
xor_expr = 317
and_expr = 318
shift_expr = 319
arith_expr = 320
term = 321
factor = 322
power = 323
atom_expr = 324
atom = 325
testlist_comp = 326
trailer = 327
subscriptlist = 328
subscript = 329
sliceop = 330
exprlist = 331
testlist = 332
dictorsetmaker = 333
classdef = 334
arglist = 335
argument = 336
comp_iter = 337
sync_comp_for = 338
comp_for = 339
comp_if = 340
encoding_decl = 341
yield_expr = 342
yield_arg = 343
#--end constants--

sym_name = {}
@@ -42,6 +42,7 @@ BaseException
| +-- NotImplementedError
| +-- RecursionError
+-- SyntaxError
| +-- TargetScopeError
| +-- IndentationError
| +-- TabError
+-- SystemError
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.