test_global_err_then_warn in test_syntax is no longer valid

[serhiy-storchaka]

BPO	28936
Nosy	@gvanrossum, @serhiy-storchaka, @ilevkivskyi
PRs	bpo-28936: Detect lexically first syntax error first #4097
Files	syntax-error-order.patch syntax-error-order-v2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-10-26.21:29:58.937>
created_at = <Date 2016-12-11.12:33:10.639>
labels = ['3.7', 'tests']
title = 'test_global_err_then_warn in test_syntax is no longer valid'
updated_at = <Date 2017-10-26.21:29:58.936>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2017-10-26.21:29:58.936>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-10-26.21:29:58.937>
closer = 'serhiy.storchaka'
components = ['Tests']
creation = <Date 2016-12-11.12:33:10.639>
creator = 'serhiy.storchaka'
dependencies = []
files = ['45849', '45948']
hgrepos = []
issue_num = 28936
keywords = ['patch', '3.6regression']
message_count = 16.0
messages = ['282916', '282920', '283437', '283522', '283523', '304817', '304847', '304850', '304861', '304864', '304915', '305045', '305086', '305089', '305091', '305092']
nosy_count = 3.0
nosy_names = ['gvanrossum', 'serhiy.storchaka', 'levkivskyi']
pr_nums = ['4097']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue28936'
versions = ['Python 3.6', 'Python 3.7']

[serhiy-storchaka]

bpo-27999 invalidated test_global_err_then_warn in Lib/test/test_syntax.py:567. That test was added in bpo-763201 for testing that SyntaxWarning emitted in symtable_global() doesn't clobber a SyntaxError. In bpo-27999 a SyntaxWarning was converted to SyntaxError, and the test no longer serves its purpose.

bpo-28512 makes tests more strong (it tests the position of SyntaxError), but it can't be applied in 3.6, because _check_error() catches different SyntaxError.

[ilevkivskyi]

Serhiy, here is a patch that might be helpful. It detects global-and-parameter errors sooner (when possible). This will cause the following:

>>> if 1:
...     def error(a):
...         global a
...     def error2():
...         b = 1
...         global b
... 
  File "<stdin>", line 3
SyntaxError: name 'a' is parameter and global

However, in more complex (nested) cases, the global-after-assign will still be detected sooner:

>>> def error(a):
...     def inner():
...         global a
...     def inner2():
...         b = 1
...         global b
... 
  File "<stdin>", line 6
SyntaxError: name 'b' is assigned to before global declaration

Maybe there is a way to delay the detection of this error until second pass in symtable.c, but don't see now how to do this.

[serhiy-storchaka]

I don't know what should we do with this. Is there a bug that needs to be fixed? Or maybe the test can be just removed?

[ilevkivskyi]

I don't think this is a bug, but detecting first a SyntaxError that appears textually first might be seen as an improvement. I would say such behaviour seems more intuitive. A possible downside could be a (probably very minor) slow-down of compilation.

I don't have any strong opinion here.

[ilevkivskyi]

Updated patch as proposed by Serhiy.

[serhiy-storchaka]

Do you mind to create a pull request Ivan?

[ilevkivskyi]

OK, I made a PR with the fix and a test that checks the line number for syntax error (so that the original purpose test_global_err_then_warn is preserved).

[gvanrossum]

Am I needed here?

[ilevkivskyi]

Am I needed here?

As I understand, Serhiy is going to review the PR, so I think no.

[serhiy-storchaka]

Sorry for disturbing you Guido. I had added you as the committer of bpo-27999. Do you have thoughts about this issue?

[gvanrossum]

I think this is very minor but if you two can agree that the code is right I think it's a nice little improvement, and I like that that particular test's usefulness is restored.

PS. Long-term we should really build error recovery into our antiquated parser and report as many errors as we can, without cascading. But that's probably a Python 4 project.

[serhiy-storchaka]

Is it correct that the parameter can be annotated in the function body?

def f(x):
    x: int

Or that the local variable can be annotated after assignment?

def f():
    x = 1
    x: int

[gvanrossum]

Those seem things that the type checker should complain about, but I don't think Python should.

[ilevkivskyi]

Is it correct that the parameter can be annotated in the function body?

I agree with Guido, this is rather a task for type checkers.

[serhiy-storchaka]

New changeset 8c83c23 by Serhiy Storchaka (Ivan Levkivskyi) in branch 'master':
bpo-28936: Detect lexically first syntax error first (bpo-4097)
8c83c23

[serhiy-storchaka]

Thank you for your patch Ivan.