Highlight invalid ranges in SyntaxErrors

[pablogsal]

BPO	43914
Nosy	@terryjreedy, @aroberge, @merwok, @ambv, @lysnikolaou, @pablogsal, @miss-islington
PRs	bpo-43914: Highlight invalid ranges in SyntaxErrors #25525 bpo-43914: Add new SyntaxError attributes to What's New #25582 bpo-43914: Whats New 310: add new SyntaxError attributes #28558 [3.10] bpo-43914: What's New 3.10: add new SyntaxError attributes (GH-28558) #28562 bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls #28576 [3.10] bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls (GH-28576) #28582

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-09-25.09:33:00.119>
created_at = <Date 2021-04-22.18:08:10.767>
labels = ['interpreter-core', '3.10', '3.11']
title = 'Highlight invalid ranges in SyntaxErrors'
updated_at = <Date 2021-10-04.19:18:42.967>
user = 'https://github.com/pablogsal'

bugs.python.org fields:

activity = <Date 2021-10-04.19:18:42.967>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-25.09:33:00.119>
closer = 'lukasz.langa'
components = ['Parser']
creation = <Date 2021-04-22.18:08:10.767>
creator = 'pablogsal'
dependencies = []
files = []
hgrepos = []
issue_num = 43914
keywords = ['patch']
message_count = 10.0
messages = ['391620', '391697', '391729', '391808', '402611', '402615', '402616', '402711', '402713', '403163']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'aroberge', 'eric.araujo', 'lukasz.langa', 'lys.nikolaou', 'pablogsal', 'miss-islington']
pr_nums = ['25525', '25582', '28558', '28562', '28576', '28582']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue43914'
versions = ['Python 3.10', 'Python 3.11']

[pablogsal]

To improve the user experience understanding what part of the error messages associated to SyntaxErrors are wrong, we can highlight the whole error range and not only place the caret at the first character. In this way:

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^
SyntaxError: Generator expression must be parenthesized

becomes

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized

[pablogsal]

New changeset a77aac4 by Pablo Galindo in branch 'master':
bpo-43914: Highlight invalid ranges in SyntaxErrors (bpo-25525)
a77aac4

[merwok]

What a great, useful change!

[terryjreedy]

Great!!!  I also tried
>>> try: compile("a xyzjdkjfk", '', 'single')
except SyntaxError as e:
	print(e, e.msg, e.lineno, e.offset, e.end_lineno, e.end_offset)

invalid syntax. Perhaps you forgot a comma? (, line 1) invalid syntax. Perhaps you forgot a comma? 1 1 1 12

The language change that enables the display change is the addition of end_lineno and end_offset attributes. I will look into using these in IDLE. I will submit a PR to augment the What's New entry, so others can also improve their error marking.

[ambv]

New changeset 71f8ff4 by Terry Jan Reedy in branch 'main':
bpo-43914: Whats New 310: add new SyntaxError attributes (GH-28558)
71f8ff4

[ambv]

New changeset c523022 by Łukasz Langa in branch '3.10':
[3.10] bpo-43914: What's New 3.10: add new SyntaxError attributes (GH-28558) (GH-28562)
c523022

[ambv]

This is complete now. Thanks! ✨ 🍰 ✨

[pablogsal]

New changeset e5f13ce by Pablo Galindo Salgado in branch 'main':
bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls (GH-28576)
e5f13ce

[miss-islington]

New changeset 9e209d4 by Miss Islington (bot) in branch '3.10':
bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls (GH-28576)
9e209d4

[pablogsal]

New changeset 0eb57c3 by Pablo Galindo (Łukasz Langa) in branch '3.10':
[3.10] bpo-43914: What's New 3.10: add new SyntaxError attributes (GH-28558) (GH-28562)
0eb57c3

New changeset 3397e31 by Pablo Galindo (Miss Islington (bot)) in branch '3.10':
bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls (GH-28576)
3397e31