Truncate SyntaxErrors before newline character

[MichaReiser]

Fixes #3520

Before

error: Failed to parse  C:\Users\Micha\Downloads\fix_except0.py:39:14: invalid syntax. Got unexpected token """
    try_stmt< 'try' ':' (simple_stmt | suite)
                  cleanup=(except_clause ':' (simple_stmt | suite))+
                  tail=(['except' ':' (simple_stmt | suite)]
                        ['else' ':' (simple_stmt | suite)]
                        ['finally' ':' (simple_stmt | suite)]) >
    """
C:\Users\Micha\Downloads\fix_except0.py:39:14: E999 SyntaxError: invalid syntax. Got unexpected token """
    try_stmt< 'try' ':' (simple_stmt | suite)
                  cleanup=(except_clause ':' (simple_stmt | suite))+
                  tail=(['except' ':' (simple_stmt | suite)]
                        ['else' ':' (simple_stmt | suite)]
                        ['finally' ':' (simple_stmt | suite)]) >
    """
   |
39 |     BM_compatible = True
40 |
41 |     PATTERN  """
   |              ^ E999
42 |     try_stmt< 'try' ':' (simple_stmt | suite)
43 |                   cleanup=(except_clause ':' (simple_stmt | suite))+
   |

Found 1 error.

After

error: Failed to parse C:\Users\Micha\Downloads\fix_except0.py:39:14: unexpected token """⏎...
C:\Users\Micha\Downloads\fix_except0.py:39:14: E999 SyntaxError: unexpected token """⏎...
   |
39 |     BM_compatible = True
40 |
41 |     PATTERN  """
   |              ^ E999
42 |     try_stmt< 'try' ':' (simple_stmt | suite)
43 |                   cleanup=(except_clause ':' (simple_stmt | suite))+
   |

Found 1 error.

I decided to implement this in Ruff rather than RustPython becuse this behaviour must only apply in a context where each error should only span a single line.

Ideally, the parser emit the full range for the token so that we can proberly highlight it in the diagnostic and use the Token text as it appears in the source document (RustPython assumes " even if the source text uses ').

Alternatives

An alternative would be to escape newlines. I decided not to escape newlines because it can lead to extremelly long lines in the output.

[MichaReiser]

Current dependencies on/for this PR:

• main
  • PR Truncate SyntaxErrors before newline character #4124 👈

This comment was auto-generated by Graphite.

[MichaReiser]

I copied this from RustPython but wrapped all tok variables.

[github-actions]

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

[charliermarsh]

Sorry, I looked at this before, but I felt like I was missing the context and got side-tracked. Does this not have the potential to confuse by way of emitting partial error messages? Why truncate at the first newline?

[MichaReiser]

Sorry, I looked at this before, but I felt like I was missing the context and got side-tracked. Does this not have the potential to confuse by way of emitting partial error messages? Why truncate at the first newline?

It only truncates the token text, not the message itself. I guess, it could be confusing if the unexpected token was a newline, but I think that would already have been confusing today, because Ruff doesn't use special formatting for whitespace.

[charliermarsh]

Sounds reasonable -- go for it.