gh-106922: Support multi-line error locations in traceback

[williamwen42]

Complete implementation for #106922. Implementing (1) requires substantial changes for an implementation of (2) or (3), so I just did it all in one go.

Some changes compared to the initial issue:
We do not always output all lines that correspond to an error:

• If locations are missing, we only output the first line (missing locations assumes we only want to output one line)
• If there are too many lines to output, in order to declutter output, we only output "significant lines" - the first and last lines, and the beginning and the end of the anchors, and 1 line above and below these lines (see tests for examples, e.g. test_many_lines_binary_op).

Order of looking things in the PR:

1. traceback.py

• Changes to FrameSummary fields
• Changes to the traceback printing algorithm, primarily format_frame_summary and _extract_caret_anchors_from_line_segment - comments should sufficiently explain the algorithm

2. traceback.c

• Algorithm essentially mirrors traceback.py - comments should sufficiently explain the algorithm
• I needed to write my own implementation for dedent and for tracking significant lines (struct SignificantLines i.e. a set). If there are existing implementations, then I would be happy to use those instead.

3. test_traceback.c

• Change some tests to check for multi-line outputs. Add some new multi-line tests.

4. Other testing files

• Change some tests to check for multi-line outputs.

cc @pablogsal @isidentical @ammaraskar @iritkatriel @ezyang

• Issue: Support multi-line error locations in traceback and other related improvements (PEP-657, 3.11) #106922

[cpython-cla-bot]

All commit authors signed the Contributor License Agreement.

[bedevere-app]

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

[bedevere-app]

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

[pablogsal]

Thanks for the PR @williamwen42! I will review this shortly but two initial comments:

• I think you mistakenly commited two lib2to3 files.
• We are missing a NEWS entry, can you add one following these instructions
• I think we may be able to review everything in one go, but in general I would prefer separate PRs because big PRs are notoriously more difficult to review

[pablogsal]

I am also a bit concerned over how much C code the required implementation requires. I need to review it but my initial impression is that this may be a bit too much. What do others think? @isidentical @ammaraskar @iritkatriel

[bedevere-app]

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

[williamwen42]

@pablogsal any idea when you can take a look at this PR? There are some failing tests but those should just be small fixes that don't have a large impact on the PR. LMK if you would like an even more detailed description of the changes.

[pablogsal]

@williamwen42 We have refactored the whole traceback machinery so CPython uses mainly the traceback module (in Python) and falls back to the C implementation but we ripped out all the error locations in the C implementation so is easier to maintain. This means that we can remove the C parts of this PRs. You may have some conflicts on the Python part yet, but they shouldn't be too big.

I know you probably spent a lot of time doing the C part, but this change was involving a lot of C code that we woudn't be able to merge because the complexity outweighs what we are gaining from it, but given that now is only on the Python side, then we can consider it.

[pablogsal]

CC: @lysnikolaou

[terryjreedy]

Since 'attempt 2' is merged, close this?