-
-
Notifications
You must be signed in to change notification settings - Fork 30.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tracebacks should contain the first line of continuation lines #56667
Comments
Currently, python tracebacks shows the last of continuation lines For example: Traceback (most recent call last):
File "./tcms-run", line 48, in <module>
summary=options.summary)
File "/tmp/nitrate/Nitrate.py", line 600, in __init__
raise NitrateError("Need either id or test plan") If the traceback contained the beginning of the continuation line Traceback (most recent call last):
File "./tcms-run", line 48, in <module>
run = TestRun(plan=plan, distro=options.distro,
File "/tmp/nitrate/Nitrate.py", line 600, in __init__
raise NitrateError("Need either id or test plan") Version: Both Python 2 and Python 3. Trivial reproducer: def fun1(par):
raise Exception
def fun2():
fun1(
par="value")
fun2() Actual results: Traceback (most recent call last):
File "/tmp/traceback.py", line 10, in <module>
fun2()
File "/tmp/traceback.py", line 8, in fun2
par="value")
File "/tmp/traceback.py", line 4, in fun1
raise Exception
Exception |
This is an interesting proposal. The line number comes from Python/traceback.c:120:
tb->tb_lineno = PyFrame_GetLineNumber(frame);
and this function is defined in Objects/frameobject.c:35:
int PyFrame_GetLineNumber(PyFrameObject *f) {
if (f->f_trace)
return f->f_lineno;
else
return PyCode_Addr2Line(f->f_code, f->f_lasti);
}
and documented as "Return the line number that frame is currently executing.", so that would explain why it's pointing to the last line. I'm not sure if there's an easy way to get the line where the beginning of the expression is, but if you find a way to get it, we could try to use it in PyFrame_GetLineNumber and see how it works. |
This is an enhancement request (at best), not a bug report. I would close it because I think it somewhat dubious, unreasonable, and not helpful. First, I think tracebacks *should* report 'the spot of the foul' for each call frame. What would sometimes be helpful when multiple operations are performed in one line would be to also have the column number reported, along with a ^ to indicate the exact point of error. So I think instead misreporting the location of an instruction pointer could be deceptive and not helpful. I think it unreasonable to ask the traceback reporter to guess which one line would be most helpful to report. The OP suggests somehow tracing back to the *first* line of a multiline statement. But on python-ideas, G. Rodola noted the following assert \
1 == 0, \
"error"
Traceback (most recent call last):
File "foo.py", line 3, in <module>
"error"
AssertionError: error and requested that the *second* or *middle* line should be reported. (But what if the asserted expression itself covered multiple lines?) I say sometimes 'not helpful' since the traceback already says what function the pointer is in, if it is in one, and therefore what function what called by the previous frame. In the 'reproducer', 'raise Exception' is in fun1, so in fun2, 'par="value")' must have been preceded by 'fun1(...' on a previous line. So replacing 'par="value")' with 'fun1(...' would not add information. I think debugging multiline statements will usually requires looking at the whole statement and a few more. To make this easy, one can use a more sophisticated environment than dumb consoles. For instance, if one executes a top-level script from an Idle edit window, one can right click on a traceback in the Shell window and select 'Go to file/line'. Idle will open the file, if necessary, and highlight the line. |
I think all the lines involved in a multi-line expression should be reported. Take for example this simple code: With Python 2 the traceback is:
Traceback (most recent call last):
File "deleteme.py", line 3, in <module>
if c.isdigit() or c.isupper()]
ValueError: invalid literal for int() with base 16: 'G' Which is not very helpful, because you don't see the int() call that is causing the error in the first place and you don't see where the G comes from. Incidentally, for this specific case, the traceback is a bit better on Python3:
Traceback (most recent call last):
File "deleteme.py", line 2, in <module>
for c in '0123456789ABGDEF'
File "deleteme.py", line 3, in <listcomp>
if c.isdigit() or c.isupper()]
ValueError: invalid literal for int() with base 16: 'G' But this still misses the int() call. IMHO it would be better to have something like:
Traceback (most recent call last):
File "deleteme.py", lines 1-3, in <module>
x = [int(c, 16)
for c in '0123456789ABCDEFG'
if c.isdigit() or c.isupper()]
ValueError: invalid literal for int() with base 16: 'G' If this is rejected for some reason, I think printing the first line -- rather than the last -- would still be an improvement, because in most of the cases the first line contains more information (e.g. the name of the function with a multi-line list of args, a raise/assert with the name of the exception and part of the message, the operation done for each element of a genexp/listcomp). |
No, I was suggesting (assuming it's reasonably possible in terms of implementation) to report the whole block (3 lines) as-is. self.assertEqual(1,
2) Currently prints: Traceback (most recent call last):
File "foo.py", line 7, in test_foo
2)
AssertionError: 1 != 2 While instead it should: Traceback (most recent call last):
File "foo.py", line 6, in test_foo
self.assertEqual(1,
2)
AssertionError: 1 != 2 |
I agree that printing all the lines would be best, and printing the first line would be second best. I suspect once someone works out how to do the latter it is just as easy to do the former, though :) |
OK, you would make tracebacks even longer by adding lines that might or might not be be useful. Suppose the error is not the assertion itself, but in one of the expressions (in this case, being compared for equality): My experience on python-list is that people more often have a problem with a whole line being too much context, not too little. Someone writes 'print(complicated_expression)' and gets a Unicode error. Is the problem the expression or the printing of the result to a limited charset console? Both cases have been posted. I sometimes advise people to break their elegant one-liners into two or more lines to make the actual error expression more obvious. If we were to revise the CPython compiler and traceback generator, that is what I think the aim should be. Such a change would include printing multiple lines when the expression spans multiple lines. This might have to be limited to debug builds. |
The point is to print one logical line. Printing a fragment of the logical line is clearly not very helpful...I have found myself a number of times looking for the bug in the fragment printed, and (quickly, but still...) then remembering that the traceback just prints the last physical line and the problem might be earlier. Providing a caret is probably impossible. I don't think the bytecode contains enough information to even make a guess as to where a caret should go. I'm not even sure we can get the logical line. |
Given the unbounded and recursive nature of 'logical line', I disagree with that as a general, versus special-case, statement. Consider: Or consider: Quite aside from this, 'd()/e(),' and 'g=h()/i(),' are both legal logical lines in themselves (though with different semantics) as well as being fragments of the r call. Moreover, the multiline r call above may itself be just a fragment of a call to another function (further indented). The practical difficulty for the proposal, even if limited to multiline calls, is that any bare call could either be part of a larger expression by returning something or be a statement operating through side-effects. Similarly, any 'var=f()' call could either be an assignment statement or keyword arg expression in some other call. |
I tried to run this, with e() == 0, and got:
Traceback (most recent call last):
File "deleteme.py", line 6, in <module>
d()/e(),
ZeroDivisionError: division by zero This is actually a good point, because in this specific case the traceback is correct, and showing all the lines would make things worse.
I assume it wouldn't be too difficult to limit the output to 3-5 lines max. Someone should try to fiddle with the code I indicated in msg144813, and see what can be done. If there are enough informations it might be possible to keep reporting a single line for the aforementioned case, but report more lines for the cases listed in the other messages; if there aren't enough informations we will have to abandon the idea (or find a way to retrieve them). |
idlelib has two files that *might* be helpful to anyone who wants to pursue this: PyParse.py and HyperParser.py. One use of the latter is for calltips. When the user types '(', the HyperParser is used first to determine whether the '(' is preceded by an operator (making it a grouper) or not (making it a call operator). In the later case, it parses backwards to find the beginning of the callable expression, which is then evaluated to get the callable (if it does not contain an embedded call) and hence the signature. Parsing forward and back from a given line might be similar. |
I have been able to get the logical line details (start of it), now working on to print the required lines. |
Can you preserve just the *additional* indentation? That is, strip the excess leading whitespace, but preserve the logical indentation, like textwrap.dedent does? |
Here is yet one example. x = 1; y = 0; z = 0
(
1/x +
1/y +
1/z
) The traceback looks correct: Traceback (most recent call last):
File "1.py", line 4, in <module>
1/y +
ZeroDivisionError: division by zero But for following code: x = 0; y = 1; z = 0
(
1/x +
1/y +
1/z
) the traceback is the same and it is totally wrong. This is because generated bytecode is: 1 0 LOAD_CONST 0 (0) 4 18 LOAD_CONST 1 (1) 5 33 LOAD_CONST 1 (1) 1/x and 1/y have the same line number. |
Seems PR 8774 fixes all mentioned examples (as well as examples in bpo-34372), except that tracebacks show the single line number of the first line of the multiline expression instead of the range of lines as was suggested. Showing the range of lines requires much more significant changes and changing the marshal format. The drawback is minor increasing the size of code objects in memory (due to saving additional line numbers in lnotab), pyc files, and disassembly output for multiline expressions. |
The other drawback is changing how the trace function is notified in a few scenarios. |
Supposing I like the old behavior (line number of the end of the statement), is there any way to recover that line number from the traceback after this change? |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: