TypeError when f_trace is None and tracing.

[xdegaye]

BPO	20041
Nosy	@birkenfeld, @abalkin, @xdegaye, @serhiy-storchaka, @nightlark
Files	tracer.py f_trace.patch tests.patch none_f_trace.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2016-06-04.17:34:11.251>
created_at = <Date 2013-12-21.14:01:27.004>
labels = ['interpreter-core', 'type-bug']
title = 'TypeError when f_trace is None and tracing.'
updated_at = <Date 2021-08-18.06:37:50.621>
user = 'https://github.com/xdegaye'

bugs.python.org fields:

activity = <Date 2021-08-18.06:37:50.621>
actor = 'rmast'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2016-06-04.17:34:11.251>
closer = 'serhiy.storchaka'
components = ['Interpreter Core']
creation = <Date 2013-12-21.14:01:27.004>
creator = 'xdegaye'
dependencies = []
files = ['33245', '33246', '33260', '43198']
hgrepos = []
issue_num = 20041
keywords = ['patch']
message_count = 8.0
messages = ['206735', '206896', '267094', '267157', '267158', '267265', '267281', '267288']
nosy_count = 7.0
nosy_names = ['georg.brandl', 'belopolsky', 'xdegaye', 'python-dev', 'serhiy.storchaka', 'Winterflower', 'rmast']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue20041'
versions = ['Python 3.5', 'Python 3.6']

[xdegaye]

Section 3.2 of 'The Python Language Reference' states:
f_trace, if not None, is a function called at the start of each source code line

Run the attached tracer.py and see that although f_trace is None in both cases when 'cmd' is either
'delete' or 'set', the second case raises a TypeError exception:
$ python tracer.py
f_trace: None
delete start
delete done
f_trace: None
set start
Traceback (most recent call last):
  File "tracer.py", line 19, in <module>
    foo('set')
  File "tracer.py", line 15, in foo
    print(cmd, 'done')
  File "tracer.py", line 15, in foo
    print(cmd, 'done')
TypeError: 'NoneType' object is not callable

Also, the frame.f_lineno may be wrong in a traceback when f_trace is set to None because
PyFrame_GetLineNumber() does not handle this case.

The attached patch fixes this issue.
The patch also fixes bpo-11992 and bpo-20040.
The patch also fixes the dispatch_call() method of Bdb in the bdb module when the frame is a
generator and the previous command is next, until or return.
The patch also provides a backward compatible solution to the performance enhancement described in
bpo-16672.

[xdegaye]

Adding the corresponding tests.

[serhiy-storchaka]

I think the code would be simpler if convert the argument of frame_settrace() to NULL if it is Py_None.

[xdegaye]

On a 'call' trace event, the bdb dispatch_call() function returns None when there is nothing to trace in this function (no breakpoints). With the changes made by the patch in ceval.c, the costly maybe_call_line_trace() function is not called on each line in this case since f->f_trace is Py_None. This provides the performance enhancements described in bpo-16672 without breaking the _hotshot extension module or other extension modules using PyEval_SetTrace().

I agree that the code would be much simpler when f->f_trace is set to NULL by the frame_settrace() setter or trace_trampoline() when it is Py_None. The performance gain described above by using Py_None may not be worth the complexity.

Thanks for looking into this Serhiy.

[serhiy-storchaka]

This may be worthwhile optimization, but this is different issue. Let first fix a TypeError, and then open new issue for the optimization.

[xdegaye]

This patch fixes the TypeError.
New bpo-27218: improve tracing performance with f_trace set to Py_None.

[serhiy-storchaka]

LGTM.

[python-dev]

New changeset 4d916be61d46 by Serhiy Storchaka in branch '2.7':
Issue bpo-20041: Fixed TypeError when frame.f_trace is set to None.
https://hg.python.org/cpython/rev/4d916be61d46

New changeset 74ad78d2dd8d by Serhiy Storchaka in branch '3.5':
Issue bpo-20041: Fixed TypeError when frame.f_trace is set to None.
https://hg.python.org/cpython/rev/74ad78d2dd8d

New changeset f993dbeb2ad2 by Serhiy Storchaka in branch 'default':
Issue bpo-20041: Fixed TypeError when frame.f_trace is set to None.
https://hg.python.org/cpython/rev/f993dbeb2ad2