pdb skips frames after hitting a breakpoint and running step

[xdegaye]

BPO	13183
Nosy	@loewis, @birkenfeld, @orsenthil, @pitrou, @meadori, @xdegaye
Files	restore_trace.patch restore_trace_2.patch: py3 patch and improved test case restore_trace.py27.patch: py2 patch and test case

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/orsenthil'
closed_at = <Date 2012-05-06.09:54:32.748>
created_at = <Date 2011-10-14.18:56:31.249>
labels = ['type-bug', 'library']
title = 'pdb skips frames after hitting a breakpoint and running step'
updated_at = <Date 2012-05-09.15:44:04.557>
user = 'https://github.com/xdegaye'

bugs.python.org fields:

activity = <Date 2012-05-09.15:44:04.557>
actor = 'orsenthil'
assignee = 'orsenthil'
closed = True
closed_date = <Date 2012-05-06.09:54:32.748>
closer = 'georg.brandl'
components = ['Library (Lib)']
creation = <Date 2011-10-14.18:56:31.249>
creator = 'xdegaye'
dependencies = []
files = ['23410', '23418', '23419']
hgrepos = []
issue_num = 13183
keywords = ['needs review']
message_count = 24.0
messages = ['145557', '145633', '145634', '154889', '154933', '159706', '159734', '159741', '159742', '159761', '159763', '159765', '159769', '159771', '159774', '159775', '159813', '159818', '160065', '160067', '160068', '160069', '160168', '160301']
nosy_count = 9.0
nosy_names = ['loewis', 'georg.brandl', 'orsenthil', 'pitrou', 'meador.inge', 'neologix', 'xdegaye', 'tshepang', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue13183'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']

[xdegaye]

Pdb skips frames after hitting a breakpoint and running step commands
that walk back the frame stack.

Run the following two tests with the two files named foo.py and bar.py:

=== foo.py ============================================

from bar import bar

def foo():
    bar()

def nope():
    pass

def foobar():
    foo()
    nope()

foobar()

=== bar.py ============================================

def bar():
    print('1')

=== test_1 ============================================

$ python3 --version
Python 3.2

$ python3 -m pdb foo.py
> /path/to/foo.py(1)<module>()
-> from bar import bar
(Pdb) from bar import bar
(Pdb) break bar
Breakpoint 1 at /path/to/bar.py:1
(Pdb) continue
> /path/to/bar.py(2)bar()
-> print('1')
(Pdb) step
1
--Return--
> /path/to/bar.py(2)bar()->None
-> print('1')
(Pdb) step
--Call--
> /path/to/foo.py(6)nope()
-> def nope():
(Pdb)

=== test_2 ============================================

$ python3 -m pdb foo.py
> /path/to/foo.py(1)<module>()
-> from bar import bar
(Pdb) break nope
Breakpoint 1 at /path/to/foo.py:6
(Pdb) from bar import bar
(Pdb) break bar
Breakpoint 2 at /path/to/bar.py:1
(Pdb) continue
> /path/to/bar.py(2)bar()
-> print('1')
(Pdb) step
1
--Return--
> /path/to/bar.py(2)bar()->None
-> print('1')
(Pdb) step
--Return--
> /path/to/foo.py(4)foo()->None
-> bar()
(Pdb)

===========================================================

Note: stop_here, break_anywhere and dispatch_call are methods of the
Bdb class.

test_1 fails to stop in foo() after the second 'step' command because
the trace function is not set for all the frames being created in the
foo module, since stop_here() and break_anywhere() are both False
whenever dispatch_call() is invoked in this module. So after the
second 'step' command, trace_dispatch is not invoked by the
interpreter until a new frame is created, which happens when nope() is
called.

test_2 succeeds and stops in foo() after the second 'step' command.
After setting the dummy breakpoint 1 in the foo module in test_2,
break_anywhere() becomes True in the foo module and the trace function
is set for all the frames created in this module (with an associated
performance penalty).

The problem exists in all python versions.

The attached patch fixes this problem by restoring the trace function
on returning from a frame when the command is 'step'.

The patch includes a test case.

[xdegaye]

Uploaded restore_trace_2.patch that improves the test case.

[xdegaye]

Uploaded restore_trace.py27.patch with a fix and test case for python 2.7.

[xdegaye]

This message is just a reminder that this 4 months old issue raises
the point that the step command in pdb is broken. A patch and test
case have been proposed.
No comment so far.

As the author of pyclewn, a Vim front end to pdb and gdb, I would be
grateful for any progress on this problem.

[orsenthil]

It's an interesting issue. Thanks for the report and patch, Xavier. I am setting patch to needing review.

[orsenthil]

Hello Xavier,

This issue required some tracing through the calls and I see the problem that you have mentioned and patch fixes the problem.

One comment on the patch, for the tests in the module, this line -

self.frame_returning = None 

does not seem to have a coverage. Is there a specific significance in setting this to None and would a test help?

     def dispatch_return(self, frame, arg):
         if self.stop_here(frame) or frame == self.returnframe:
+            self.frame_returning = frame
             self.user_return(frame, arg)
+            self.frame_returning = None
             if self.quitting: raise BdbQuit
         return self.trace_dispatch

Sorry for the delay, I shall commit the fix in 3.3.

[xdegaye]

Hi Senthil,

Thanks for your help with this issue.

self.frame_returning is both a flag to indicate that we are returning
from the current frame and a value (the current frame). We need both
as set_step() (the method invoked when the user runs the step command)
does not know the current frame and wether we are returning from the
current frame.

Here is a raw sketch of the call chain in the case where the user
types the step command on returning from the current frame (Pdb
subclasses both bdb.Bdb and cmd.Cmd):

Bdb::dispatch_return
Pdb::user_return (Bdb overriden method)
Pdb::interaction
Cmd::cmdloop
Cmd::onecmd
Pdb::do_step
Bdb::set_step

So self.frame_returning must be set to None after the call to
self.user_return() so that its value is not used in another later step
command, where we are not returning from this frame. Actually it is
more explicit and more robust to use a try-finally clause, such as:

    def dispatch_return(self, frame, arg):
        if self.stop_here(frame) or frame == self.returnframe:
            try:
                self.frame_returning = frame
                self.user_return(frame, arg)
            finally:
                self.frame_returning = None
            if self.quitting: raise BdbQuit
        return self.trace_dispatch

Xavier

[python-dev]

New changeset 96cb47f8142e by Senthil Kumaran in branch '3.2':
bpo-13183 - Fix pdb skipping frames after hitting a breakpoint and running step. Patch by Xavier de Gaye
http://hg.python.org/cpython/rev/96cb47f8142e

New changeset 5ea23739e9ba by Senthil Kumaran in branch '2.7':
bpo-13183 - Fix pdb skipping frames after hitting a breakpoint and running step. Patch by Xavier de Gaye
http://hg.python.org/cpython/rev/5ea23739e9ba

New changeset ab63e874265e by Senthil Kumaran in branch 'default':
bpo-13183 - Fix pdb skipping frames after hitting a breakpoint and running step. Patch by Xavier de Gaye
http://hg.python.org/cpython/rev/ab63e874265e

[orsenthil]

Hello Xavier,

Thanks for the explanation. Understood the reason for setting the frame_returning to None. The patch is committed in all python versions and the issue is fixed now. Thanks for the patch and prodding through this.

--
Senthil

[loewis]

The test fails on Windows. Whereas on Unix, the two step commands produce this output:

-> print('1')
(Pdb) step
1
--Return--

/net/pao/export/home/staff/loewis/work/33/bar.py(2)bar()->None
-> print('1')
(Pdb) step
--Return--
/net/pao/export/home/staff/loewis/work/33/main.py(5)foo()->None
-> bar()
(Pdb) quit

on Windows, they produce this output:

-> print('1')
(Pdb) step
--Call--

c:\users\martin\33\python\lib\encodings\cp850.py(18)encode()
-> def encode(self, input, final=False):
(Pdb) step
c:\users\martin\33\python\lib\encodings\cp850.py(19)encode()
-> return codecs.charmap_encode(input,self.errors,encoding_map)[0]
(Pdb) quit

I.e. the stepping enters the print, and breaks in the codec.

Reopening the issue.