pdb go stack up/down

[MarkusPrller]

BPO	9633
Nosy	@birkenfeld, @ncoghlan, @blueyed, @merwok, @asvetlov, @meadori, @xdegaye, @iritkatriel
PRs	bpo-22577: in pdb, prevent changes to locals from being lost after a jump #11041
Files	issue9633.patch: py3k patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2010-08-18.12:17:16.520>
labels = ['3.8', 'type-bug', 'library', '3.9', '3.10']
title = 'pdb go stack up/down'
updated_at = <Date 2020-10-22.22:46:16.422>
user = 'https://bugs.python.org/MarkusPrller'

bugs.python.org fields:

activity = <Date 2020-10-22.22:46:16.422>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2010-08-18.12:17:16.520>
creator = 'Markus.Pr\xc3\xb6ller'
dependencies = []
files = ['18706']
hgrepos = []
issue_num = 9633
keywords = ['patch']
message_count = 7.0
messages = ['114213', '114218', '114231', '115358', '176270', '304391', '379375']
nosy_count = 9.0
nosy_names = ['georg.brandl', 'ncoghlan', 'blueyed', 'eric.araujo', 'asvetlov', 'meador.inge', 'xdegaye', 'Markus.Pr\xc3\xb6ller', 'iritkatriel']
pr_nums = ['11041']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue9633'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[MarkusPrller]

Hello,

with python 2.7 I encounter the following problem:
I have created the following sample script:

import pdb

def function_1(number):
    stack_1 = number
    function_2(stack_1)
    
def function_2(number):
    stack_2 = number + 1
    function_3(stack_2)
    
def function_3(number):
    stack_3 = number + 1
    pdb.set_trace()
    print stack_3
    
function_1(1)

This is what I have done in the pdb session:

c:\tst_pdb.py(14)function_3()
-> print stack_3
(Pdb) l
9 function_3(stack_2)
10
11 def function_3(number):
12 stack_3 = number + 1
13 pdb.set_trace()
14 -> print stack_3
15
16 function_1(1) [EOF]
(Pdb) stack_3
3
(Pdb) !stack_3 = 177
(Pdb) !print stack_3
177
(Pdb) u
c:\tst_pdb.py(9)function_2()
-> function_3(stack_2)
(Pdb) l
4 stack_1 = number
5 function_2(stack_1)
6
7 def function_2(number):
8 stack_2 = number + 1
9 -> function_3(stack_2)
10
11 def function_3(number):
12 stack_3 = number + 1
13 pdb.set_trace()
14 print stack_3
(Pdb) !print stack_2
2
(Pdb) !stack_2 = 144
(Pdb) !print stack_2
144
(Pdb) d
c:\tst_pdb.py(14)function_3()
-> print stack_3
(Pdb) l
9 function_3(stack_2)
10
11 def function_3(number):
12 stack_3 = number + 1
13 pdb.set_trace()
14 -> print stack_3
15
16 function_1(1) [EOF]
(Pdb) stack_3
3
(Pdb) u
c:\tst_pdb.py(9)function_2()
-> function_3(stack_2)
(Pdb) !print stack_2
2
(Pdb)

I walked through the stack and changed the values of the variables stack_x but the values weren't saved when I moved one frame up/down

[merwok]

Thank you for the report. I don’t know pdb much, but I assume you have checked the doc to make sure this is indeed a bug. Can you test it with 3.1 and 3.2 too?

Bug tracker tip: You can find if a core developer is interested in a module in Misc/maintainers.rst and make them nosy (or assign to them, if there is a star near their name, like in Georg’s case).

[birkenfeld]

The problem here is that changes in the locals are only saved back to the frame when leaving the trace function, and up/down don't do that.

This could be fixed by making Pdb.curframe_locals a dictionary for all visited frames while interaction is running. I'll look into it for 3.2.

[meadori]

I believe this is slightly tricky because 'bdb.format_stack_entry' makes references to '.f_locals' and 'bdb.format_stack_entry' is invoked in several places in 'pdb'. One option would be to add a extra parameter to 'bdb.format_stack_entry' to accept a dictionary of locals to operate with.

I implemented this approach and added a doctest to verify. See attached patch. I didn't update the 'bdb' documentation to note the new parameter, but will if this approach seems reasonable.

[xdegaye]

It is not only the up and down commands; the where, longlist and
source commands may also overwrite changes made to f_locals.

In Markus sample script above, and with the patch applied, when the
change made to stack_2 is followed by a step command, stack_2 value
goes back to '2' again. This is because the self.curframe_locals list
is built from scratch again when processing the new trace function
triggered after the step command and a new call to frame_getlocals
(and thus to PyFrame_FastToLocals) is made on the frame of function_2
to populate self.curframe_locals, overwritting the previous value
'144' of stack_2.

With a dictionary mapping frames to f_locals (and only updating this
dictionary when adding entries for frames that do not exist in the
dictionary), the above problem is fixed. However, running step
repeatedly until returning to function_2, causes the value of stack_2
to become '2' again when in function_2. This is because, just after
returning from the frame of function_3, the following calls are made
in call_trampoline to invoke the first (after tracing function_3)
trace function in function_2 frame; the 'frame' argument of
PyFrame_FastToLocals is the frame of function_2 and so, the call to
PyFrame_FastToLocals overwrites the value '144' of stack_2, and the
value of stack_2 is back to '2' again:

  PyFrame_FastToLocals(frame);
  result = PyEval_CallObject(callback, args);
  PyFrame_LocalsToFast(frame, 1);

Only the f_locals of the top level frame is saved by
PyFrame_LocalsToFast after exiting the trace function, so it is not
possible to keep consistent the changes made in the f_locals of the
lower level frames.

Another solution would be to ensure that changes made to locals at the
top level frame are not overwritten, and to explicitly make readonly
the locals of the other frames. See how this is done at
http://code.google.com/p/pdb-clone/source/detail?r=5643890608fce2eac318de61f1d6d065d5a7d538

[ncoghlan]

Note that we're currently looking into this as something that could be potentially addressed by PEP-558 and any related changes to the way that function locals interact with trace hooks: https://bugs.python.org/issue30744#msg304388

[iritkatriel]

I've reproduced the bug on 3.10.