IDLE - shell becomes unresponsive if debugger windows is closed while active.

[serwy]

BPO	15348
Nosy	@terryjreedy, @serwy, @asvetlov, @roseman, @jimbo1qaz
Files	issue15348.patch stable_idle_debugger.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2019-09-19.17:33:03.622>
created_at = <Date 2012-07-13.19:38:09.694>
labels = ['expert-IDLE', 'type-bug']
title = 'IDLE - shell becomes unresponsive if debugger windows is closed while active.'
updated_at = <Date 2019-09-19.17:33:03.613>
user = 'https://github.com/serwy'

bugs.python.org fields:

activity = <Date 2019-09-19.17:33:03.613>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2019-09-19.17:33:03.622>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2012-07-13.19:38:09.694>
creator = 'roger.serwy'
dependencies = []
files = ['27482', '27502']
hgrepos = []
issue_num = 15348
keywords = ['patch']
message_count = 18.0
messages = ['165411', '172363', '172439', '172440', '172441', '172442', '172443', '172444', '172445', '172446', '172447', '245520', '255002', '255035', '255040', '255041', '255042', '352809']
nosy_count = 7.0
nosy_names = ['terry.reedy', 'roger.serwy', 'asvetlov', 'markroseman', 'THRlWiTi', 'python-dev', 'jimbo1qaz']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue15348'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5', 'Python 3.6']

[serwy]

The IDLE shell does not respond to commands if the debugger window is closed by clicking "X". This is due to PyShell's "executing" flag not being reset.

Steps to reproduce:

1. Start IDLE with a shell.
2. Enable debugger.
3. Press enter in the shell. (activates the debugger)
4. Close the debugger by clicking "X".

The shell is no longer responsive to new commands. Selecting "Restart Shell" does not fix the problem either.

[serwy]

Attached is a patch against 3.4 to solve the issue.

The debugger was originally written for running IDLE without a subprocess. As a result, calls to idb.run() from Debugger.py would block until completed. If .interacting == 1 then clicking "X" would not close the debugger window. Once the debugger finishes, the .interacting flag would reset to 0, which then allows for the Debugger.close() method to destroy the debugger window. Clicking "Quit" is one way to have the debugger finish.

When running *with* a subprocess, the idb.run() returns immediately, since it is using the IdbProxy object instead. As a consequence, the .interacting flag is also reset to 0 immediately despite the debugger still being active. Thus, clicking "X" would allow the debugger to close without properly stopping the debugger via clicking "Quit".

A useful side-effect of the interactive debugger being active is that the PyShell.executing flag is set to true. When the debugger stops, the call to PyShell.endexecuting() from poll_subprocess() resets the .executing flag. The patch checks if PyShell's executing is true in order to prevent the Debugger Window from being closed by clicking "X". This makes the behavior is consistent with running the IDLE debugger without a subprocess.

When running IDLE without a subprocess is fully removed (see bpo-16123), then maintaining the Debugger code will become simpler. Until then, testing against both code paths is necessary.

[serwy]

While trying to address bpo-15347, I discovered one too many corner cases where the debugger breaks IDLE. The stable_idle_debugger.diff against 3.4 contains necessary changes to make IDLE more reliable while debugging. Since the IDLE debugger is not documented these changes should not be controversial.

Here are all the cases the patch fixes:

1. Closing the debugger while debugging causes the shell to not respond to new commands when using a subprocess.
2. Pressing F5 in an editor several times with an active debugger causes IDLE to become unresponsive.
   2.1) When not using the subprocess, the patch fixes an AttributeError in runcode in PyShell.py, since .interp is not an attribute of the interpreter.
3. Closing the shell window while the debugger is debugging prevents IDLE from terminating if no editors are open. bpo-15347
4. Closing the shell with an idle debugger causes a traceback in the shell (with no subprocess).

The patch creates the following behavior:

1. The debugger cannot be closed by clicking "X" while it is debugging. Instead, the "Quit" button gets flashed as a reminder to stop the debugger before closing.
2. Pressing F5 in the editor window while the debugger is debugging brings focus to the debugger window.
3. Closing the shell will close the debugger properly. The .close() method of Debugger.py quits the nested eventloop in .interaction() if needed.

I tested these issues and solutions on Linux. Can someone review the patch for Windows and Mac?

[jimbo1qaz]

Is it possible to make the X button quit the debugger if enabled?

[serwy]

Clicking "X" while the debugger is enabled, but not actively debugging a program, will close the debugger window.

[jimbo1qaz]

What's the intended behavior? stop debugging and quit the program? stop debugging and continue freerunning? I'm using 3.3, can you make the patch stop debugging the active running program when you click the X?

[serwy]

If the debugger is active, then clicking "X" will flash the "Quit" button. You must click the quit button first before closing the debugger window.

[jimbo1qaz]

Then frigging change it!

[serwy]

Closing the active debugger with "X" creates a problem with references and callbacks. I encountered too many corner cases where I could not implement that behavior simply.

[serwy]

But of course, you are more than welcomed to try to submit a patch yourself. Just make sure that those corner cases I described earlier are handled reasonable when running IDLE with and without a subprocess.

[terryjreedy]

I have never used the debugger, but I verified the bug. While something was 'hung', the debugger would not restart because something was 'busy'. I also verified that clicking [quit] first and then [x] works. I will try to test the second patch sometime.

Jimbo, the first priority is to fix crashes and hangs. We control the response to buttons we add, but the [x] close button involves the gui and operating system. So working with it can be trickier than it seems it ought to be.

[terryjreedy]

bpo-24455 is probably related.

[terryjreedy]

"not responsive to new commands" means that a statements can be typed but <Enter> is ignored and no execution takes place.

[terryjreedy]

The patch for bpo-24455 replaced root.mainloop,.quit with tcl vwait, set as the suspend/resume mechanism for Debugger.interation. After this, closing the shell as described in bpo-15347, or the debugger with [x] as described here, on my Windows 10 machine, resulted in

File "F:\Python\dev\34\lib\idlelib\Debugger.py", line 20, in user_line
self.gui.interaction(message, frame)
_tkinter.TclError: invalid command name ".86740760.86830984.86827736"

instead of a hang. This is an improvement as the Exception can be caught and ignored. People testing of other systems did not report this. With TclError caught, closing the debugger with [x] then produced

File "F:\Python\dev\34\lib\idlelib\Debugger.py", line 21, in user_line
self.gui.interaction(message, frame)
RuntimeError: Unknown object id: 'gui_adapter'

I applied the patch with Runtime also caught and ignored.

The exceptions did not occur when [Quit] was pressed first. So I think [x] should do whatever Quit does before closing.

[terryjreedy]

(I was wrong about seeing the exception when closing the shell.)

A major difference between [Quit] and [x] is that [Quit] stops the running program whereas [x] closes the visible gui but leaves the program to run through to the end, as though the [go] button were pressed. I added 'as e' and 'print(e)' to the try except and there are multiple RuntimeErrors, 1 per line traced by [step], including IDLE code for input() and print() calls, with [x], but none with [quit] first.

The TclError only occurs in 3.4.3, not in 2.7.10.

The string 'gui_adapter' comes from RemoteDebugger.py, line 28.
gui_adap_oid = "gui_adapter"
It represents the debugger window in the idle process. My interpretation of the traceback is that when
self.gui.interaction(message, frame)
is executed in the user process, the interaction call actually calls debugger.interaction in the idle process via RemoteDebugger.GUIProxy.interaction. The latter does not appear in the traceback because IDLE rpc calls are deleted from tracebacks before they are printed.

Replacing self.abort_loop with self.quit (which calls abort_loop) eliminates the Runtime exceptions, as the backend debugger in the user process is stopped before the idle process frontend gui disappears.

However, self.idb.set_quit (within self.quit) raises
AttributeError: 'Idb' object has no attribute 'botframe'
if one attempts to immediately close the debugger window (with menu or button) before it is activated. It is then left as a zombie window. The exception is in the Tkinter callback, so it is printed to console rather than the IDLE shell, and included RemoteDebugger and rpc calls, but not the line in Bdb.set_quit(self),
self.stopframe = self.botframe
that actually raised the exception. So the call must either be conditioned or the exception caught. I decided to go with latter now.

[python-dev]

New changeset 427ad0ede016 by Terry Jan Reedy in branch '2.7':
bpo-15348: Stop debugger engine (normally in user process)
https://hg.python.org/cpython/rev/427ad0ede016

New changeset 4eae64a9cd26 by Terry Jan Reedy in branch '3.4':
bpo-15348: Stop debugger engine (normally in user process)
https://hg.python.org/cpython/rev/4eae64a9cd26

[terryjreedy]

I don't like the idea of [x] flashing quit instead of closing. This patch, on top of the one for bpo-24455, may not be perfect, but the two are definite improvements. Unless I see something critically bad first, I want them in the upcoming releases. Mark claims that his second patch for bpo-15347 improves behavior for no subprocess. But that was without this patch.

There is one situation not mentioned that I know is still glitchy.  Debug a program with input(prompt).  Run up to and including that line.  While the program is waiting for input, close with [x].  The shell prints '[DEBUG OFF]' after the prompt and '>>> ' on the next line.  But that is a fake prompt in that a statement entered will not be executed, but will be seen as the response to the pending input().

While there is a pending input (or any other blocked statement), the [Quit] button is disabled. The close method should detect this 'program executing' condition and ask whether to kill it, just as if one tries to close IDLE under the same condition.

I am leaving this open, at least for now, at least for this.

[terryjreedy]

The input glitch mentioned above no longer exists, so closing.