Make IPython.core.debugger interruptible by KeyboardInterrupt

[itamarst]

Together with a number of other PRs, this is a step towards addressing #10516

There are three issues:

1. IPython.core.debugger would swallow KeyboardInterrupts and not let them get raised. This fixes that issue in a new class, InterruptiblePdb, so as not to change existing behavior.
2. pdb had the same issue, and by default all debuggers in Jupyter kernel usage need to be interruptible by KeyboardInterrupt to fix this bug. Make pdb on Windows interruptible ipykernel#490 overrides pdb and the default debugger to use the new InterruptiblePdb.
3. Finally, ipykernel's mechanism for interrupting subprocesses doesn't raise KeyboardInterrupt on Windows when waiting for input(). Fixed by Allow interrupting input() on Windows, as part of effort to make pdb interruptible ipykernel#498

[Carreau]

Thanks, marking as 7.14.

There seem to be some unexpected interaction as IPython tests are failing, but they should not depend on PyZMQ. I'll have to dig into this.

[itamarst]

I can take a look too, just, step 1 is convincing pyzmq maintainer to accept that PR (or convincing myself that PR is not necessary).

[Carreau]

if that's @minrk then he can also merge the fixes here if necessary and likely do synchronized releases.

[itamarst]

Ah, great, just need to convince him it's necessary :)

[itamarst]

Looks like I was wrong and pyzmq is fine, so just down to these two PRs.

[itamarst]

After further research, I have determined that the interrupt mechanism used by ipykernel on Windows doesn't interrupt pdb (or rather, I suspect, input()). Some research brought up this epic comment (https://stackoverflow.com/a/35792192) which explains how to make a signal handler for Windows that actually works, and I reproduced it in the test.

Unfortunately this will require some corresponding changes to ipykernel, and still has some... oddities. Like, I need to send the signal multiple times (perhaps there's a race condition? A flag that needs to be set?). Python 3.6 isn't working in AppVeyor so that suggests more issues. If I can just get it working on e.g. POSIX plus Python 3.8 Windows that will still be an improvement, so I will try to do that next.

[itamarst]

I switched to a simpler test that doesn't involve subprocesses. I will next try to write a process-interrupting test in an ipykernel branch, which is a better place to do it since it is the one that actually implements that logic, and see if I can fix the Windows issue I encountered there.

[itamarst]

@Carreau OK I think this is ... plausibly maybe even correct and working now 😁

[Carreau]

OK, let's try.

[tonyfast]

Anyone have an idea as to while my pdb.Pdb class is being updated to IPython.core.debugger.InterruptiblePdb? I can't figure out where in the debugger phases this class is being forced to change. It is causing issues with running doctest in a shell session.

[itamarst]

@tonyfast It happens in ipykernel.

[tonyfast]

Is there an option or flag to not do this?

I don't want to use pdb and it is disrupting a testing pattern in the stdlib module which is harshing my buzz.

[tonyfast]

Could this be something that the InteractiveShell handles with traitlets when shell.pdb is True?