misc notebook: connection file cleanup, first heartbeat, startup flush #1187
Conversation
Kernels would not linger, but the KernelManagers are not garbage-collected on shutdown. This means that connection files for kernels still running at notebook shutdown would not be removed. Also disable the unnecessary (and actively unhelpful) SIGINT handler inherited from the original copy/paste from the qt app.
prevents startup script/file output from being attached to the first cell of a frontend
|
includes stdout/err flush fix mentioned in #1191 |
Heartbeats start immediately, causing false heart failures on slow systems that can take a while to start kernel subprocesses. Also adds a 'flush' to the heartbeat callback (just like in IPython.parallel), to protect against server load being detected as heart failures.
|
include heartbeat delay mentioned in #1198. I still don't like the way heartbeats are done in either the notebook or the base KernelManager, and the base KernelManager also really must be configurable. But that's for another time. |
|
Mmh, I'm gettting these messages in the starting consoles: |
|
hm, any earlier info? That's always a side effect of a previous error. I'll look into it. PyZMQ version? |
|
No, prior to that it's all normal output. It seems to only happen on FF, but it doesn't happen always. I can't seem to get what triggers it. I'm using pyzmq 2.1.9, the system one from ubuntu 11.10 (64bit). |
|
Ah, I know what this is. Try opening a new notebook, then closing the tab within 5 seconds. You should see it again. I'll push a fix shortly. |
| """ | ||
| self.log.info('Shutting down kernels') | ||
| km = self.kernel_manager | ||
| while km.kernel_ids: |
There was a problem hiding this comment.
I'm wondering if there should be a safety exit here. While loops whose exit condition might not be triggered are dangerous. If for some reason the kernel_ids list doesn't fully empty out, this guy will hang forever.
Perhaps this logic would be cleaner as
for i in range(3): # try no more than 3 times
for k in km.kernel_ids:
km.kill_kernel(k)
if not km.kernel_ids:
break
else:
self.log.warn('Unkillable kernels...')What do you think?
There was a problem hiding this comment.
km.kill_kernel is the explicit method for deleting kernel ids, and cannot fail to do so without actually raising and exiting the loop. Your proposal as it is will not work because kill_kernel changes km.kernel_ids, so a copy would have to be made. But if I do make the copy, I don't think there's any need to try multiple times:
for k in list(km.kernel_ids):
km.kill_kernel(k)should do just fine.
There was a problem hiding this comment.
Ah, thanks for the clarification. Then yes, I think that's a cleaner-looking code; if nothing else it's obvious that it can't get stuck infinitely in a while loop even for someone who doesn't know how the kill_kernel function behaves internally.
Other than this, I think it's good to go. I checked the behavior and read the rest of the code and see no other issues. Thanks!
There was a problem hiding this comment.
Makes sense. Change above has been pushed.
makes single-iteration clearer than while loop, as reviewed by @fperez.
Notebook cleanups and fixes: connection file cleanup, first heartbeat, startup flush. Kernels would not linger, but the KernelManagers are not garbage-collected on shutdown. This means that connection files for kernels still running at notebook shutdown would not be removed. Now, kernels are explicitly killed at server shutdown, allowing the KernelManagers to cleanup files. Small changes along the way: * disables the unnecessary (and actively detrimental) SIGINT handler inherited from the original copy/paste from the qt app. * put webapp initialization in `init_webapp` out of `initialize`, to preserve convention of there being no unique code in `initialize()`. * don't warn about http on all interfaces if running in 100% read-only mode, because no login or execution is possible. Closes #1232.
Notebook cleanups and fixes: connection file cleanup, first heartbeat, startup flush. Kernels would not linger, but the KernelManagers are not garbage-collected on shutdown. This means that connection files for kernels still running at notebook shutdown would not be removed. Now, kernels are explicitly killed at server shutdown, allowing the KernelManagers to cleanup files. Small changes along the way: * disables the unnecessary (and actively detrimental) SIGINT handler inherited from the original copy/paste from the qt app. * put webapp initialization in `init_webapp` out of `initialize`, to preserve convention of there being no unique code in `initialize()`. * don't warn about http on all interfaces if running in 100% read-only mode, because no login or execution is possible. Closes ipython#1232.
Kernels would not linger, but the KernelManagers are not garbage-collected on shutdown. This means that connection files for kernels still running at notebook shutdown would not be removed.
Now, kernels are explicitly killed at server shutdown, allowing the KernelManagers to cleanup files.
Small changes along the way:
init_webappout ofinitialize, to preserve convention of there being no unique code ininitialize().(Yay, my first PR for 0.13!)