Kernel info request is blocked after a long execute followed by a page reload

[jtpio]

For now the kernel_info_request message is sent on the shell channel.

This can lead to the following behavior when a cell is executed, keeping the kernel in a busy state for some time, and the page reloaded right after:

1. Open a notebook with ipykernel as the kernel
2. Execute a cell that takes some time to execute. For example:

import time
time.sleep(99999)

3. Reload the page
4. After the page is reloaded, JupyterLab sends a new kernel_info_request on the shell channel.
5. However since the kernel is currently executing the code from the previous execute_request, it can't process the kernel_info_request and send the kernel_info_reply
6. The kernel is then restarted every 10 seconds, and a new WebSocket is recreated.

This is also visible in the server logs:

[I 15:14:33.500 LabApp] Adapting from protocol version 5.1 (kernel f391e165-dad1-43b0-8f8d-ea426dfa4077) to 5.3 (client).
[I 15:14:43.532 LabApp] Adapting from protocol version 5.1 (kernel f391e165-dad1-43b0-8f8d-ea426dfa4077) to 5.3 (client).
[I 15:14:53.563 LabApp] Adapting from protocol version 5.1 (kernel f391e165-dad1-43b0-8f8d-ea426dfa4077) to 5.3 (client).
[I 15:15:03.571 LabApp] Adapting from protocol version 5.1 (kernel f391e165-dad1-43b0-8f8d-ea426dfa4077) to 5.3 (client).
[I 15:15:13.601 LabApp] Adapting from protocol version 5.1 (kernel f391e165-dad1-43b0-8f8d-ea426dfa4077) to 5.3 (client).

One solution would be to send the kernel_info_request on the control channel instead of shell. According to the "Messaging in Jupyter" documentation:

Control: This channel is identical to Shell, but operates on a separate socket to avoid queueing behind execution requests. The control channel is used for shutdown and restart messages, as well as for debugging messages.

[jasongrout]

We do the kernel info request for several reasons:

1. It triggers a kernel status message, so that we can get into a known kernel state.
2. It gives us information about the kernel (like the help menu links).

Right now, IIRC, we block the kernel from being "ready" until that info reply comes back.

With #7252, the kernel is immediately ready as soon as the websocket is connected. While we still issue the kernel info request, you only care if you are explicitly waiting on the new kernel connection info promise for the reply (for example, for the help menu links). I think this should make the situation better.

[jasongrout]

I don't know how sending the request on the control channel will help in a single-threaded kernel, since execution is blocking the control channel as well. I think the problem lies in the frontend that insists on the kernel info reply coming back, and I think #7252 addresses that (or should address it).

[jtpio]

While we still issue the kernel info request, you only care if you are explicitly waiting on the new kernel connection info promise for the reply (for example, for the help menu links). I think this should make the situation better.

That would make the situation better yes. But there are still some cases where a kernel info is useful (and that means waiting for the kernel_info_reply), for example looking up a potential debugger key.

I don't know how sending the request on the control channel will help in a single-threaded kernel,

Indeed it doesn't help in a single-threaded kernel. But some kernels process messages from the control channel in a different thread.

[jasongrout]

Are you asking if you can send a kernel info request on the control channel, or are you asking if we should send the initial request on the control channel by default?

edited for clarity

[jtpio]

Not sure yet. We could wait for #7252 to land first which already helps.

For now kernel_info_request is documented as being a message on the shell channel. In the long run maybe it could be sent on both shell and control (and the client can choose)?

[jtpio]

Linking to this JEP suggesting moving the kernel_info request to the control channel proposal: jupyter/enhancement-proposals#82

[krassowski]

FYI jupyter/enhancement-proposals#82 was merged, can we switch kernel_info_request to the control channelt now, or do we need to wait until support for this is implemented (in jupyter-client/kernels)?