Reconnect to running session: keeping output

[oersted]

There's been a lot of requests in the past to:

• Reconnect to a long-running notebook and keep receiving new output.
• Reconnect to a notebook running on the server side and recover all output when reconnecting.

I'd like to resurface this issue. To the best of my knowledge, these features have been planned at least since 2015 (earliest reference I found: link). It hasn't been implemented yet because it directly conflicts with the architecture of the stack and would require a big refactor.

I don't know if this is easier in the JupyterLab context or not. But now that this project is in very active development, it might be a good opportunity to revisit this and try to get it in now at the foundations.

I found to open issues on the topic in the old notebooks repo:

• Reconnect to running session jupyter/notebook#1150
• Restoring computation output after disconnect in Notebook jupyter/notebook#641

[blink1073]

Hi @oersted, this work needs to be done on the server side, and JupyterLab uses the same server as the classic notebook, so the work to be done is the same.

[grahamanderson]

Is there some kind of alternative?
For myself, I have long running (data-science-y and scraping) Jupyter notebooks running (remotely) on a GCloud compute instance.
Kind of defeats the purpose if the script halts the moment I close my macbook. Or, am I mistaken?
On my end, when I lose browser connection, I can't reconnect to the Jupyter notebook without halting the script and losing output
Is there a plausible work-around?

[vidartf]

You could make a python client ("headless frontend") that auto-saves when new messages are received, and automatically shuts down when last pending execution is finished. If you spin this up in a separate process (from the notebook) before starting your execution, I think you should be able to do what you want. But this is not really a JupyterLab issue.

[ZmeiGorynych]

So am I correct in understanding that when you close the browser, the notebook keeps running, there is simply no way to connect to it again to view the output? If so, the logical thing to do is to have it dump summaries to file every so often, and use another, quick-to-restart notebook to display these.

[grahamanderson]

Thanks for responding :) Perhaps, it's not a JuypterLab specific issue.
That said, what is the process for...

1. executing code on an AWS/GCloud hosted notebook
2. putting your macbook to sleep
3. logging back into the notebook as it's running code?

How do Jupyter experts work with long running (remotely hosted) scripts?

Am I supposed to login to the notebook--only after the scripts have finished? Sometimes, scripts takes days to execute...

On my end, my code is inserting rows into a SQLite3 (which I can check) and sending me notifications by pushbullet. So, there are notifications. I'm also experimenting with KeepAlive Settings and Tmux on both ends (local and Gcloud)

If this question is not appropriate to JupyterLab (which is currently open in Chrome), what is an appropriate forum to ask this question?

[blink1073]

Hi @grahamanderson, I think the mailing list would be the best forum.

[Sarnath]

so, will this issue be addressed? Its so irritating when you have no idea what is happening on your long-running notebook. :(

[blink1073]

@Sarnath, this will be implemented when we implement real time collaboration, which is in progress.