Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data loss as notebook does not get saved #219

Open
asteppke opened this issue Dec 2, 2023 · 6 comments
Open

Data loss as notebook does not get saved #219

asteppke opened this issue Dec 2, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@asteppke
Copy link

asteppke commented Dec 2, 2023 •
edited

With good experiences we are now using the jupyter-collaboration extension (version 2.0.0a4) for a few weeks.

Unfortunately, under some circumstances, and without apparent warnings severe data loss occurs. In this case one opens a notebook in jupyterlab, edits it and it gets automatically saved. This is indicated by the file modification time and the entries in the corresponding jupyterlab logfile:

[I 2023-11-30 17:33:34.155 YDocExtension] Saving the content from room json:notebook:c96e8a68-09a2-4bf8-8a63-92553b34a88f
[I 2023-11-30 17:33:34.157 YDocExtension] Saving file: Rocking comparisons 01.ipynb
[I 2023-11-30 17:33:34.159 ServerApp] Saving Rocking comparisons 01.ipynb

After a break of several hours the connection is re-established to the notebook server, and further editing takes place. Now jupyterlab does not save the changes anymore from here on, any further input does only exist in memory. There is no warning or error from the UI side and if the server is shutdown the notebook does not contain any of the new input.

The logfile indicates a Tornado error but I am not certain if this is related. In any case no save events are logged or take place:

[I 2023-12-01 09:32:00.716 YDocExtension] Content in room json:notebook:452e70df-615e-4442-8ac6-49684152ae7e loaded from the ystore MySQLiteYStore
[I 2023-12-01 09:32:00.916 YDocExtension] Creating FileLoader for: Rocking comparisons 01.ipynb
[I 2023-12-01 09:32:00.916 YDocExtension] Initializing room json:notebook:c96e8a68-09a2-4bf8-8a63-92553b34a88f
[I 2023-12-01 09:32:00.952 YDocExtension] Watching file: Rocking comparisons 01.ipynb
[I 2023-12-01 09:32:01.157 ServerApp] Connecting to kernel bcc63d67-bca9-4d7b-974b-93efd55c863c.
[I 2023-12-01 09:32:01.958 YDocExtension] Out-of-band changes in room json:notebook:c96e8a68-09a2-4bf8-8a63-92553b34a88f
[I 2023-12-01 09:32:01.985 YDocExtension] Content in room json:notebook:c96e8a68-09a2-4bf8-8a63-92553b34a88f loaded from the ystore MySQLiteYStore
[I 2023-12-01 09:32:02.522 ServerApp] Connecting to kernel c15b9221-d24e-4ef8-9fbf-041a4eaa93b7.
[I 2023-12-01 09:32:10.945 ServerApp] Connecting to kernel e72ddd99-54b6-4418-8252-72eaf71e944e.
Task exception was never retrieved
future: <Task finished name='Task-977034' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() done, defined at /applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py:1085> exception=WebSocketClosedError()>
Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1087, in wrapper
    await fut
tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1089, in wrapper
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
Task exception was never retrieved
future: <Task finished name='Task-977053' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() done, defined at /applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py:1085> exception=WebSocketClosedError()>
Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1087, in wrapper
    await fut
tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1089, in wrapper
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
Task exception was never retrieved
future: <Task finished name='Task-977054' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() done, defined at /applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py:1085> exception=WebSocketClosedError()>
Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1087, in wrapper
    await fut
tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1089, in wrapper
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
Task exception was never retrieved
future: <Task finished name='Task-977097' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() done, defined at /applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py:1085> exception=WebSocketClosedError()>
Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1087, in wrapper
    await fut
tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1089, in wrapper
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
Task exception was never retrieved
future: <Task finished name='Task-978193' coro=<WebSocketProtocol13.write_message.<locals>.wrapper() done, defined at /applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py:1085> exception=WebSocketClosedError()>
Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1087, in wrapper
    await fut
tornado.iostream.StreamClosedError: Stream is closed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/applications/conda/envs/analysis_edge/lib/python3.11/site-packages/tornado/websocket.py", line 1089, in wrapper
    raise WebSocketClosedError()
tornado.websocket.WebSocketClosedError
[I 2023-12-01 09:32:22.712 YDocExtension] Cleaning room: json:notebook:080a189a-46f5-4518-8a69-bcb92444cf55
[I 2023-12-01 09:32:29.311 ServerApp] Connecting to kernel a16f85d0-dc99-4bcc-af0f-b57b40de73d6.
[I 2023-12-01 09:32:29.372 ServerApp] New client connecting to room: json:notebook:080a189a-46f5-4518-8a69-bcb92444cf55
[I 2023-12-01 09:32:49.042 ServerApp] Connecting to kernel dd32046e-bb57-4acc-8bdb-01c7b8418eba.
[I 2023-12-01 09:33:26.689 YDocExtension] Cleaning room: json:notebook:0b8b95e2-ed0f-41e1-94c4-9842a8c95189
[I 2023-12-01 09:33:26.690 YDocExtension] Cleaning room: json:notebook:e2e164f6-d961-496e-ad72-d7fff497a673
[I 2023-12-01 09:33:26.690 YDocExtension] Cleaning room: json:notebook:3e415acd-0ecc-4cd3-899f-c7cd63b20df6
[I 2023-12-01 09:33:26.690 YDocExtension] Cleaning room: json:notebook:452e70df-615e-4442-8ac6-49684152ae7e
[I 2023-12-01 09:33:26.690 YDocExtension] Cleaning room: JupyterLab:globalAwareness
[I 2023-12-01 09:33:26.690 YDocExtension] Cleaning room: json:notebook:f68c7ace-eb60-4b2d-95d0-2b2aa6d4b202
[I 2023-12-01 09:33:26.690 YDocExtension] Cleaning room: json:notebook:8a1347d4-2c6f-4d29-af3d-3f9441b452db
[I 2023-12-01 09:33:26.690 YDocExtension] Cleaning room: json:notebook:c96e8a68-09a2-4bf8-8a63-92553b34a88f
[I 2023-12-01 09:33:26.690 YDocExtension] Cleaning room: json:notebook:080a189a-46f5-4518-8a69-bcb92444cf55
[I 2023-12-01 09:33:27.688 ServerApp] New client connecting to room: json:notebook:0b8b95e2-ed0f-41e1-94c4-9842a8c95189
[I 2023-12-01 09:33:27.717 ServerApp] New client connecting to room: json:notebook:e2e164f6-d961-496e-ad72-d7fff497a673
[I 2023-12-01 09:33:27.785 ServerApp] New client connecting to room: json:notebook:3e415acd-0ecc-4cd3-899f-c7cd63b20df6
[I 2023-12-01 09:33:27.811 ServerApp] New client connecting to room: json:notebook:452e70df-615e-4442-8ac6-49684152ae7e
[I 2023-12-01 09:33:27.852 ServerApp] New client connecting to room: json:notebook:f68c7ace-eb60-4b2d-95d0-2b2aa6d4b202
[I 2023-12-01 09:33:27.881 ServerApp] New client connecting to room: JupyterLab:globalAwareness
[I 2023-12-01 09:33:27.912 ServerApp] New client connecting to room: json:notebook:8a1347d4-2c6f-4d29-af3d-3f9441b452db
[I 2023-12-01 09:33:27.985 ServerApp] New client connecting to room: json:notebook:c96e8a68-09a2-4bf8-8a63-92553b34a88f
[I 2023-12-01 09:33:28.064 ServerApp] New client connecting to room: json:notebook:080a189a-46f5-4518-8a69-bcb92444cf55
[I 2023-12-01 09:47:26.712 YDocExtension] Cleaning room: JupyterLab:globalAwareness
[I 2023-12-01 09:47:26.714 YDocExtension] Cleaning room: json:notebook:0b8b95e2-ed0f-41e1-94c4-9842a8c95189
[I 2023-12-01 09:47:26.714 YDocExtension] Cleaning room: json:notebook:e2e164f6-d961-496e-ad72-d7fff497a673
[I 2023-12-01 09:47:26.714 YDocExtension] Cleaning room: json:notebook:f68c7ace-eb60-4b2d-95d0-2b2aa6d4b202
[I 2023-12-01 09:47:26.714 YDocExtension] Cleaning room: json:notebook:080a189a-46f5-4518-8a69-bcb92444cf55
[I 2023-12-01 09:47:26.714 YDocExtension] Cleaning room: json:notebook:8a1347d4-2c6f-4d29-af3d-3f9441b452db
[I 2023-12-01 09:47:26.714 YDocExtension] Cleaning room: json:notebook:452e70df-615e-4442-8ac6-49684152ae7e
[I 2023-12-01 09:47:26.714 YDocExtension] Cleaning room: json:notebook:c96e8a68-09a2-4bf8-8a63-92553b34a88f
[I 2023-12-01 09:47:26.714 YDocExtension] Cleaning room: json:notebook:3e415acd-0ecc-4cd3-899f-c7cd63b20df6
[...]

As far as I have observed everything starts out correctly and I do not know who to trigger the behavior explicitly. A disconnect and re-connect to the server after some time does seem to be correlated with it though.

At least a temporary workaround seems to be to explicitly use Save Notebook As... to save the notebook under a new file name, this re-starts the autosave functionality.

How can this be debugged further?

Context

  • Linux (RHEL8)
  • Chrome
  • JupyterLab version 4.0.9
  • ystore-database is set to be written to /tmp because the notebook directory is on NFS and this caused troubles in the past
Conda environment 
jupyter-collaboration     2.0.0a4                  pypi_0    pypi
jupyter-lsp               2.2.1              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.0              pyhd8ed1ab_0    conda-forge
jupyter_core              5.5.0           py311h38be061_0    conda-forge
jupyter_events            0.9.0              pyhd8ed1ab_0    conda-forge
jupyter_server            2.11.1             pyhd8ed1ab_0    conda-forge
jupyter_server_fileid     0.9.0              pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.4.4              pyhd8ed1ab_1    conda-forge
jupyter_ydoc              1.1.1              pyhd8ed1ab_0    conda-forge
jupyterlab                4.0.9              pyhd8ed1ab_0    conda-forge
@asteppke asteppke added the bug Something isn't working label Dec 2, 2023
@jtpio jtpio mentioned this issue Dec 12, 2023
Open
@jtpio
Copy link
Member

jtpio commented Dec 14, 2023

Thanks @asteppke for the report.

Would you be able to try with the new 2.0.0 release, to see if it might fix the issue? https://github.com/jupyterlab/jupyter-collaboration/releases/tag/v2.0.0

@tsnow03 tsnow03 mentioned this issue Jan 25, 2024
Closed
@tsnow03
Copy link

tsnow03 commented Jan 25, 2024 •
edited

Using jupyter-collaboration v2.0.1 we are seeing similar problems. When this extension is added to our Docker environment, the Python notebooks within JupyterLab 4.0.11 lose the ability to click the save button in the notebooks as well as the Save Notebook option in the File dropdown. Notebooks also stop autosaving. This seems to be stemming from jupyter-collaboration causing a disabling of the @jupyterlab/filebrowser-extension:defaultFileBrowser extension. See CryoInTheCloud/hub-image#104 for details.

@davidbrochart
Copy link
Collaborator

the Python notebooks within JupyterLab 4.0.11 lose the ability to click the save button in the notebooks as well as the Save Notebook option in the File dropdown.

This is expected.

Notebooks also stop autosaving.

This can happen if your file is automatically loaded when you open JupyterLab and the previous session was with RTC disabled. In this case, try closing the file and opening it again.

@tsnow03 tsnow03 mentioned this issue Jan 26, 2024
Merged
@tsnow03
Copy link

tsnow03 commented Jan 29, 2024 •
edited

Thank you so much for your response! This was tested in Binder so nothing had been previously opened, but I did try creating and opening a new notebook, closing and re-opening and it still did not have the ability to press the save button. Any other possibilities? You are welcome to test this out on Binder in this pull request I just made if you want to try it out. I'm also happy to just paste in a screen shot of it.

@davidbrochart
Copy link
Collaborator

davidbrochart commented Jan 29, 2024 •
edited

This is the normal behavior, manual saving is disabled in collaborative mode, as the document is automatically saved after each change.

@tsnow03
Copy link

tsnow03 commented Jan 30, 2024

Ah ok that makes sense. The autosave does seem to be working as expected. Thank you for identifying that for me!

@fcollonval fcollonval mentioned this issue Feb 1, 2024
Open
This was referenced Feb 28, 2024
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jtpio @davidbrochart @asteppke @tsnow03