Metadata linked to view state and Real-Time Collaboration

[fcollonval]

Problem

In the official notebook format, they are metadata at the cells level that are related to view state.

---

The following metadata keys are defined at the cell level:

Key	Value	Interpretation
collapsed	bool	Whether the cell’s output container should be collapsed
scrolled	bool or ‘auto’	Whether the cell’s output is scrolled, unscrolled, or autoscrolled
deletable	bool	If False, prevent deletion of the cell
editable	bool	If False, prevent editing of the cell (by definition, this also prevents deleting the cell)
format	‘mime/type’	The mime-type of a Raw NBConvert Cell
name	str	A name for the cell. Should be unique across the notebook. Uniqueness must be verified outside of the json schema.
tags	list of str	A list of string tags on the cell. Commas are not allowed in a tag
jupyter	dict	A namespace holding jupyter specific fields. See docs below for more details
execution	dict	A namespace holding execution specific fields. See docs below for more details

The following metadata keys are defined at the cell level within the jupyter namespace

Key	Value	Interpretation
source_hidden	bool	Whether the cell’s source should be shown
outputs_hidden	bool	Whether the cell’s outputs should be shown

---

Synchronizing those in Real Time Collaboration may be a weird user experience; for example if a user decides to hide an output, another one that was looking at it may get confused and re-expand it.

There is also the special case of trusted - it is for now transiently stored in the cell metadata; aka the server add it when loading a file and then the frontend removes it when the cell model is instantiated. This is hard to work with in RTC because you cannot remove it as user will not get the proper notebook state if they connect on an already opened notebook. On the other end, if we don't remove it, it feels wrong to get the trusted key in the cell metadata editor.

A solution for the trusted case would be to use the state map of the shared document.

Additional context

Some previous discussion at the notebook level: #12052

[williamstein]

Synchronizing those in Real Time Collaboration may be a weird user experience; for example if a user decides to hide an output, another one that was looking at it may get confused and re-expand it.

For what it is worth, I had to address exactly this problem in CoCalc. The solution I implemented in all cases is that everything is synchronized in realtime, e.g., if a user hides the output then it is hidden for everybody. We have not received any complaint about this, so it's hopefully not a horrible mistake. Personally, when I implemented this, I never thought any other behavior would make sense. In any case, if you want to see what it feels like, you can try it out in CoCalc. (never mind, this is already how Jupyter RTC works.) Wait, currently in Jupyter RTC collapsing collapses output for everybody, but expanding only expands for the current user. Weird.

A good invariant is (*) "when I refresh my browser I get back what I was looking at". That's very much broken in current Jupyter RTC, since if one user collapses and expands the output of a cell, then the other user sees it as collapsed still. However, if they then refresh their browser, it becomes expanded. The invariant (*) implies that this view state stuff should all be synced, I think.

[ellisonbg]

@williamstein thanks for your perspective here, that is really valuable to hear that your users haven't complained about that behavior. Based on that, I think it is a good starting point to sync this state in the shared model among users. Over time, as we get more feedback from users we can consider other options as needed.

[firai]

Out of curiosity, is the active view anchored to the scroll position in the notebook or the active cell? Could collapsing or expanding particularly long cells/output by one user cause the cells that other users are viewing to be bumped out of view if the collapse state is actively synced, if the cells that are being viewed by others are below the cell that is modified?

Seems like collapsing/expanding by one user causing others' active views to be bumped out of view could only happen if the collapsed/expanded cell was within view before the operation (even if just slightly), which I suppose is either desirable (viewing the same cell) or rare (just happen to be looking at the cell below or couple of cells below). Otherwise, the view is unaffected (maybe because of the virtual notebook?).

Wait, currently in Jupyter RTC collapsing collapses output for everybody, but expanding only expands for the current user. Weird.

It seems like both collapsing and expanding are synced in my instance?

[fcollonval]

For now we are moving forward with synchronizing everything.