Remove modeldb

[dmonad]

This PR implements a lightweight alternative to #12359. As we discussed last year, we wanted to remove modeldb as it maintains the same information as shared-model. This PR does just that.

As we can remove modeldb, we can make the shared-models package the only source of truth, which will greatly simplify double-binding logic and can help us to avoid the initialization problem (concurrent initializations by different peers can lead to users overwriting content). There should be no syncing back-and-forth between, for example, the notebook widgets and shared models. When somebody wants to add a new widget, they need to modify the shared notebook model, which will trigger the same event handlers as if there would be a remote insertion of a new cell. This simplifies the codebase + we can throw out duplicate implementations of the same functionality (overall this PR removes 1300 lines of code).

The most important contribution of this PR is that it completely avoids the initialization problem. It is important for us that there is only one way to initialize content (via shared-models). The shared-model package initializes a document safely with a single cell. That means users don't have to wait for the server to respond before editing. This PR removes all initialization code that we previously used, which was fairly problematic in shared editing scenarios.

Future PRs could move the shared models defined here again into separate packages (e.g. into the cells package as #12359 does). Although I recommend keeping the shared state separate from the application-specific state (e.g. models containing dirty flag, mimeType, selections, and another application-specific state like the editor instance that renders the model). This also enables others to reuse the shared-models package.

Code changes

Throughout the codebase, there are many places where we follow a fairly complicated event flow. This PR simplifies the event flow and enforces that the shared model is the only source of truth for all editor content. This leads to a significant reduction in the number of lines as we can avoid a lot of the complicated "double-binding" logic (syncing one state with another).

This PR retains the double-binding logic for metadata (which is an ObservableMap). It gives users the ability to have fine-grained access to metadata that many depend on. Ideally, we move the metadata field into the shared-model package and remove the double-binding logic. This can be done in a future PR.

User-facing changes

None

Backwards-incompatible

We remove the modeldb API, which might be relevant to extension authors.

---

@hbcarlos @davidbrochart @SylvainCorlay @echarles

[jupyterlab-probot]

Thanks for making a pull request to jupyterlab!
To try out this branch on binder, follow this link:

[davidbrochart]

Thanks @dmonad.
It looks like you need to rebase.

[echarles]

Thx for working on this @dmonad. The intent as shown in the first note of this PR sounds clear to me an a good step forward.

I have quickly scanned the source, and the changes are bringing more clarity and simplification as expected. I am happy to dig more and bring concrete reviews depending on the evolution of the discussion happening in jupyterlab/frontends-team-compass#150 and jupyterlab/frontends-team-compass#149

I see a lot of calls like ...model.sharedModel... (eg editor.model.sharedModel.getSource()). At some point, we could think to add shortcut methods to get rid of that call and simply expose editor.model.getSource() if possible. The whole idea would be to have a model hosted in the shared-models package that can be access by the other packages.

This PR is an excellent initiative to have something concrete to discuss and take a decision on the next steps.

[SylvainCorlay]

Many thanks to @fcollonval for completing this!

[fcollonval]

Important note the memory leak job highlight that a notebook is not cleared from memory when closed. This should be fixed before merging.

[fcollonval]

bot please run benchmarks

[fcollonval]

bot please update documentation snapshots