Schema Versioning and Migrations

[ThePaulMcBride]

Hey folks,

Versioning

I've been working on a project that requires some form of schema versioning. With the way prosemirror works, if a user receives an update that contains data that is not supported in their schema, it gets deleted. So we need some way to let a connected user know that someone else has joined with a newer version and that they should reload.

I've seen several discussions about this already, both in the Discord and here on Github.

As far as I can tell, there is no official solution or guidance on this kind of thing but its feels like a core need for a realtime collaboration tool.

Can we use this discussion to try and pool together some knowledge on this and help the team come up with a baked in solution?

I'm gonna tag a few people that I've seen chatting about this, I hope you don't mind:
@jamesopti @janthurau

Some previous discussions:

• How can I notify the client if the user no longer has access to the document during editing? #130
• How to implement validation / disaster recovery? #135

Migrations

Related to this is schema migrations.

Ideally we would all be writing apps where we know exactly what we want to build form the start, but unfortunately that is not the case. So at some point it's likely we'll need to make a break change to the document schema. Again, prosemirror is pretty good at recovering from this by just deleting node it doesn't understand. That isn't alway ideal though, so we'd need some way to write migrations. Is this something that should be part of tiptap and/or hocuspocus?

[levkus]

Hi! I don’t have a solution yet, but I’d like to add to describing the problem with my case.

We have a custom mark extension that we use to “tag” parts of the documents with some additional data. After the user applies these marks, we use the onStoreDocument hook of HocusPocus to parse the document and convert these marks to JSON objects that get stored in a separate database. This way we get a single source of truth across all users collaborating on a document.

However, If in some moment in time, a user with an older version of frontend goes to the page with a document that has these marks, their instance of Prosemirror will delete these marks, which will trigger the onStoreDocument hook and as a result delete the objects I mentioned above from the database.

This seems really dangerous, and that’s why I started to think about ways to prevent it. Right now the only solution I see is to force users to update frontend (reload the page) in case a new schema version gets released and writing some sort of migrations on the older documents to be able to eventually delete older code.