'sync' event is fired before document is loaded

[Flamenco]

Describe the bug

Sync is called with isSynced true, but my document item is empty. This behavior is intermittent.

I need to add a delay before accessing the document.

Here is a helper function that includes a delay that resolves the problem. Without the delay, my doc.getText().length is sometimes '0', even though a document exists in the backend.

My test show the bigger the document, the more delay i need to add.

For reference, my backend is the published server example using a leveldb store.

await waitForSync()
console.log(doc.getText().length)

async function waitForSync() {
    return new Promise((resolve, reject) => {
      if (wsProvider.synced) {
        setTimeout(() => {
          return resolve(undefined)
        }, 200)
      }
      wsProvider.once('sync', () => {
        setTimeout(() => {
          resolve(undefined)
        }, 200)
      })
    });
  }

[dmonad]

Hi @Flamenco,

The "synced" event is fired when the client received the initial state from the server. However, it seems that the server loads the content from leveldb after the client is already synced. This is most likely a bug in the y-leveldb integration.

[Flamenco]

@dmonad OK I will look under the hood in the leveldb code. I'm closing this for now.

[Flamenco]

@dmonad,

y-websocket has this code:

    websocket.onopen = () => {
      provider.wsLastMessageReceived = time.getUnixTime()
      provider.wsconnecting = false
      provider.wsconnected = true
      provider.wsUnsuccessfulReconnects = 0
      provider.emit('status', [{
        status: 'connected'
      }])
      // always send sync step 1 when connected
      const encoder = encoding.createEncoder()
      encoding.writeVarUint(encoder, messageSync)
      syncProtocol.writeSyncStep1(encoder, provider.doc)
      websocket.send(encoding.toUint8Array(encoder))
      // broadcast local awareness state
      if (provider.awareness.getLocalState() !== null) {
        const encoderAwarenessState = encoding.createEncoder()
        encoding.writeVarUint(encoderAwarenessState, messageAwareness)
        encoding.writeVarUint8Array(encoderAwarenessState, awarenessProtocol.encodeAwarenessUpdate(provider.awareness, [provider.doc.clientID]))
        websocket.send(encoding.toUint8Array(encoderAwarenessState))
      }
    }

So it appears as you mentioned, sync is sent before any documents are actually loaded from persistence.

[grootgordon]

I met the same problem when yjs server load doc history data from MySQL Database.
It's look like client request history data failed, but the websocket connect server successfully.
I suspect the server-side authentication caused this error, in my case: the yjs server call another auth server and wait for result.

How to fix:

1. workaround way
   use resyncInterval option for sync interval to server
   more explain: https://github.com/yjs/y-websocket/blob/master/src/y-websocket.js#L248
   and resyncInterval default is -1, that means,never resync!

export const provider = new WebsocketProvider(
 requestURL,
 roomName,
 doc,
 {
   connect: true,
   resyncInterval: 1000,
   disableBc: true
 }
);

2. server side need to cache client msg to a queue, when auth done, handle the cache msg

[thecodingwizard]

I ran into this issue as well. I believe it was because in the default implementation of the y-websocket server, sometimes the sync events are fired before the document is loaded from leveldb.

In particular, on this line: https://github.com/yjs/y-websocket/blob/master/bin/utils.js#L37 ldb.getYDoc returns a promise, but getYDoc in utils.js does not wait for this promise to resolve before returning the document, which causes sync step 1 to be sent before the document is loaded from leveldb (an empty document is used instead). Then, sync step 2 is sent, leading the client to believe the document is synced (even though the document is still empty). Finally, leveldb is loaded, and the document is updated on the server and synced with the client.

I fixed this by modifying getYDoc in utils.js to return both the document and a promise for when leveldb finishes loading the document, and the server will wait until the document is loaded before processing messages (a hacky implementation of this can be found here: cpinitiative/ide@d4380ce and a bugfix here: cpinitiative/ide@48b4240)

[totorofly]

I've encountered the same issue, where occasionally the ProseMirror document initially renders with blank content (having the corresponding schema's type),

• firstTime with blank content:

and then waits until the full document content is loaded from the y-websocket server before overlaying it on ProseMirror.

• secondTime with full document content

Is there any plan from the official team to address this issue?