feat: Use notify push for sync messages during editing

[juliushaertl]

Signed-off-by: Julius Härtl jus@bitgrid.net

📝 Summary

• Resolves: Using notify_push for syncing #1805

This is a first PoC to make use of notify_push for syncing steps back to other editors instead of regular polling. We still poll every 30 seconds as a fallback and to ensure that the autosave requests are still triggered until #4424 is there (even with that some fallback sync might still be worth).

🚧 TODO

• More manual testing in different scenarios
  • Check what happens with the base doc changing externally (did we use the sync to detect the conflict?)

🏁 Checklist

• Code is properly formatted (npm run lint / npm run stylelint / composer run cs:check)
• Sign-off message is added to all commits
• Tests (unit, integration and/or end-to-end) passing and the changes are covered with tests
• Documentation (README or documentation) has been updated or is not required

[cypress]

1 flaky tests on run #11336 ↗︎

0

149

2

0

1

Details:

feat: Use notify push for sync messages during editing
Project: Text	Commit: 90c956236b
Status: Passed	Duration: 04:17 💡
Started: Jul 27, 2023 6:27 PM	Ended: Jul 27, 2023 6:32 PM

  cypress/e2e/nodes/Links.spec.js • 1 flaky test

View Output Video

Test		Artifacts
... > Link website with selection		Output Screenshots

_{This comment has been generated by cypress-bot as a result of this project's GitHub integration settings.}

[max-nextcloud]

I would expect to just add the steps that just arrived. Say we start at version 0 and two users push their steps without syncing in between:

• user1 pushes step1
• user2 pushes step2

Now the first push will add [step1] and the second push will add [step1, step2] to the queue because user2 still has version 0.

Or in other words.. the content of the queue gets send to all users and what is added to it should not depend on the version of the user adding it.

[juliushaertl]

Yes, definitely, so we probably need some sync calls to still happen to catch up in the beginning and maybe some kind of detection of missing steps (if we can)?

Otherwise maybe there is a way to make use of the y.js native catch up https://github.com/yjs/y-protocols/blob/master/sync.js#L15-L17

[max-nextcloud]

y-websocket has a timer that will send Sync messages of type 1 periodically:
https://github.com/yjs/y-websocket/blob/35be3431b8db14f07dfaacab5dfa1a4cdec13fac/src/y-websocket.js#L300-L311

We could enable that. But if the frequency is too high it will create quite a bit of load as all clients would query all other clients. If the frequency is too low it will take too long to resync to be helpful.

My favorite approach right now would be for the saving client to also save a Sync message type 1 alongside the current state. Then every client can get that as a baseline and apply the saved state and reply to the message type 1 with a message type 2 that will contain its local diff.

[juliushaertl]

This should now be resolved by just using the notify_push queue as a fast path to distribute any incoming messages to other and the own clients.

[max-nextcloud]

Thanks a lot for looking into this. I'm really curious to play with it. 😍

We still poll every 30 seconds as a fallback and to ensure that the autosave requests are still triggered until #4424 is there (even with that some fallback sync might still be worth).

Just one remark. Autosave is independent from the polling backend these days. It's triggered by

text/src/services/SyncService.js

Line 91 in 14d29b1

this.autosave = debounce(this._autosave.bind(this), AUTOSAVE_INTERVAL)

Never the less I agree that having a fallback for a start might still be worthwhile.