Enable fast (pipelined) synchronization in TypeScript client

[KtorZ]

Describe your idea, in simple words.

We could have a 'startSync' / 'stopSync' functions which would use requestNext & findIntersect under the hood to run a fast asynchronous synchronization.

Why is it a good idea?

The thing is (and I should document that better in the user-manual for it's only barely mentioned in a note) that consuming the entire chain using non-pipelined request is slow (about ~3h for the whole chain). But, Ogmios and WebSocket supports pipelining (up to a certain level) to make this much better.

So, typically when syncing a large number of blocks, one would want to make sure to always have some messages in flights. I haven't really tested much what's the best settings are, mostly because it also depends on the underlying connection and network latency. But as a rule of thumb, starting with a burst of requestNext and then, yielding a new requestNext on each event makes better use of the network bandwidth (bringing a full sync down to ~15 minutes).

There's some example of that in the chain-sync smoke tests in vanilla-js: https://github.com/KtorZ/cardano-ogmios/blob/master/server/static/tests/chain-sync.js#L82-L93

Are you willing to work on it yourself?

👍

[rhyslbw]

Understood. I can work on this in a follow-up PR.