Allow running a sequencer node that syncs from DA first in full node mode

[Manav-Aggarwal]

To increase reliability of a sequencer node, in case of losing associated data, a new sequencer node should be able to be started which syncs blocks from DA first (or from starts from a snapshot and then syncs remaining blocks from DA). Once it's done syncing, it should switch to sequencer mode to start producing blocks.

This sort of requires switching node type mid-process.

[tac0turtle]

this seems more like it needs testing, as it should be supported today.

the main thing is that the sequencer needs to stop producing blocks in order to make sure everything is posted to DA otherwise the other node will never catch up

[Manav-Aggarwal]

yes maybe it can be something we can add as a flag for the sequencer, so if certain flags are added alongside the aggregator flag, it runs the full node mode first and retrieved blobs, then once it's caught it up, it switches the mode automatically. Notice that in the block manager, the aggregator flag right now controls which go routines run on startup, so this requires playing a bit around with it to stop and start certain go routines.

Albeit, doing it manually first of running a full node and then manually stopping and starting with the aggregator flag is a good first step.

[tac0turtle]

yes maybe it can be something we can add as a flag for the sequencer, so if certain flags are added alongside the aggregator flag, it runs the full node mode first and retrieved blobs, then once it's caught it up, it switches the mode automatically. Notice that in the block manager, the aggregator flag right now controls which go routines run on startup, so this requires playing a bit around with it to stop and start certain go routines.

this leads to double signing, how does the node know that the other aggregator has stopped? I dont usually have a preference but in this case id rather not try to automate this since its so delicate that manual fixes. Its similar to creating a proxy server in a backend or just using nginx. Simplest is always the best.

[tac0turtle]

To add context to this issue, we need a way to let the aggregator run the sync loop in order to sync to the head of the chain, once it sees its at the head it can start proposing. In order to make this secure we need a way to let other nodes know what the head of the chain is to allow them to know if they are syncing old blocks or are at the head. I dont believe this exists today

[tac0turtle]

@alpe this is overlapping with the HA system correct?

[julienrbrt]

The system needed here is very similar to what we need for #3016 FYI.