HA: Disable replication from old main

[antoniofilipovic]

The goal of this PR is to disable replication from the old main once it comes back up. We are introducing uuid as a discriminating factor between databases (not just storages but whole databases). When the old main dies, we first swap uuid and set it to uuid of the new main, then we proceed with failover to avoid the old main still replicating data to the instance.

The coordinator will now be the owner of the information of the main uuid. This means the coordinator will generate uuid and send it to the main on PromoteReplicaToMainRpc.

In regular replication without a coordinator, each time a replica is registered, the main will send its own id.

TODO:

• On replica restart set uuid to none to not accept messages in HA cluster

[master < Task] PR

• Check, and update documentation if necessary
• Provide the full content or a guide for the final git message

To keep docs changelog up to date, one more thing to do:

• Write a release note here, including added/changed clauses
• Tag someone from docs team in the comments

[as51340]

Great work, however I would spend some time in making code more consistent and readable 💪

[andrejtonev]

💪

[antoniofilipovic]

@kgolubic RN: REPLICAs now listen only to one MAIN instance. When using replication as part of high availability, coordinator orchestrates which instance is REPLICA listening to. When using only replication without HA, main sends its ID to each REPLICA which REPLICA listens to.