global-synchronizer/production-operations/validator-disaster-recovery#validator-disaster-recovery

[giscus[bot]]

global-synchronizer/production-operations/validator-disaster-recovery#validator-disaster-recovery

https://docs.canton.network/global-synchronizer/production-operations/validator-disaster-recovery#validator-disaster-recovery

[stas-sbi]

• PartyID is <party_hint>::<fingerprint_of_public_key>
• By default a Party with PartyID is hosted on a participant PAR::<party_hint>::<fingerprint_of_public_key>
• Procedure of recovering balances using identities dump moves the party to another participant, e.g. PAR::<another_participant_id>::<the same as before fingerprint_of_public_key>
• For the restoration procedure to work identities dump should have id set to the id of participant which is currently hosting the party. Use the following API call to Scan in order to find out the participant id and update the identities dump:

  SCAN_URL=https://scan.sv-1.dev.global.canton.network.TLD
  domain_id=$(curl -fsS "$SCAN_URL/api/scan/v0/dso" | jq -r '.dso_rules.domain_id')
  party_hint=myvalidator
  party_id="$party_hint::1220YYYY"
  
  participant_id=$(
    curl -fsS "$SCAN_URL/api/scan/v0/domains/$domain_id/parties/$party_id/participant-id" |
      jq -r .participant_id
  )
  
  cat /tmp/identities.json |
    jq --arg participant_id "$participant_id" '.id = $participant_id' > /tmp/identities.updated.json

  and then the recovery could be done like this (example for docker compose):

  MIGRATION_ID=1
  SPONSOR_SV_URL=https://sv.sv-1.dev.global.canton.network.TLD
  participant_name=$(echo "$participant_id" | awk -F :: '{print $2}')
  NEW_PARTICIPANT_NAME="this-is-my-new-participant"
  
  echo "Migrating $PARTY_HINT from $participant_name to $NEW_PARTICIPANT_NAME"
  ./start.sh -w -s "$SPONSOR_SV_URL" -o "" -p "$PARTY_HINT" -m "$MIGRATION_ID" -P "$NEW_PARTICIPANT_NAME" -i /tmp/identities.updated.json

[martinflorian-da]

For the restoration procedure to work identities dump should have id set to the id of participant which is currently hosting the party

AFAICT this is already reflected in the current state of the docs (I'm working on a PR to improve the wording though).

I suggest keeping @stas-sbi's comment here as documentation of "find out the participant ID via scan". For the official docs I'd instead document getting it from a Canton console, for consistency with the rest of the instructions.

[stas-sbi]

Reading the page very carefully I managed to find this (I'm guessing this is the part you are referring to):

If your node is in a state where you cannot take a fresh dump, use the old dump but edit the id field in your identity dump to the participant id of the new node

This doesn't explain what "the new node" is and how to get the id. One would consider "the new node" to be the participant they are trying to restore to.

[stas-sbi]

Summarizing there are three participant nodes we are referring to:

• original (id of the participant the dump has been taken from)
• current (id of the participant currently hosting the identity as recorded in the topology regardless of whether the node itself is alive)
• new (the participant the identity will be moved to)

[martinflorian-da]

@stas-sbi Sorry for the late response; I think that you are referring to an older state, before my recent wording improvement? I do hope that I made the namings of the nodes more clear but am open for feedback of course. Here is the PR (for the diff) that is included in 0.4.15: canton-network/splice#2054

[branciard]

For "Recover the Coin balance of an external party"
In the Warning bloc

The target validator that you use to host the party after recovery must be a completely new validator

The new validator party hint must be the same as old validator where the external party was hosted or can be another party hint ?

[HuzShakir-Wolf]

Is it possible to recover a replicated party (not originally hosted on the participant) from node-identities backup?

Let's say the original participant is terminated (where the party was generated) and the party is only alive on the new participant. Can we still recover that party with an ACS snapshot from the scan API?

[AmardeepSambaru]

Hi Team,

I am a node operator and we are new to Canton. We are currently planning to migrate a validator from one server to another and would appreciate your guidance on the complete process.

Could you please provide detailed steps from start to finish? Specifically, we would like to understand:

What components and data need to be backed up (validator keys, configuration, database, etc.)
Any prerequisites or checks to perform before initiating the migration
The correct procedure to safely stop the existing validator and bring up the new one
How to ensure there is no downtime or double-signing risk during the transition
Any configuration changes required on the new server
Post-migration validation steps to confirm everything is working correctly

Additionally, if there are any best practices, common pitfalls, or recommended documentation, please share those as well.

Your guidance will help us ensure a smooth and secure migration.

Thanks.

[stas-sbi]

This doc is supposed to be comprehensive. I can suggest to go through the described steps on an experimental deployment. If you find any steps missing or not descriptive enough, feel free to comment here and we'll see how we can improve it.

[AmardeepSambaru]

Hi

I’m a bit confused.Could you please clarify which of the following methods is appropriate for migration?

Restoring a validator from backups
Recovery from an identities backup (re-onboarding a validator and recovering user balances)
Disaster recovery from loss of the CometBFT storage layer of the global synchronizer