Summary
The current backup/restore process does not reliably preserve and restore the Shuttermint validator key. This can result in a restored Keyper syncing successfully and appearing healthy, while no longer controlling the validator public key that is part of the active Shuttermint validator set.
Problem
During the API validator-set recovery work, we found that some restored/resynced Keypers were able to sync the chain, but were not signing Shuttermint consensus blocks.
The likely cause is that the Ethereum key is restored, but the Shuttermint validator key is not restored correctly. If priv_validator_key.json changes, the Keyper gets a different Shuttermint validator address/public key and can no longer sign for its existing validator slot.
This is particularly problematic when the chain is stuck, because a new validator public key can only be applied through a validator update after the chain progresses.
Required Fix
Fix the backup/restore process so that the Shuttermint validator key is preserved and restored correctly.
The process should explicitly include and validate:
priv_validator_key.json
- validator address
- validator public key
- relevant validator state handling for safe resync/restore
Verification
Add explicit checks before and after restore:
- Record the Shuttermint validator address before backup/restore.
- Record the Shuttermint validator public key before backup/restore.
- After restore, verify that both values are unchanged.
- Fail if the restored validator key does not match the original one.
Acceptance Criteria
- A restored Keyper keeps the same Shuttermint validator address and public key.
- Restore instructions include a clear validator-key verification step.
- If the validator key is missing or changed, the process surfaces this explicitly.
- The process does not silently generate or use a new validator key when an existing validator key should be restored.
- This needs to be reproduced on both DAppNode packages (Gnosis and API)
Summary
The current backup/restore process does not reliably preserve and restore the Shuttermint validator key. This can result in a restored Keyper syncing successfully and appearing healthy, while no longer controlling the validator public key that is part of the active Shuttermint validator set.
Problem
During the API validator-set recovery work, we found that some restored/resynced Keypers were able to sync the chain, but were not signing Shuttermint consensus blocks.
The likely cause is that the Ethereum key is restored, but the Shuttermint validator key is not restored correctly. If
priv_validator_key.jsonchanges, the Keyper gets a different Shuttermint validator address/public key and can no longer sign for its existing validator slot.This is particularly problematic when the chain is stuck, because a new validator public key can only be applied through a validator update after the chain progresses.
Required Fix
Fix the backup/restore process so that the Shuttermint validator key is preserved and restored correctly.
The process should explicitly include and validate:
priv_validator_key.jsonVerification
Add explicit checks before and after restore:
Acceptance Criteria