Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Begin Dynamic Rescan of Validating Keys #6963

Merged
merged 75 commits into from Aug 31, 2020
Merged

Conversation

rauljordan
Copy link
Contributor

@rauljordan rauljordan commented Aug 11, 2020

What type of PR is this?

Feature

What does this PR do? Why is it needed?

This PR adds a much requested feature of being able to dynamically add new validator keys while a validator client is running. We accomplish this to a process called rescanning, which looks for changes to our all-accounts.keystore.json file. This allows for straightforward, dynamic key management for validator clients by allowing us to run accounts-v2 import while a validator process is running.

  1. Changes DB logic to rescan for new validator keys and create appropriate buckets for slashing protection
  2. Listens filesystem changes to all-accounts.keystore.json and reloads validator keys at runtime if the file changed. This file can be changed by importing new keys into a running validator client by using prysm.sh validator accounts-v2 import
  3. Includes a new package, asyncutil, which adds the function Debounce to a Go channel, allowing us to debounce a large number of events sent over a channel and instead only process a handler after a set interval passes.

Which issues(s) does this PR fix?

Fixes #5202

Works at Runtime

Confirmed this functionality works at runtime. Started my validator client with a single key, then ran a script which runs import 1023 separate times for 1023 separate keystore files, and the validator client would debounce these changes after a period of time. Once it was all done, all 1024 keys were attesting.

During import and debouncing (logs added for debug purposes)
Screen Shot 2020-08-27 at 3 02 53 PM

After import complete, all keys attesting
Screen Shot 2020-08-27 at 3 22 20 PM

@rauljordan rauljordan self-assigned this Aug 11, 2020
@rauljordan rauljordan added AccountsV2 Enhancement New feature or request Priority: High High priority item labels Aug 11, 2020
@rauljordan rauljordan marked this pull request as ready for review August 12, 2020 01:02
@rauljordan rauljordan requested a review from a team as a code owner August 12, 2020 01:02
@rauljordan rauljordan added the Blocked Blocked by research or external factors label Aug 12, 2020
@stale
Copy link

stale bot commented Aug 21, 2020

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the Stale There hasn't been any activity here in some time... label Aug 21, 2020
@@ -83,6 +83,7 @@ this command outputs a deposit data string which is required to become a validat
"list of hex string public keys",
Flags: []cli.Flag{
flags.WalletDirFlag,
flags.WalletPasswordFileFlag,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out backup didn't have the wallet password file flag

@rauljordan rauljordan removed the Blocked Blocked by research or external factors label Aug 27, 2020
@rauljordan rauljordan changed the title WIP: Begin Dynamic Rescan of Validating Keys Begin Dynamic Rescan of Validating Keys Aug 27, 2020
@prylabs-bulldozer prylabs-bulldozer bot merged commit ecbab20 into master Aug 31, 2020
@delete-merged-branch delete-merged-branch bot deleted the dynamic-rescan branch August 31, 2020 23:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request Priority: High High priority item Ready For Review A pull request ready for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add validator keys while validator client is running
4 participants