New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update key and db store system to support multi beacon #837

Merged

willscott merged 47 commits into drand:master from Zondax:file-system

Oct 29, 2021

Contributor

emmanuelm41 commented Oct 20, 2021

No description provided.

emmanuelm41 force-pushed the file-system branch 3 times, most recently from e200cd1 to ee4370d Compare

October 21, 2021 16:05

emmanuelm41 marked this pull request as ready for review

October 21, 2021 16:50

emmanuelm41 requested review from nikkolasg and willscott

October 21, 2021 17:15

emmanuelm41 force-pushed the file-system branch from e91d448 to bf25abe Compare

October 22, 2021 02:05

nikkolasg reviewed

View reviewed changes

key/store.go Show resolved Hide resolved

key/store.go Outdated Show resolved Hide resolved

demo/node/node_subprocess.go Show resolved Hide resolved

common/migration/migration.go Outdated Show resolved Hide resolved

cmd/drand-cli/control.go Outdated Show resolved Hide resolved

emmanuelm41 requested a review from nikkolasg

October 22, 2021 16:00

Collaborator

nikkolasg commented Oct 22, 2021

For posterity: the migration plan implemented here is to run the checks automatically at each command so the user doesn't have to run a command explicitely.

nikkolasg reviewed

View reviewed changes

common/migration/migration.go Outdated Show resolved Hide resolved

nikkolasg reviewed

View reviewed changes

common/migration/migration.go Outdated Show resolved Hide resolved

nikkolasg reviewed

View reviewed changes

common/migration/migration.go Outdated Show resolved Hide resolved

nikkolasg reviewed

View reviewed changes

core/config.go Show resolved Hide resolved

nikkolasg reviewed

View reviewed changes

core/drand.go Show resolved Hide resolved

nikkolasg reviewed

View reviewed changes

demo/lib/orchestrator.go Show resolved Hide resolved

nikkolasg reviewed

View reviewed changes

demo/node/node_subprocess.go Show resolved Hide resolved

nikkolasg reviewed

View reviewed changes

fs/fs.go Outdated Show resolved Hide resolved

emmanuelm41 mentioned this pull request

Remove temporary FIXME on code #842

Closed

emmanuelm41 requested a review from nikkolasg

October 22, 2021 17:20

nikkolasg reviewed

View reviewed changes

cmd/drand-cli/daemon.go Outdated Show resolved Hide resolved

emmanuelm41 requested a review from nikkolasg

October 22, 2021 18:02

nikkolasg approved these changes

View reviewed changes

emmanuelm41 added 7 commits

October 22, 2021 18:34


          add new util functions to fs package

b2419c6


          add beacon id to tests

fd43a81


          create package to handle files migration

a682dc0


          redesing part of key store to load all beacons on the file system


          apply multi beacon and migration process to cli commands

f8305a4


          remove useless field on config struct

88919fb


          change docs

87382ad


          fix two typos

570d85a

willscott reviewed

View reviewed changes

common/constants/values.go Outdated Show resolved Hide resolved

common/migration/migration.go Outdated Show resolved Hide resolved

common/migration/migration.go Outdated Show resolved Hide resolved

fs/fs.go Outdated Show resolved Hide resolved

key/store.go Show resolved Hide resolved

common/migration/migration.go Outdated Show resolved Hide resolved

cmd/drand-cli/cli.go Outdated Show resolved Hide resolved

cmd/drand-cli/cli.go Outdated Show resolved Hide resolved

emmanuelm41 added 4 commits

October 22, 2021 20:23


          add unit test for MoveFile

70964d9


          change migration for validation on some commands

e36fdee


          add command to migrate folders manually

948011b


          add tests

d2c58b5

emmanuelm41 requested a review from willscott

October 23, 2021 13:59

emmanuelm41 added 4 commits

October 23, 2021 11:06


          refactor tests to DRY code

14a3233


          fix test

5ce1bf2


          move and/or rename packages

b2c0b2c


          rename some functions

a9f7d70

emmanuelm41 requested a review from yiannisbot

October 25, 2021 15:21

willscott reviewed

View reviewed changes

cmd/drand-cli/cli.go Outdated Show resolved Hide resolved

emmanuelm41 requested a review from willscott

October 25, 2021 17:11

emmanuelm41 added 5 commits

October 25, 2021 15:43


          create a sub level folder fo multi beacon to make rollback possible

afbfed3


          remove migration check for backup beacon cmd

2f69a97


          fix subprocess test file

407d2cf


          copy migrated files instead of moving them to allow rollback

869d09c


          fix migrate test

ba6e0ae

Contributor Author

emmanuelm41 commented Oct 25, 2021

@yiannisbot @willscott now we copy files instead of moving them. The new multi beacon structure after migrating it will be

So a rollback can be performed.

nikkolasg reviewed

View reviewed changes

cmd/drand-cli/cli_test.go Outdated Show resolved Hide resolved

cmd/drand-cli/cli_test.go Show resolved Hide resolved

cmd/drand-cli/cli_test.go Outdated Show resolved Hide resolved

cmd/drand-cli/control.go Outdated Show resolved Hide resolved

core/migration/migration.go Outdated Show resolved Hide resolved

emmanuelm41 added 3 commits

October 26, 2021 16:30


          apply some small changes and fixes

d916394


          fix lint issues

dcc97fe


          fix error generated on prev commit

f61ae99

Member

yiannisbot commented Oct 29, 2021

@yiannisbot @willscott now we copy files instead of moving them. The new multi beacon structure after migrating it will be

So a rollback can be performed.

This approach seems fine, as per our discussion earlier in the week.

Member

yiannisbot commented Oct 29, 2021

@nikkolasg @willscott the changes requested seem to have been resolved now. Can you have another look to see if everything is ok here?🙏

Collaborator

nikkolasg commented Oct 29, 2021

I'm good , LGTM - @willscott ?

willscott approved these changes

View reviewed changes

willscott merged commit a93f2b1 into drand:master

emmanuelm41 deleted the file-system branch

October 29, 2021 18:36

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment