Make DB Backup More Efficient #8543

nisdas · 2021-03-02T15:44:19Z

What type of PR is this?

Bug Fix

What does this PR do? Why is it needed?

Allows a beacon node to take a live backup without killing the main node.
Breaks up the backup process to multiple writes for each key in a bucket, allowing a node
to have a lower memory footprint at the cost of making backups longer. This prevents long running read
transactions which Bolt doesn't handle well.

Which issues(s) does this PR fix?

Resolves #8486 partially

Other notes for review

…eth-sharding into makeBackupBetter

beacon-chain/db/kv/backup_test.go

nisdas added 5 commits March 2, 2021 23:40

checkpoint

b8a16b2

Merge branch 'develop' into makeBackupBetter

2a43697

add test

def0e0a

Merge branch 'makeBackupBetter' of https://github.com/prysmaticlabs/g…

3308ca9

…eth-sharding into makeBackupBetter

Merge branch 'develop' into makeBackupBetter

b1a0d42

nisdas marked this pull request as ready for review March 3, 2021 10:02

nisdas requested a review from a team as a code owner March 3, 2021 10:02

nisdas requested review from 0xKiwi, terencechain and rkapka March 3, 2021 10:02

nisdas added the Ready For Review A pull request ready for code review label Mar 3, 2021

rauljordan reviewed Mar 3, 2021

View reviewed changes

beacon-chain/db/kv/backup_test.go Outdated Show resolved Hide resolved

Update beacon-chain/db/kv/backup_test.go

c4c0c23

rauljordan approved these changes Mar 3, 2021

View reviewed changes

rauljordan added the OK to merge label Mar 3, 2021

prylabs-bulldozer bot merged commit 565d510 into develop Mar 3, 2021

delete-merged-branch bot deleted the makeBackupBetter branch March 3, 2021 15:37

Provide feedback