chanbackup: archive old channel backup files#9232
chanbackup: archive old channel backup files#9232guggero merged 3 commits intolightningnetwork:masterfrom
Conversation
|
Important Review skippedAuto reviews are limited to specific labels. 🏷️ Labels to auto review (1)
Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
Abdulkbk
force-pushed
the
archive-channel-backups
branch
from
bd7fb87 to
71ee590
Compare
Abdulkbk
force-pushed
the
archive-channel-backups
branch
from
71ee590 to
375e0fb
Compare
|
@Abdulkbk, remember to re-request review from reviewers when ready |
|
LGTM 👌 |
Abdulkbk
changed the title
|
Could you do a rebase and push so the CI can run again, wanna check what's going in the logs but they are expired. |
Abdulkbk
force-pushed
the
archive-channel-backups
branch
from
375e0fb to
ade9d50
Compare
|
I've rebased and pushed |
yyforyongyu
added
enhancement
Abdulkbk
force-pushed
the
archive-channel-backups
branch
3 times, most recently
from
bdcf713 to
8cf0088
Compare
|
Hi @yyforyongyu , thank you for the review. I've addressed your feedback and left some comments/questions. |
Abdulkbk
force-pushed
the
archive-channel-backups
branch
from
8cf0088 to
1051ab4
Compare
|
I think this PR is ready for another look :) |
| }, | ||
|
|
||
| // Test with deleteOldBackup set to false - should create | ||
| // archive. |
guggero
left a comment
guggero
left a comment
There was a problem hiding this comment.
Thanks for the feature. Have a couple of improvement proposals, but we're quite close.
chanbackup/backupfile.go
Outdated
| // and not error out. | ||
| _, err = os.Stat(b.fileName) | ||
| if err != nil { | ||
| log.Debugf("Unable to get backup file info: %v", err) |
There was a problem hiding this comment.
If you saw this error message as a user, would you be able to:
- Know what's going on?
- Tell if you need to react to it or not?
I think we should either:
- Test if the error is a "file not found error" (see ) and if it is not log anything.
Line 8 in 70c874b
- Directly use
lnrpc.FileExistsbefore even trying to do anything. - Update the error message to be more informative, perhaps something like "Cannot archive channel backup file, unable to get file info: %v".
There was a problem hiding this comment.
I go with 2, called the lnrpc.FileExists directly and if it returns true we proceed with creating the archive.
chanbackup/backupfile.go
Outdated
| ); err != nil { | ||
| return fmt.Errorf("unable to archive old backup file:"+ | ||
| " %w", err) | ||
| if !b.deleteOldBackup { |
There was a problem hiding this comment.
One more flag to pass into createArchiveFile, then this can just be an early return.
No need to create that many levels of nesting.
There was a problem hiding this comment.
One more flag to pass into createArchiveFile,
I didn't quite understand this part, but now that we're using lnrpc.FileExists and the previous nesting is gone, does this solve the issue?
if !b.noBackupArchive {
// Archive the main backup file if it exists before replacing
// it with a new one.
backupExists := lnrpc.FileExists(b.fileName)
if backupExists {
log.Infof("Archiving old channel backup to %v",
b.archiveDir)
err := createArchiveFile(
b.archiveDir, b.fileName,
)
if err != nil {
return fmt.Errorf("unable to archive old "+
"channel backup file: %w", err)
}
}
}
Abdulkbk
force-pushed
the
archive-channel-backups
branch
2 times, most recently
from
76a3b56 to
55f328c
Compare
Thank you for taking a look @guggero. I have addressed your feedback and added some comments as well. |
chanbackup/backupfile.go
Outdated
|
|
||
| // Archive the main backup file if it exists before replacing it with a | ||
| // new one. | ||
| backupExists := lnrpc.FileExists(b.fileName) |
There was a problem hiding this comment.
nit: the word "backup" is a bit overloaded in this context (does it mean the old channel.backup file? or the archival copy we're going to create)? My suggestion: oldFileExists.
There was a problem hiding this comment.
Updated the comment and now using oldFileExists
chanbackup/backupfile.go
Outdated
| if err != nil { | ||
| return fmt.Errorf("unable to archive old channel "+ | ||
| "backup file: %w", err) | ||
| if !b.noBackupArchive { |
There was a problem hiding this comment.
This is what I had in mind concerning reducing indentation (and having the "happy path" on the un-indented path whenever possible, which helps with readability):
diff --git a/chanbackup/backupfile.go b/chanbackup/backupfile.go
index e44a2005e..b5564b73e 100644
--- a/chanbackup/backupfile.go
+++ b/chanbackup/backupfile.go
@@ -136,22 +136,10 @@ func (b *MultiFile) UpdateAndSwap(newBackup PackedMulti) error {
return fmt.Errorf("unable to close file: %w", err)
}
- if !b.noBackupArchive {
- // Archive the main backup file if it exists before replacing
- // it with a new one.
- backupExists := lnrpc.FileExists(b.fileName)
- if backupExists {
- log.Infof("Archiving old channel backup to %v",
- b.archiveDir)
-
- err := createArchiveFile(
- b.archiveDir, b.fileName,
- )
- if err != nil {
- return fmt.Errorf("unable to archive old "+
- "channel backup file: %w", err)
- }
- }
+ // Make a backup copy of the old backup file before replacing it (if the
+ // user didn't opt out of that).
+ if err := b.maybeCreateArchiveFile(); err != nil {
+ return fmt.Errorf("unable to archive old backup file: %w", err)
}
// Finally, we'll attempt to atomically rename the temporary file to
@@ -185,10 +173,31 @@ func (b *MultiFile) ExtractMulti(keyChain keychain.KeyRing) (*Multi, error) {
return packedMulti.Unpack(keyChain)
}
-// createArchiveFile creates an archive file with a timestamped name in the
+// maybeCreateArchiveFile creates an archive file with a timestamped name in the
// specified archive directory, and copies the contents of the main backup file
-// to the new archive file.
-func createArchiveFile(archiveDir string, fileName string) error {
+// to the new archive file. Unless the user opted out of archiving old backups,
+// or the old backup file doesn't exist.
+func (b *MultiFile) maybeCreateArchiveFile(archiveDir string,
+ fileName string) error {
+
+ // User can skip archiving of old backup files to save disk space.
+ if b.noBackupArchive {
+ log.Debug("Skipping archive of old backup file as configured")
+
+ return nil
+ }
+
+ // Archive the main backup file if it exists before replacing it with a
+ // new one.
+ oldFileExists := lnrpc.FileExists(b.fileName)
+ if !oldFileExists {
+ log.Debug("No old backup file to archive")
+
+ return nil
+ }
+
+ log.Infof("Archiving old channel backup to %v", b.archiveDir)
+
// Generate archive file path with timestamped name.
baseFileName := filepath.Base(fileName)
timestamp := time.Now().Format("2006-01-02-15-04-05")
I will address them. It will be a learning experience for me 👍 |
Abdulkbk
force-pushed
the
archive-channel-backups
branch
from
55f328c to
c415d84
Compare
Abdulkbk
force-pushed
the
archive-channel-backups
branch
from
c415d84 to
c27c16a
Compare
|
Ready for another look @guggero |
In this commit, we first check if a previous backup file exists, if it does we copy it to archive folder before replacing it with a new backup file. We also added a test for archiving chan backups.
In this commit, we add the --no-backup-archive with a default as false. When set to true then previous channel backup file will not be archived but replaced. We also modify TestUpdateAndSwap test to make sure the new behaviour works as expected.
Abdulkbk
force-pushed
the
archive-channel-backups
branch
from
c27c16a to
3bf1548
Compare
guggero
left a comment
guggero
left a comment
There was a problem hiding this comment.
Very nice, tACK, LGTM 🎉
5 participants
Closes #8906
Change Description
From the feature description:
This PR allows for the archiving of channel backups, enabling users to choose whether to permanently delete previous backup files or archive them by moving them to a designated archives folder.
Problem:
LND currently overwrites old channel backup files when creating new ones. This means:
Solution:
We modify the current flow of creating
channel.backup. We start by creating a folder in the same directory where we store thechannel.backupfile (chan-backup-archives). We ensure that thechannel.backupfile is copied to the archive directory and timestamped before finally replacing it with the new file.We set the LND's default behavior to archive old channel backups, but we provide a configuration option that can be passed as an argument or specified in
lnd.confto disable this behavior. This option allows LND to deletechannel.backupfiles instead of archiving them. This can be achieved by settingno-backup-archive=trueinlnd.confor passed as argument:Steps to Test
no-backup-archive=trueis not set inlnd.conf):channel.backupfile is stored. You will find a new folder calledchan-backup-archives, where the backup files are archived.To test disabling this feature
chan-backup-archivesdirectory will not be created (if it does not already exist). If it already exists a new archive will not be created.Pull Request Checklist
Testing
Code Style and Documentation
[skip ci]in the commit message for small changes.📝 Please see our Contribution Guidelines for further guidance.